|
2020-10-19
§
|
| 12:15 |
<marostegui> |
Deploy schema change on smnwiki T265321 T264900 |
[production] |
| 12:14 |
<urbanecm@deploy1001> |
rebuilt and synchronized wikiversions files: Creating smnwiki (T264859) |
[production] |
| 12:12 |
<urbanecm@deploy1001> |
Synchronized dblists: Creating smnwiki (T264859) (duration: 00m 55s) |
[production] |
| 12:11 |
<urbanecm@deploy1001> |
Synchronized wmf-config/db-codfw.php: Creating smnwiki (T264859) (duration: 00m 55s) |
[production] |
| 12:10 |
<urbanecm@deploy1001> |
Synchronized wmf-config/db-eqiad.php: Creating smnwiki (T264859) (duration: 00m 56s) |
[production] |
| 11:51 |
<moritzm> |
updating idp-test1001 to CAS 6.2.4 |
[production] |
| 11:46 |
<moritzm> |
updating idp-test2001 to CAS 6.2.4 |
[production] |
| 11:43 |
<Urbanecm> |
End of `[urbanecm@mwmaint2001 ~/updateVarDumps/script]$ while read wiki; do echo "Processing $wiki"; mwscript extensions/AbuseFilter/maintenance/updateVarDumps.php --wiki=$wiki --print-orphaned-records-to=/tmp/urbanecm/$wiki-orphaned.log --progress-markers > $wiki.log; done < ../small-group2.dblist` # T246539 # small-group2.dblist is wikis from small.dblist that are also in group2.dblist |
[production] |
| 11:42 |
<Urbanecm> |
End of `mwscript extensions/AbuseFilter/maintenance/updateVarDumps.php --wiki=enwikisource --print-orphaned-records-to=/tmp/urbanecm/enwikisource-orphaned.log --progress-markers` (T246539) |
[production] |
| 11:40 |
<Urbanecm> |
[urbanecm@mwmaint2001 ~/updateVarDumps/script]$ while read wiki; do echo "Processing $wiki"; mwscript extensions/AbuseFilter/maintenance/updateVarDumps.php --wiki=$wiki --print-orphaned-records-to=/tmp/urbanecm/$wiki-orphaned.log --progress-markers > $wiki.log; done < ../small-group2.dblist # T246539 # small-group2.dblist is wikis from small.dblist that are also in group2.dblist |
[production] |
| 11:31 |
<jmm@cumin2001> |
END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=1) |
[production] |
| 11:24 |
<Urbanecm> |
EU B&C window done |
[production] |
| 11:24 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: ce92c9814bf9c12cab1a9592dfb32f935d255d93: Restore bureaucrat abilities at uzwiki (T265746) (duration: 00m 56s) |
[production] |
| 11:20 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: 26b97261f2b9d1991ea08fe32b6007ba6fe5088f: Disable EditorJourney (UnderstandingFirstDay) (T252391) (duration: 01m 10s) |
[production] |
| 11:17 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 11:13 |
<Urbanecm> |
Manually run `mwscript extensions/AbuseFilter/maintenance/updateVarDumps.php --wiki=$wiki --print-orphaned-records-to=/tmp/urbanecm/$wiki-orphaned.log --progress-markers > $wiki.log` for several small group2 wikis (T246539) |
[production] |
| 10:57 |
<Urbanecm> |
Start `mwscript extensions/AbuseFilter/maintenance/updateVarDumps.php --wiki=enwikisource --print-orphaned-records-to=/tmp/urbanecm/enwikisource-orphaned.log --progress-markers` in a tmux session named updateVarDumps at mwmaint2001 (T246539) |
[production] |
| 10:53 |
<Urbanecm> |
[urbanecm@mwmaint2001 ~/updateVarDumps/script]$ mwscript extensions/AbuseFilter/maintenance/updateVarDumps.php --wiki=jawikivoyage --print-orphaned-records-to=- --progress-markers # T246539 |
[production] |
| 09:09 |
<gehel@cumin2001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 08:40 |
<jayme> |
updated helm to 2.16.12-1 on deploy*,chartmuseum*,contint* |
[production] |
| 08:37 |
<godog> |
upgrade rsyslog to 8.2008.0-1~bpo10+1 on centrallog2001 - T259780 |
[production] |
| 08:31 |
<godog> |
swift codfw-prod: bump object weight for ms-be2057 - T261633 |
[production] |
| 08:26 |
<jayme> |
updated helm to 2.16.12-1 on deploy2001 |
[production] |
| 08:24 |
<jayme> |
imported helm 2.16.12-1 to buster-wikimedia stretch-wikimedia jessie-wikimedia - T263616 |
[production] |
| 08:01 |
<godog> |
re-enable compaction for prometheus[12]003 - T261281 |
[production] |
| 07:53 |
<gehel@cumin2001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 07:36 |
<elukey@cumin1001> |
END (FAIL) - Cookbook sre.hadoop.init-hadoop-workers (exit_code=99) |
[production] |
| 07:36 |
<elukey@cumin1001> |
START - Cookbook sre.hadoop.init-hadoop-workers |
[production] |
| 07:16 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db2125 ', diff saved to https://phabricator.wikimedia.org/P13022 and previous config saved to /var/cache/conftool/dbconfig/20201019-071614-marostegui.json |
[production] |
| 06:46 |
<elukey@deploy1001> |
Finished deploy [analytics/turnilo/deploy@334627e]: Upgrade to 1.27 (duration: 00m 10s) |
[production] |
| 06:45 |
<elukey@deploy1001> |
Started deploy [analytics/turnilo/deploy@334627e]: Upgrade to 1.27 |
[production] |
|
2020-10-16
§
|
| 21:46 |
<pt1979@cumin2001> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 21:43 |
<pt1979@cumin2001> |
START - Cookbook sre.dns.netbox |
[production] |
| 20:27 |
<robh@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 20:25 |
<robh@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 19:39 |
<robh@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 19:37 |
<robh@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 17:43 |
<thcipriani> |
restarting gerrit due to gc thrashing |
[production] |
| 16:25 |
<andrew@deploy1001> |
Finished deploy [horizon/deploy@89b308c]: prevent creation of VMs with non-ceph flavors (duration: 04m 08s) |
[production] |
| 16:21 |
<andrew@deploy1001> |
Started deploy [horizon/deploy@89b308c]: prevent creation of VMs with non-ceph flavors |
[production] |
| 15:36 |
<ayounsi@cumin1001> |
END (PASS) - Cookbook sre.network.cf (exit_code=0) |
[production] |
| 15:36 |
<ayounsi@cumin1001> |
START - Cookbook sre.network.cf |
[production] |
| 15:11 |
<bblack@cumin1001> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) |
[production] |
| 15:01 |
<bblack@cumin1001> |
START - Cookbook sre.hosts.decommission |
[production] |
| 13:41 |
<effie> |
pooling mw2279.codfw.wmnet T264698 |
[production] |
| 12:11 |
<jiji@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 12:09 |
<jiji@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 10:35 |
<reedy@deploy1001> |
Synchronized php-1.36.0-wmf.13/extensions/ProofreadPage/: Revert excessive escaping T265571 (duration: 01m 12s) |
[production] |
| 09:23 |
<ema> |
text@esams (except for cp3050/cp3052): upgrade varnish to 6.0.6-1wm2, restart varnishkafka instances T264074 |
[production] |