|
2020-12-09
§
|
| 18:01 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 17:59 |
<clarakosi@deploy1001> |
helmfile [codfw] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'production' . |
[production] |
| 17:58 |
<clarakosi@deploy1001> |
helmfile [eqiad] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'production' . |
[production] |
| 17:57 |
<clarakosi@deploy1001> |
helmfile [staging] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'staging' . |
[production] |
| 17:49 |
<dzahn@cumin1001> |
conftool action : set/pooled=inactive; selector: name=mw128[1-3].eqiad.wmnet |
[production] |
| 17:24 |
<mutante> |
depooling 3 API appservers in eqiad to physically move to another rack |
[production] |
| 17:23 |
<dzahn@cumin1001> |
conftool action : set/pooled=no; selector: name=mw128[1-3].eqiad.wmnet |
[production] |
| 16:52 |
<mbsantos@deploy1001> |
helmfile [eqiad] Ran 'sync' command on namespace 'mobileapps' for release 'production' . |
[production] |
| 16:49 |
<mbsantos@deploy1001> |
helmfile [codfw] Ran 'sync' command on namespace 'mobileapps' for release 'production' . |
[production] |
| 16:48 |
<mbsantos@deploy1001> |
helmfile [staging] Ran 'sync' command on namespace 'mobileapps' for release 'staging' . |
[production] |
| 16:10 |
<ema> |
deployment-cache-text06: deploy varnish 6.0.0-1wm1 T264398 |
[production] |
| 16:06 |
<moritzm> |
updating mwdebug1003, parse2001, deploy1002, deploy2002 to wikidiff 1.10.0-1~wmf1+buster1 |
[production] |
| 16:05 |
<moritzm> |
importing wikidiff2 1.10.0-1~wmf1+buster1 to component/php72 T250515 |
[production] |
| 15:56 |
<hnowlan@puppetmaster1001> |
conftool action : set/pooled=yes; selector: dc=eqiad,cluster=maps,service=kartotherian,name=maps1001.eqiad.wmnet |
[production] |
| 15:47 |
<hnowlan> |
reimaging restbase2009 after disk replacement |
[production] |
| 15:29 |
<moritzm> |
restarting nginx on htmldump1001 to pick up OpenSSL security updates |
[production] |
| 13:54 |
<godog> |
experiment with rsync.service increased niceness on ms-be2057 - T269337 |
[production] |
| 13:27 |
<jmm@cumin2001> |
END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=99) |
[production] |
| 13:25 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 13:03 |
<XioNoX> |
standardize Private-Peer BGP group on all cr* |
[production] |
| 12:30 |
<hnowlan@puppetmaster1001> |
conftool action : set/pooled=no; selector: dc=eqiad,cluster=maps,service=kartotherian,name=maps1001.eqiad.wmnet |
[production] |
| 12:29 |
<hnowlan@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 12:29 |
<hnowlan@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 12:24 |
<Urbanecm> |
Eu B&C window done |
[production] |
| 12:23 |
<urbanecm@deploy1001> |
Synchronized w/static.php: cfb36023ac873c00e680032999b7c21c2a105132: Remove unsupported arg in MediaWiki::doPostOutputShutdown() call (duration: 01m 02s) |
[production] |
| 12:20 |
<hnowlan@puppetmaster1001> |
conftool action : set/pooled=no; selector: dc=codfw,cluster=restbase,service=restbase,name=restbase2009.codfw.wmnet |
[production] |
| 12:16 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: 3414289c8c7272185e30cacc3df5d5dbc719219d: Add extended-confirmed group and restriction level for bgwiki (T269709) (duration: 01m 19s) |
[production] |
| 11:06 |
<godog> |
reboot ms-be1019 / ms-be1020 - T268435 |
[production] |
| 10:56 |
<godog> |
change librenms alerts and transport groups to use alertmanager - T267018 |
[production] |
| 10:45 |
<moritzm> |
installing openssl updates on Buster |
[production] |
| 09:24 |
<jbond42> |
make message mandatory for disable-puppet |
[production] |
| 09:03 |
<godog> |
swift codfw-prod: add ms-be20[58-61] - T269337 |
[production] |
| 01:07 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Explicitly set wgAbuseFilterAflFilterMigrationStage ahead of train roll-out T269712 (duration: 01m 03s) |
[production] |
| 00:53 |
<urbanecm@deploy1001> |
Synchronized php-1.36.0-wmf.20/vendor/: 3278ffd107888757c4620383160a6d5fa67d05b5: Bump wikimedia/parsoid to v0.13.0-a19 (T269685) (duration: 01m 16s) |
[production] |
|
2020-12-08
§
|
| 23:24 |
<mholloway-shell@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: WikimediaEvents: Enable SessionTick on group0 T248987 (duration: 02m 00s) |
[production] |
| 22:46 |
<razzi@cumin1001> |
END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) |
[production] |
| 22:17 |
<twentyafterfour@deploy1001> |
rebuilt and synchronized wikiversions files: group0 wikis to 1.36.0-wmf.21 |
[production] |
| 22:10 |
<twentyafterfour@deploy1001> |
Pruned MediaWiki: 1.36.0-wmf.18 (duration: 02m 00s) |
[production] |
| 21:58 |
<twentyafterfour@deploy1001> |
Pruned MediaWiki: 1.36.0-wmf.16 (duration: 02m 43s) |
[production] |
| 21:51 |
<razzi@cumin1001> |
START - Cookbook sre.ganeti.makevm |
[production] |
| 21:36 |
<pt1979@cumin2001> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 21:28 |
<pt1979@cumin2001> |
START - Cookbook sre.dns.netbox |
[production] |
| 21:25 |
<reedy@deploy1001> |
Pruned MediaWiki: 1.36.0-wmf.14 [keeping static files] (duration: 02m 44s) |
[production] |
| 21:21 |
<reedy@deploy1001> |
Pruned MediaWiki: 1.36.0-wmf.13 [keeping static files] (duration: 03m 59s) |
[production] |
| 21:05 |
<twentyafterfour@deploy1001> |
Finished scap: testwikis wikis to 1.36.0-wmf.21 (duration: 39m 02s) |
[production] |
| 20:41 |
<ejegg> |
updated standalone SmashPig listener from e3103b96ca to 5a69abd40f |
[production] |
| 20:28 |
<twentyafterfour@deploy1001> |
Started scap: testwikis wikis to 1.36.0-wmf.21 |
[production] |
| 20:16 |
<twentyafterfour> |
syncing new branch 1.376.0-wmf.21 |
[production] |
| 18:47 |
<mbsantos@deploy1001> |
helmfile [eqiad] Ran 'sync' command on namespace 'wikifeeds' for release 'production' . |
[production] |
| 18:44 |
<mbsantos@deploy1001> |
helmfile [codfw] Ran 'sync' command on namespace 'wikifeeds' for release 'production' . |
[production] |