|
2020-12-09
§
|
| 18:42 |
<cmjohnson@cumin1001> |
START - Cookbook sre.dns.netbox |
[production] |
| 18:41 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 18:17 |
<dzahn@cumin1001> |
conftool action : set/pooled=inactive; selector: name=mw2243.codfw.wmnet |
[production] |
| 18:16 |
<mutante> |
depooling mw2243 (jobrunner) for reimaging (T245757) |
[production] |
| 18:15 |
<dzahn@cumin1001> |
conftool action : set/pooled=no; selector: name=mw2243.codfw.wmnet |
[production] |
| 18:12 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 18:11 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 18:05 |
<mutante> |
mw1281,mw1282,mw1283 shut down for T266164 |
[production] |
| 18:02 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 18:02 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 18:02 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 18:02 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 18:01 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 18:01 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 17:59 |
<clarakosi@deploy1001> |
helmfile [codfw] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'production' . |
[production] |
| 17:58 |
<clarakosi@deploy1001> |
helmfile [eqiad] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'production' . |
[production] |
| 17:57 |
<clarakosi@deploy1001> |
helmfile [staging] Ran 'sync' command on namespace 'changeprop-jobqueue' for release 'staging' . |
[production] |
| 17:49 |
<dzahn@cumin1001> |
conftool action : set/pooled=inactive; selector: name=mw128[1-3].eqiad.wmnet |
[production] |
| 17:24 |
<mutante> |
depooling 3 API appservers in eqiad to physically move to another rack |
[production] |
| 17:23 |
<dzahn@cumin1001> |
conftool action : set/pooled=no; selector: name=mw128[1-3].eqiad.wmnet |
[production] |
| 16:52 |
<mbsantos@deploy1001> |
helmfile [eqiad] Ran 'sync' command on namespace 'mobileapps' for release 'production' . |
[production] |
| 16:49 |
<mbsantos@deploy1001> |
helmfile [codfw] Ran 'sync' command on namespace 'mobileapps' for release 'production' . |
[production] |
| 16:48 |
<mbsantos@deploy1001> |
helmfile [staging] Ran 'sync' command on namespace 'mobileapps' for release 'staging' . |
[production] |
| 16:10 |
<ema> |
deployment-cache-text06: deploy varnish 6.0.0-1wm1 T264398 |
[production] |
| 16:06 |
<moritzm> |
updating mwdebug1003, parse2001, deploy1002, deploy2002 to wikidiff 1.10.0-1~wmf1+buster1 |
[production] |
| 16:05 |
<moritzm> |
importing wikidiff2 1.10.0-1~wmf1+buster1 to component/php72 T250515 |
[production] |
| 15:56 |
<hnowlan@puppetmaster1001> |
conftool action : set/pooled=yes; selector: dc=eqiad,cluster=maps,service=kartotherian,name=maps1001.eqiad.wmnet |
[production] |
| 15:47 |
<hnowlan> |
reimaging restbase2009 after disk replacement |
[production] |
| 15:29 |
<moritzm> |
restarting nginx on htmldump1001 to pick up OpenSSL security updates |
[production] |
| 13:54 |
<godog> |
experiment with rsync.service increased niceness on ms-be2057 - T269337 |
[production] |
| 13:27 |
<jmm@cumin2001> |
END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=99) |
[production] |
| 13:25 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 13:03 |
<XioNoX> |
standardize Private-Peer BGP group on all cr* |
[production] |
| 12:30 |
<hnowlan@puppetmaster1001> |
conftool action : set/pooled=no; selector: dc=eqiad,cluster=maps,service=kartotherian,name=maps1001.eqiad.wmnet |
[production] |
| 12:29 |
<hnowlan@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 12:29 |
<hnowlan@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 12:24 |
<Urbanecm> |
Eu B&C window done |
[production] |
| 12:23 |
<urbanecm@deploy1001> |
Synchronized w/static.php: cfb36023ac873c00e680032999b7c21c2a105132: Remove unsupported arg in MediaWiki::doPostOutputShutdown() call (duration: 01m 02s) |
[production] |
| 12:20 |
<hnowlan@puppetmaster1001> |
conftool action : set/pooled=no; selector: dc=codfw,cluster=restbase,service=restbase,name=restbase2009.codfw.wmnet |
[production] |
| 12:16 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: 3414289c8c7272185e30cacc3df5d5dbc719219d: Add extended-confirmed group and restriction level for bgwiki (T269709) (duration: 01m 19s) |
[production] |
| 11:06 |
<godog> |
reboot ms-be1019 / ms-be1020 - T268435 |
[production] |
| 10:56 |
<godog> |
change librenms alerts and transport groups to use alertmanager - T267018 |
[production] |
| 10:45 |
<moritzm> |
installing openssl updates on Buster |
[production] |
| 09:24 |
<jbond42> |
make message mandatory for disable-puppet |
[production] |
| 09:03 |
<godog> |
swift codfw-prod: add ms-be20[58-61] - T269337 |
[production] |
| 01:07 |
<jforrester@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: Explicitly set wgAbuseFilterAflFilterMigrationStage ahead of train roll-out T269712 (duration: 01m 03s) |
[production] |
| 00:53 |
<urbanecm@deploy1001> |
Synchronized php-1.36.0-wmf.20/vendor/: 3278ffd107888757c4620383160a6d5fa67d05b5: Bump wikimedia/parsoid to v0.13.0-a19 (T269685) (duration: 01m 16s) |
[production] |