|
2020-12-10
§
|
| 10:33 |
<jbond@cumin1001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 10:33 |
<jbond@cumin1001> |
END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=99) |
[production] |
| 10:33 |
<jbond@cumin1001> |
START - Cookbook sre.hosts.reboot-single |
[production] |
| 10:32 |
<elukey@cumin1001> |
START - Cookbook sre.hosts.decommission |
[production] |
| 10:29 |
<volans> |
upgraded spicearack to 0.0.46 on cumin[12]001 |
[production] |
| 10:16 |
<volans> |
uploaded spicerack_0.0.46 to apt.wikimedia.org buster-wikimedia |
[production] |
| 09:57 |
<ema> |
A:cp rolling ats-{tls,backend}-restart for openssl upgrades (CVE-2020-1971) |
[production] |
| 09:28 |
<ema> |
cp3054: downgrade varnish to 6.0.0-1wm1 T264398 |
[production] |
| 09:28 |
<effie> |
disable puppet on all hosts running nutcracker for 647204 - T265643 |
[production] |
| 09:26 |
<effie> |
disable puppet on all mw* hosts for 647204 - T265643 |
[production] |
| 09:11 |
<effie> |
disable puppet on all hosts running redis - T265643 |
[production] |
| 08:22 |
<godog> |
swift codfw-prod: more weight to ms-be20[58-61] - T269337 |
[production] |
| 06:38 |
<kart_> |
Upgraded Apertium to 2020-12-09-115733-production |
[production] |
| 06:35 |
<kartik@deploy1001> |
helmfile [eqiad] Ran 'sync' command on namespace 'apertium' for release 'production' . |
[production] |
| 06:35 |
<kartik@deploy1001> |
helmfile [eqiad] Ran 'sync' command on namespace 'apertium' for release 'plain' . |
[production] |
| 06:30 |
<kartik@deploy1001> |
helmfile [codfw] Ran 'sync' command on namespace 'apertium' for release 'plain' . |
[production] |
| 06:30 |
<kartik@deploy1001> |
helmfile [codfw] Ran 'sync' command on namespace 'apertium' for release 'production' . |
[production] |
| 06:27 |
<kartik@deploy1001> |
helmfile [staging] Ran 'sync' command on namespace 'apertium' for release 'staging' . |
[production] |
| 00:42 |
<ejegg> |
updated payments-wiki from 756c2f7ce0 to df80a99b40 |
[production] |
| 00:26 |
<robh> |
cr2-eqsin bad fan being swapped via T267544 |
[production] |
|
2020-12-09
§
|
| 23:21 |
<mutante> |
repooling parse2001 after buster reimage - T268524 |
[production] |
| 23:19 |
<dzahn@cumin1001> |
conftool action : set/pooled=no; selector: name=mw2243.codfw.wmnet |
[production] |
| 23:16 |
<mutante> |
repooling parse2001 after buster reimage - T245757 |
[production] |
| 23:15 |
<dzahn@cumin1001> |
conftool action : set/pooled=yes; selector: name=parse2001.codfw.wmnet |
[production] |
| 23:04 |
<mutante> |
zero.wikimedia.beta.wmflabs.org removed from beta_sites (deployment-prep) T187716 |
[production] |
| 22:32 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 22:30 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 21:40 |
<bstorm> |
shutting down labstore1006 for maintenance T268285 |
[production] |
| 20:27 |
<mutante> |
mw1281,mw1282,mw1283 - scap pull |
[production] |
| 20:26 |
<mutante> |
repooling mw1281,mw1282,mw1283 - now in rack A8 |
[production] |
| 20:26 |
<dzahn@cumin1001> |
conftool action : set/pooled=yes; selector: name=mw128[1-3].eqiad.wmnet |
[production] |
| 20:22 |
<dzahn@cumin1001> |
conftool action : set/pooled=no; selector: name=mw1283.eqiad.wmnet |
[production] |
| 20:22 |
<dzahn@cumin1001> |
conftool action : set/pooled=no; selector: name=mw1282.eqiad.wmnet |
[production] |
| 20:21 |
<dzahn@cumin1001> |
conftool action : set/pooled=no; selector: name=mw1281.eqiad.wmnet |
[production] |
| 20:18 |
<twentyafterfour> |
wmf.21 looks good on group1 wikis. Still seeing T269603 but not at an increased rate. (refs T264801) |
[production] |
| 20:13 |
<twentyafterfour@deploy1001> |
Synchronized php: group1 wikis to 1.36.0-wmf.21 (duration: 01m 02s) |
[production] |
| 20:12 |
<twentyafterfour@deploy1001> |
rebuilt and synchronized wikiversions files: group1 wikis to 1.36.0-wmf.21 |
[production] |
| 19:38 |
<urbanecm@deploy1001> |
Synchronized wmf-config/InitialiseSettings.php: ce01bbe7b05eda8065fc57c865a69370e8aae797: Enable ArticlePlaceholder at papwiki (T223693) (duration: 01m 02s) |
[production] |
| 19:17 |
<twentyafterfour@deploy1001> |
Synchronized php-1.36.0-wmf.21/includes/page/Article.php: deploy 0d99fe6d54 Article::view - remove the old subtitle from doOutputFromParserCache. Bug: T269727 (duration: 01m 04s) |
[production] |
| 18:59 |
<mutante> |
testreduce1001 - installed make |
[production] |
| 18:48 |
<cmjohnson@cumin1001> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 18:43 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 18:42 |
<cmjohnson@cumin1001> |
START - Cookbook sre.dns.netbox |
[production] |
| 18:41 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 18:17 |
<dzahn@cumin1001> |
conftool action : set/pooled=inactive; selector: name=mw2243.codfw.wmnet |
[production] |
| 18:16 |
<mutante> |
depooling mw2243 (jobrunner) for reimaging (T245757) |
[production] |
| 18:15 |
<dzahn@cumin1001> |
conftool action : set/pooled=no; selector: name=mw2243.codfw.wmnet |
[production] |
| 18:12 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 18:11 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 18:05 |
<mutante> |
mw1281,mw1282,mw1283 shut down for T266164 |
[production] |