|
2020-07-31
§
|
| 07:51 |
<moritzm> |
updating lilypond on mw* servers |
[production] |
| 07:50 |
<moritzm> |
uploaded lilypond 2.19.81+really-2.18.2-13~bpo9+1+wmf1 to stretch-wikimedia T256877 |
[production] |
| 07:07 |
<elukey> |
stop mysql replication on db1108; update port config for mysql instances and restart them; restart replication on instances |
[production] |
| 06:32 |
<elukey> |
roll restart of druid brokers on druid100[4-8] to pick up new changes |
[production] |
| 06:26 |
<jmm@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 06:26 |
<jmm@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 05:59 |
<moritzm> |
installing qemu updates on stretch |
[production] |
| 04:56 |
<andrew@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 04:54 |
<andrew@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 04:22 |
<andrew@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 04:20 |
<andrew@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 03:55 |
<andrew@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 03:53 |
<andrew@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 03:14 |
<andrew@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 03:12 |
<andrew@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 02:57 |
<andrew@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 02:55 |
<andrew@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 00:47 |
<tstarling@deploy1001> |
Synchronized wmf-config/CommonSettings.php: disable lilypond execution again (duration: 01m 10s) |
[production] |
| 00:07 |
<catrope@deploy1001> |
Synchronized php-1.36.0-wmf.2/extensions/Echo/modules/mobile/notificationsFilterOverlay.js: T258954 (duration: 01m 06s) |
[production] |
| 00:06 |
<catrope@deploy1001> |
Synchronized php-1.36.0-wmf.1/extensions/Echo/modules/mobile/notificationsFilterOverlay.js: T258954 (duration: 01m 10s) |
[production] |
| 00:00 |
<dzahn@cumin1001> |
conftool action : set/pooled=yes; selector: name=wtp2015.codfw.wmnet |
[production] |
|
2020-07-30
§
|
| 23:56 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 23:56 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 22:59 |
<dzahn@cumin1001> |
conftool action : set/pooled=yes; selector: name=wtp2014.codfw.wmnet |
[production] |
| 22:34 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 22:34 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 22:27 |
<dzahn@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 22:27 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 22:12 |
<brennen@deploy1001> |
rebuilt and synchronized wikiversions files: Revert group2 wikis to 1.36.0-wmf.1 |
[production] |
| 21:52 |
<brennen@deploy1001> |
rebuilt and synchronized wikiversions files: all wikis to 1.36.0-wmf.2 |
[production] |
| 21:52 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 21:51 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 21:51 |
<dzahn@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 21:51 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 21:45 |
<dzahn@cumin1001> |
conftool action : set/pooled=yes; selector: name=wtp2013.codfw.wmnet |
[production] |
| 21:41 |
<mutante> |
revoking and resigning puppet cert for xhgui2001.codfw.wmnet T259206 |
[production] |
| 21:40 |
<catrope@deploy1001> |
Synchronized php-1.36.0-wmf.2/extensions/GrowthExperiments/: T258609 (duration: 01m 06s) |
[production] |
| 21:40 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 21:40 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 21:39 |
<catrope@deploy1001> |
Synchronized php-1.36.0-wmf.2/skins/MinervaNeue/: T258939 (duration: 01m 08s) |
[production] |
| 21:30 |
<mutante> |
reinstalling xhgui2001 |
[production] |
| 21:27 |
<dzahn@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 21:27 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 21:27 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 21:27 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 21:10 |
<ryankemper@deploy1001> |
Finished deploy [wdqs/wdqs@e797cf0]: 0.3.42 (duration: 24m 41s) |
[production] |
| 20:45 |
<dzahn@cumin1001> |
conftool action : set/pooled=yes; selector: name=wtp2012.codfw.wmnet |
[production] |
| 20:45 |
<ryankemper@deploy1001> |
Started deploy [wdqs/wdqs@e797cf0]: 0.3.42 |
[production] |
| 20:23 |
<dzahn@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 20:23 |
<dzahn@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |