|
2020-12-03
ยง
|
| 23:47 |
<ejegg> |
adjusted timings for donations queue consumer and thank you mailer |
[production] |
| 23:44 |
<ryankemper@cumin2001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 23:39 |
<ryankemper@cumin2001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 22:50 |
<ejegg> |
updated standalone SmashPig IPN listener from 63dffcb11f to 3029b07004 |
[production] |
| 22:49 |
<razzi@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 22:49 |
<razzi@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 22:15 |
<shdubsh> |
restart elasticsearch on logstash1010 - gc issues |
[production] |
| 22:15 |
<ryankemper@cumin2001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 21:46 |
<twentyafterfour@deploy1001> |
rebuilt and synchronized wikiversions files: rollback to wmf.18 |
[production] |
| 21:39 |
<twentyafterfour> |
rolling back wmf.20 due to T269396 refs T263186 |
[production] |
| 21:37 |
<dzahn@cumin1001> |
conftool action : set/pooled=no; selector: name=parse2001.codfw.wmnet |
[production] |
| 21:26 |
<robh@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 21:26 |
<robh@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 21:25 |
<robh@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 21:23 |
<robh@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 21:23 |
<robh@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 21:21 |
<robh@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 21:21 |
<robh@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 21:21 |
<robh@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 21:21 |
<robh@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 21:21 |
<robh@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 21:05 |
<bstorm> |
running maintain-dbusers harvest-replicas to populate the user accounts on new wikireplicas servers |
[production] |
| 20:46 |
<ryankemper@cumin2001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 20:44 |
<robh@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 20:43 |
<shdubsh> |
kill slapd on serpens and restart it |
[production] |
| 20:42 |
<robh@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 20:40 |
<robh@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 20:39 |
<robh@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 20:38 |
<robh@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 20:38 |
<robh@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 20:38 |
<robh@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 20:38 |
<robh@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 20:28 |
<ryankemper@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 20:26 |
<ryankemper@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 20:12 |
<robh@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 20:10 |
<robh@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 20:08 |
<ryankemper> |
T269204 Re-imaging `wdqs2004` to upgrade it to buster: `sudo -i wmf-auto-reimage-host --conftool -p T269204 wdqs2004.codfw.wmnet` |
[production] |
| 20:03 |
<hashar@deploy1001> |
rebuilt and synchronized wikiversions files: all wikis to 1.36.0-wmf.20 |
[production] |
| 19:58 |
<milimetric@deploy1001> |
Finished deploy [analytics/aqs/deploy@95d6432]: (no justification provided) (duration: 00m 23s) |
[production] |
| 19:58 |
<milimetric@deploy1001> |
Started deploy [analytics/aqs/deploy@95d6432]: (no justification provided) |
[production] |
| 19:57 |
<milimetric@deploy1001> |
Finished deploy [analytics/aqs/deploy@95d6432]: (no justification provided) (duration: 00m 19s) |
[production] |
| 19:57 |
<shdubsh> |
restart logstash kafka in codfw - java updates |
[production] |
| 19:57 |
<milimetric@deploy1001> |
Started deploy [analytics/aqs/deploy@95d6432]: (no justification provided) |
[production] |
| 19:57 |
<hashar@deploy1001> |
Synchronized php-1.36.0-wmf.20/extensions/AbuseFilter/includes/FilterLookup.php: Use 'default' as default group when reading filters from history - T269314 (duration: 01m 05s) |
[production] |
| 19:56 |
<milimetric@deploy1001> |
Started restart [analytics/aqs/deploy@95d6432]: (no justification provided) |
[production] |
| 19:56 |
<milimetric@deploy1001> |
Started restart [analytics/aqs/deploy@95d6432]: (no justification provided) |
[production] |
| 19:55 |
<milimetric@deploy1001> |
Started restart [analytics/aqs/deploy@95d6432]: (no justification provided) |
[production] |
| 19:55 |
<milimetric@deploy1001> |
Started restart [analytics/aqs/deploy@95d6432]: (no justification provided) |
[production] |
| 19:53 |
<pt1979@cumin2001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 19:51 |
<pt1979@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |