|
2020-12-05
§
|
| 04:44 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 04:44 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 04:28 |
<ryankemper@cumin2001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 04:26 |
<ryankemper@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 04:25 |
<ryankemper@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 04:23 |
<ryankemper@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 04:23 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 04:22 |
<ryankemper@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 04:21 |
<ryankemper@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 04:21 |
<ryankemper@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 04:05 |
<ryankemper> |
T269204 reimaging the following instances to debian buster (one each from `[public, internal] x [eqiad, codfw]`): `wdqs1005`, `wdqs2002`, `wdqs1008`, `wdqs2005` |
[production] |
| 03:49 |
<ryankemper@cumin2001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 03:49 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 03:49 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 03:44 |
<ryankemper@cumin2001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 03:43 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 03:43 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 02:13 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 02:12 |
<ryankemper@cumin1001> |
END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) |
[production] |
| 02:06 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-reload (exit_code=0) |
[production] |
| 01:58 |
<ryankemper@cumin2001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 01:09 |
<eileen> |
civicrm revision changed from 5fa107d32a to b0ffb87c5d, config revision is ffe0a99133 |
[production] |
| 00:45 |
<ryankemper@cumin2001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 00:45 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 00:45 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 00:42 |
<bd808> |
`kubectl delete po renderer-794886b9cd-9nc6c -n prod` after seeing lots of listen queue full errors in the pod logs. |
[paws] |
| 00:40 |
<Urbanecm> |
End of mwscript extensions/AbuseFilter/maintenance/updateVarDumps.php --wiki=$wiki --print-orphaned-records-to=/tmp/urbanecm/$wiki-orphaned.log --progress-markers > $wiki.log in a tmux at mwmaint1002 (wiki=eswiki; T246539) |
[production] |
| 00:35 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-reload |
[production] |
| 00:35 |
<andrewbogott> |
moving cloudvirt1023 back into maintenance because T269467 continues to puzzle |
[admin] |
| 00:35 |
<ryankemper@cumin1001> |
END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) |
[production] |
| 00:34 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 00:32 |
<ryankemper@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 00:30 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 00:28 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 00:27 |
<ryankemper@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 00:27 |
<ryankemper@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 00:26 |
<ryankemper@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 00:17 |
<Urbanecm> |
deploy1001 stagging dir is DIRTY: /srv/mediawiki-staging (master u+1): last commit bce412514eadaa47dbede56c4b4918da492443ce, author Mukunda Modell (cc twentyafterfour) |
[production] |
| 00:09 |
<ryankemper> |
T269204 reimaging the following instances to debian buster: `wdqs1004`, `wdqs2001`, `wdqs1003` |
[production] |
| 00:05 |
<James_F> |
Docker: [quibble*] Switch from composer-php70 to use composer-scratch |
[releng] |
|
2020-12-04
§
|
| 23:12 |
<James_F> |
Zuul: [mediawiki/extensions/IPInfo] Re-enable phan, move to prod section T266136 |
[releng] |
| 22:33 |
<andrewbogott> |
moving cloudvirt1023 back into the ceph aggregate; it doesn't need upgrades after all T269467 |
[admin] |
| 22:23 |
<andrewbogott> |
moving cloudvirt1023 out of the ceph aggregate and into maintenance for T269467 |
[admin] |
| 22:19 |
<andrewbogott> |
paysing mwoffliner3 for migration do a different hypervisor |
[mwoffliner] |
| 21:24 |
<James_F> |
Docker: Building composer-package images using composer-scratch, hopefully successfully this time. |
[releng] |
| 21:08 |
<James_F> |
Docker: Building composer-package images using composer-scratch |
[releng] |
| 21:06 |
<andrewbogott> |
putting cloudvirt1025 and 1026 back into service because I'm pretty sure they're fixed. T269313 |
[admin] |
| 20:16 |
<James_F> |
Docker: Fix composer-scratch images to use compose v1.10.17. |
[releng] |
| 19:45 |
<James_F> |
Docker: Pushing composer-phpN images that use composer-scratch |
[releng] |
| 19:34 |
<James_F> |
Zuul: Add experimental PHP 8.0 jobs for all the things. T250298 |
[releng] |