|
2020-12-06
§
|
| 19:15 |
<andrewbogott> |
shutting down all VMs in preparation for project removal https://wikitech.wikimedia.org/wiki/News/Cloud_VPS_2020_Purge#puppet |
[puppet] |
| 19:14 |
<andrewbogott> |
shutting down all VMs in preparation for project removal https://wikitech.wikimedia.org/wiki/News/Cloud_VPS_2020_Purge#ogvjs-integration |
[ogvjs-integration] |
| 19:13 |
<andrewbogott> |
shutting down all VMs in preparation for project removal https://wikitech.wikimedia.org/wiki/News/Cloud_VPS_2020_Purge#meza |
[meza] |
| 19:12 |
<andrewbogott> |
shutting down all VMs in preparation for project removal https://wikitech.wikimedia.org/wiki/News/Cloud_VPS_2020_Purge#lta-tracker |
[lta-tracker] |
| 19:11 |
<andrewbogott> |
shutting down all VMs in preparation for project removal https://wikitech.wikimedia.org/wiki/News/Cloud_VPS_2020_Purge#lizenzhinweisgenerator |
[lizenzhinweisgenerator] |
| 19:10 |
<andrewbogott> |
shutting down all VMs in preparation for project removal https://wikitech.wikimedia.org/wiki/News/Cloud_VPS_2020_Purge#ign2commons |
[ign2commons] |
| 19:10 |
<andrewbogott> |
shutting down all VMs in preparation for project removal https://wikitech.wikimedia.org/wiki/News/Cloud_VPS_2020_Purge#finding-glams |
[finding-glams] |
| 19:08 |
<andrewbogott> |
shutting down all VMs in preparation for project removal https://wikitech.wikimedia.org/wiki/News/Cloud_VPS_2020_Purge#fastcci |
[fastcci] |
| 19:07 |
<andrewbogott> |
shutting down all VMs in preparation for project removal https://wikitech.wikimedia.org/wiki/News/Cloud_VPS_2020_Purge#commons-corruption-checker |
[commons-corruption-checker] |
| 19:06 |
<andrewbogott> |
shutting down all VMs in preparation for project removal https://wikitech.wikimedia.org/wiki/News/Cloud_VPS_2020_Purge#blog |
[blog] |
| 19:04 |
<andrewbogott> |
shutting down all VMs in preparation for project removal https://wikitech.wikimedia.org/wiki/News/Cloud_VPS_2020_Purge#asyncwiki |
[asyncwiki] |
| 18:58 |
<andrewbogott> |
deleting project https://wikitech.wikimedia.org/wiki/News/Cloud_VPS_2020_Purge#DELETE_discovery-stats |
[discovery-stats] |
|
2020-12-05
§
|
| 16:25 |
<godog> |
swift disable sdg1 on ms-be1054 |
[production] |
| 10:12 |
<legoktm> |
deploying https://gerrit.wikimedia.org/r/645504 |
[releng] |
| 09:19 |
<legoktm> |
deploying https://gerrit.wikimedia.org/r/645494 |
[releng] |
| 09:10 |
<legoktm> |
deploying https://gerrit.wikimedia.org/r/645491 |
[releng] |
| 06:12 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 06:11 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 06:09 |
<ryankemper@cumin2001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 06:01 |
<ryankemper@cumin2001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 05:49 |
<ryankemper> |
restarted pybal on `lvs1015` per the instructions in https://wikitech.wikimedia.org/wiki/PyBal#Services_known_to_PyBal_but_not_to_IPVS |
[production] |
| 05:39 |
<ryankemper> |
restarted pybal on `lvs1016` per the instructions in https://wikitech.wikimedia.org/wiki/PyBal#Services_known_to_PyBal_but_not_to_IPVS |
[production] |
| 04:44 |
<ryankemper@cumin2001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 04:44 |
<ryankemper@cumin2001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 04:44 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 04:44 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 04:28 |
<ryankemper@cumin2001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 04:26 |
<ryankemper@cumin1001> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) |
[production] |
| 04:25 |
<ryankemper@cumin2001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 04:23 |
<ryankemper@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 04:23 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) |
[production] |
| 04:22 |
<ryankemper@cumin2001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 04:21 |
<ryankemper@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 04:21 |
<ryankemper@cumin1001> |
START - Cookbook sre.hosts.downtime |
[production] |
| 04:05 |
<ryankemper> |
T269204 reimaging the following instances to debian buster (one each from `[public, internal] x [eqiad, codfw]`): `wdqs1005`, `wdqs2002`, `wdqs1008`, `wdqs2005` |
[production] |
| 03:49 |
<ryankemper@cumin2001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 03:49 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 03:49 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 03:44 |
<ryankemper@cumin2001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 03:43 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 03:43 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 02:13 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 02:12 |
<ryankemper@cumin1001> |
END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) |
[production] |
| 02:06 |
<ryankemper@cumin1001> |
END (PASS) - Cookbook sre.wdqs.data-reload (exit_code=0) |
[production] |
| 01:58 |
<ryankemper@cumin2001> |
END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) |
[production] |
| 01:09 |
<eileen> |
civicrm revision changed from 5fa107d32a to b0ffb87c5d, config revision is ffe0a99133 |
[production] |
| 00:45 |
<ryankemper@cumin2001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 00:45 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 00:45 |
<ryankemper@cumin1001> |
START - Cookbook sre.wdqs.data-transfer |
[production] |
| 00:42 |
<bd808> |
`kubectl delete po renderer-794886b9cd-9nc6c -n prod` after seeing lots of listen queue full errors in the pod logs. |
[paws] |