|
2022-11-02
ยง
|
| 18:56 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1189.eqiad.wmnet with reason: Maintenance |
[production] |
| 18:56 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1179 (T318605)', diff saved to https://phabricator.wikimedia.org/P37849 and previous config saved to /var/cache/conftool/dbconfig/20221102-185604-ladsgroup.json |
[production] |
| 18:50 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2178', diff saved to https://phabricator.wikimedia.org/P37848 and previous config saved to /var/cache/conftool/dbconfig/20221102-185023-marostegui.json |
[production] |
| 18:45 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2149', diff saved to https://phabricator.wikimedia.org/P37847 and previous config saved to /var/cache/conftool/dbconfig/20221102-184538-ladsgroup.json |
[production] |
| 18:40 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1179', diff saved to https://phabricator.wikimedia.org/P37846 and previous config saved to /var/cache/conftool/dbconfig/20221102-184056-ladsgroup.json |
[production] |
| 18:35 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2178 (T321123)', diff saved to https://phabricator.wikimedia.org/P37845 and previous config saved to /var/cache/conftool/dbconfig/20221102-183514-marostegui.json |
[production] |
| 18:33 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db2178 (T321123)', diff saved to https://phabricator.wikimedia.org/P37844 and previous config saved to /var/cache/conftool/dbconfig/20221102-183327-marostegui.json |
[production] |
| 18:33 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db2178.codfw.wmnet with reason: Maintenance |
[production] |
| 18:33 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 8:00:00 on db2178.codfw.wmnet with reason: Maintenance |
[production] |
| 18:33 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2171:3315 (T321123)', diff saved to https://phabricator.wikimedia.org/P37843 and previous config saved to /var/cache/conftool/dbconfig/20221102-183305-marostegui.json |
[production] |
| 18:30 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2149 (T318605)', diff saved to https://phabricator.wikimedia.org/P37842 and previous config saved to /var/cache/conftool/dbconfig/20221102-183031-ladsgroup.json |
[production] |
| 18:25 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1179', diff saved to https://phabricator.wikimedia.org/P37841 and previous config saved to /var/cache/conftool/dbconfig/20221102-182548-ladsgroup.json |
[production] |
| 18:17 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2171:3315', diff saved to https://phabricator.wikimedia.org/P37840 and previous config saved to /var/cache/conftool/dbconfig/20221102-181753-marostegui.json |
[production] |
| 18:14 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mw-debug: apply |
[production] |
| 18:14 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mw-debug: apply |
[production] |
| 18:14 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply |
[production] |
| 18:12 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mw-debug: apply |
[production] |
| 18:10 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1179 (T318605)', diff saved to https://phabricator.wikimedia.org/P37839 and previous config saved to /var/cache/conftool/dbconfig/20221102-181039-ladsgroup.json |
[production] |
| 18:10 |
<jhuneidi@deploy1002> |
Synchronized php: group1 wikis to 1.40.0-wmf.8 refs T320513 (duration: 03m 43s) |
[production] |
| 18:07 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mw-debug: apply |
[production] |
| 18:07 |
<jhuneidi@deploy1002> |
rebuilt and synchronized wikiversions files: group1 wikis to 1.40.0-wmf.8 refs T320513 |
[production] |
| 18:06 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mw-debug: apply |
[production] |
| 18:06 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply |
[production] |
| 18:05 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mw-debug: apply |
[production] |
| 18:02 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2171:3315', diff saved to https://phabricator.wikimedia.org/P37838 and previous config saved to /var/cache/conftool/dbconfig/20221102-180245-marostegui.json |
[production] |
| 18:01 |
<ejegg> |
updated fundraising python tools from 4c143d97 to 72570bdd |
[production] |
| 17:52 |
<hnowlan@deploy1002> |
helmfile [staging] DONE helmfile.d/services/thumbor: sync |
[production] |
| 17:47 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2171:3315 (T321123)', diff saved to https://phabricator.wikimedia.org/P37837 and previous config saved to /var/cache/conftool/dbconfig/20221102-174737-marostegui.json |
[production] |
| 17:46 |
<pt1979@cumin2002> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts cp4052.ulsfo.wmnet |
[production] |
| 17:46 |
<pt1979@cumin2002> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 17:45 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db2171:3315 (T321123)', diff saved to https://phabricator.wikimedia.org/P37836 and previous config saved to /var/cache/conftool/dbconfig/20221102-174504-marostegui.json |
[production] |
| 17:45 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db2171.codfw.wmnet with reason: Maintenance |
[production] |
| 17:45 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 8:00:00 on db2171.codfw.wmnet with reason: Maintenance |
[production] |
| 17:44 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2157 (T321123)', diff saved to https://phabricator.wikimedia.org/P37835 and previous config saved to /var/cache/conftool/dbconfig/20221102-174451-marostegui.json |
[production] |
| 17:43 |
<pt1979@cumin2002> |
START - Cookbook sre.dns.netbox |
[production] |
| 17:41 |
<hnowlan@deploy1002> |
helmfile [staging] START helmfile.d/services/thumbor: sync |
[production] |
| 17:41 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance |
[production] |
| 17:41 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 8:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance |
[production] |
| 17:41 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1197 (T318955)', diff saved to https://phabricator.wikimedia.org/P37834 and previous config saved to /var/cache/conftool/dbconfig/20221102-174138-ladsgroup.json |
[production] |
| 17:41 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1179 (T318605)', diff saved to https://phabricator.wikimedia.org/P37833 and previous config saved to /var/cache/conftool/dbconfig/20221102-174110-ladsgroup.json |
[production] |
| 17:41 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1179.eqiad.wmnet with reason: Maintenance |
[production] |
| 17:41 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1179.eqiad.wmnet with reason: Maintenance |
[production] |
| 17:40 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1175 (T318605)', diff saved to https://phabricator.wikimedia.org/P37832 and previous config saved to /var/cache/conftool/dbconfig/20221102-174048-ladsgroup.json |
[production] |
| 17:40 |
<mutante> |
clouddumps1002 - /usr/local/bin/dump-fetch-phabdumps.sh T322221 |
[production] |
| 17:39 |
<pt1979@cumin2002> |
START - Cookbook sre.hosts.decommission for hosts cp4052.ulsfo.wmnet |
[production] |
| 17:29 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2157', diff saved to https://phabricator.wikimedia.org/P37831 and previous config saved to /var/cache/conftool/dbconfig/20221102-172944-marostegui.json |
[production] |
| 17:26 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1197', diff saved to https://phabricator.wikimedia.org/P37830 and previous config saved to /var/cache/conftool/dbconfig/20221102-172630-ladsgroup.json |
[production] |
| 17:25 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1175', diff saved to https://phabricator.wikimedia.org/P37829 and previous config saved to /var/cache/conftool/dbconfig/20221102-172540-ladsgroup.json |
[production] |
| 17:14 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2157', diff saved to https://phabricator.wikimedia.org/P37828 and previous config saved to /var/cache/conftool/dbconfig/20221102-171436-marostegui.json |
[production] |
| 17:11 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1197', diff saved to https://phabricator.wikimedia.org/P37827 and previous config saved to /var/cache/conftool/dbconfig/20221102-171122-ladsgroup.json |
[production] |