5701-5750 of 10000 results (71ms)
2022-05-02 ยง
07:29 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1184', diff saved to https://phabricator.wikimedia.org/P27300 and previous config saved to /var/cache/conftool/dbconfig/20220502-072947-ladsgroup.json [production]
07:24 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166', diff saved to https://phabricator.wikimedia.org/P27299 and previous config saved to /var/cache/conftool/dbconfig/20220502-072452-ladsgroup.json [production]
07:23 <elukey@cumin1001> START - Cookbook sre.hosts.reimage for host ores2001.codfw.wmnet with OS buster [production]
07:23 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1131 (T298563)', diff saved to https://phabricator.wikimedia.org/P27298 and previous config saved to /var/cache/conftool/dbconfig/20220502-072303-ladsgroup.json [production]
07:23 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on db1131.eqiad.wmnet with reason: Maintenance [production]
07:23 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 10:00:00 on db1131.eqiad.wmnet with reason: Maintenance [production]
07:22 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1096:3316 (T298563)', diff saved to https://phabricator.wikimedia.org/P27297 and previous config saved to /var/cache/conftool/dbconfig/20220502-072255-ladsgroup.json [production]
07:19 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1161 (T298295)', diff saved to https://phabricator.wikimedia.org/P27296 and previous config saved to /var/cache/conftool/dbconfig/20220502-071958-ladsgroup.json [production]
07:17 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1161 (T298295)', diff saved to https://phabricator.wikimedia.org/P27295 and previous config saved to /var/cache/conftool/dbconfig/20220502-071741-ladsgroup.json [production]
07:17 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on clouddb[1016,1020-1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance [production]
07:17 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 12:00:00 on clouddb[1016,1020-1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance [production]
07:17 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1161.eqiad.wmnet with reason: Maintenance [production]
07:17 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1161.eqiad.wmnet with reason: Maintenance [production]
07:17 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1113:3315 (T298295)', diff saved to https://phabricator.wikimedia.org/P27294 and previous config saved to /var/cache/conftool/dbconfig/20220502-071728-ladsgroup.json [production]
07:14 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1184 (T306560)', diff saved to https://phabricator.wikimedia.org/P27293 and previous config saved to /var/cache/conftool/dbconfig/20220502-071442-ladsgroup.json [production]
07:12 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1184 (T306560)', diff saved to https://phabricator.wikimedia.org/P27292 and previous config saved to /var/cache/conftool/dbconfig/20220502-071222-ladsgroup.json [production]
07:12 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1184.eqiad.wmnet with reason: Maintenance [production]
07:12 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1184.eqiad.wmnet with reason: Maintenance [production]
07:12 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1134 (T306560)', diff saved to https://phabricator.wikimedia.org/P27291 and previous config saved to /var/cache/conftool/dbconfig/20220502-071214-ladsgroup.json [production]
07:09 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166', diff saved to https://phabricator.wikimedia.org/P27290 and previous config saved to /var/cache/conftool/dbconfig/20220502-070947-ladsgroup.json [production]
07:07 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1096:3316', diff saved to https://phabricator.wikimedia.org/P27289 and previous config saved to /var/cache/conftool/dbconfig/20220502-070750-ladsgroup.json [production]
07:02 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1113:3315', diff saved to https://phabricator.wikimedia.org/P27288 and previous config saved to /var/cache/conftool/dbconfig/20220502-070222-ladsgroup.json [production]
06:57 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1134', diff saved to https://phabricator.wikimedia.org/P27287 and previous config saved to /var/cache/conftool/dbconfig/20220502-065709-ladsgroup.json [production]
06:54 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166 (T298565)', diff saved to https://phabricator.wikimedia.org/P27286 and previous config saved to /var/cache/conftool/dbconfig/20220502-065442-ladsgroup.json [production]
06:52 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1096:3316', diff saved to https://phabricator.wikimedia.org/P27285 and previous config saved to /var/cache/conftool/dbconfig/20220502-065245-ladsgroup.json [production]
06:47 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1113:3315', diff saved to https://phabricator.wikimedia.org/P27284 and previous config saved to /var/cache/conftool/dbconfig/20220502-064717-ladsgroup.json [production]
06:42 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1134', diff saved to https://phabricator.wikimedia.org/P27283 and previous config saved to /var/cache/conftool/dbconfig/20220502-064204-ladsgroup.json [production]
06:38 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1166 (T298565)', diff saved to https://phabricator.wikimedia.org/P27282 and previous config saved to /var/cache/conftool/dbconfig/20220502-063837-ladsgroup.json [production]
06:38 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1166.eqiad.wmnet with reason: Maintenance [production]
06:38 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1166.eqiad.wmnet with reason: Maintenance [production]
06:37 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1096:3316 (T298563)', diff saved to https://phabricator.wikimedia.org/P27281 and previous config saved to /var/cache/conftool/dbconfig/20220502-063740-ladsgroup.json [production]
06:32 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1113:3315 (T298295)', diff saved to https://phabricator.wikimedia.org/P27280 and previous config saved to /var/cache/conftool/dbconfig/20220502-063212-ladsgroup.json [production]
06:30 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1113:3315 (T298295)', diff saved to https://phabricator.wikimedia.org/P27279 and previous config saved to /var/cache/conftool/dbconfig/20220502-063055-ladsgroup.json [production]
06:30 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1113.eqiad.wmnet with reason: Maintenance [production]
06:30 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1113.eqiad.wmnet with reason: Maintenance [production]
06:30 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1144:3315 (T298295)', diff saved to https://phabricator.wikimedia.org/P27278 and previous config saved to /var/cache/conftool/dbconfig/20220502-063047-ladsgroup.json [production]
06:26 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1134 (T306560)', diff saved to https://phabricator.wikimedia.org/P27277 and previous config saved to /var/cache/conftool/dbconfig/20220502-062659-ladsgroup.json [production]
06:21 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1134 (T306560)', diff saved to https://phabricator.wikimedia.org/P27276 and previous config saved to /var/cache/conftool/dbconfig/20220502-062139-ladsgroup.json [production]
06:21 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1134.eqiad.wmnet with reason: Maintenance [production]
06:21 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1134.eqiad.wmnet with reason: Maintenance [production]
06:21 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1135 (T306560)', diff saved to https://phabricator.wikimedia.org/P27275 and previous config saved to /var/cache/conftool/dbconfig/20220502-062131-ladsgroup.json [production]
06:21 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166 (T298565)', diff saved to https://phabricator.wikimedia.org/P27274 and previous config saved to /var/cache/conftool/dbconfig/20220502-062131-ladsgroup.json [production]
06:15 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1144:3315', diff saved to https://phabricator.wikimedia.org/P27273 and previous config saved to /var/cache/conftool/dbconfig/20220502-061540-ladsgroup.json [production]
06:06 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1135', diff saved to https://phabricator.wikimedia.org/P27272 and previous config saved to /var/cache/conftool/dbconfig/20220502-060626-ladsgroup.json [production]
06:06 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166', diff saved to https://phabricator.wikimedia.org/P27271 and previous config saved to /var/cache/conftool/dbconfig/20220502-060626-ladsgroup.json [production]
06:01 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
06:00 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1144:3315', diff saved to https://phabricator.wikimedia.org/P27270 and previous config saved to /var/cache/conftool/dbconfig/20220502-060035-ladsgroup.json [production]
05:59 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
05:59 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
05:56 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]