4351-4400 of 10000 results (94ms)
2022-10-20 ยง
20:21 <wm-bot2> Set cloudvirt 'cloudvirt1041.eqiad.wmnet' maintenance (downtime id: b9027020-8a90-4c40-b0e6-6c8767f87917, use this to unset). - cookbook ran by andrew@bullseye [admin]
20:21 <wm-bot2> Set cloudvirt 'cloudvirt1040.eqiad.wmnet' maintenance (downtime id: f0d60fab-64e9-4da2-826c-229d31d0fbc2, use this to unset). - cookbook ran by andrew@bullseye [admin]
20:21 <robh@cumin2002> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host lvs4008.mgmt.ulsfo.wmnet with reboot policy FORCED [production]
20:21 <dzahn@cumin2002> START - Cookbook sre.gitlab.reboot-runner rolling reboot on A:eqiad and (A:gitlab-runner) [production]
20:21 <wm-bot2> Draining 'cloudvirt1043.eqiad.wmnet'. - cookbook ran by andrew@bullseye [admin]
20:21 <wm-bot2> Safe rebooting 'cloudvirt1043.eqiad.wmnet'. - cookbook ran by andrew@bullseye [admin]
20:20 <wm-bot2> Draining 'cloudvirt1042.eqiad.wmnet'. - cookbook ran by andrew@bullseye [admin]
20:20 <wm-bot2> Safe rebooting 'cloudvirt1042.eqiad.wmnet'. - cookbook ran by andrew@bullseye [admin]
20:20 <wm-bot2> Draining 'cloudvirt1041.eqiad.wmnet'. - cookbook ran by andrew@bullseye [admin]
20:20 <wm-bot2> Safe rebooting 'cloudvirt1041.eqiad.wmnet'. - cookbook ran by andrew@bullseye [admin]
20:20 <wm-bot2> Draining 'cloudvirt1040.eqiad.wmnet'. - cookbook ran by andrew@bullseye [admin]
20:20 <wm-bot2> Safe rebooting 'cloudvirt1040.eqiad.wmnet'. - cookbook ran by andrew@bullseye [admin]
20:20 <wm-bot2> Safe reboot of 'cloudvirt1044.eqiad.wmnet' finished successfully. - cookbook ran by andrew@bullseye [admin]
20:20 <wm-bot2> Unset cloudvirt 'cloudvirt1044.eqiad.wmnet' maintenance. - cookbook ran by andrew@bullseye [admin]
20:19 <wm-bot2> Safe reboot of 'cloudvirt1047.eqiad.wmnet' finished successfully. - cookbook ran by andrew@bullseye [admin]
20:19 <wm-bot2> Unset cloudvirt 'cloudvirt1047.eqiad.wmnet' maintenance. - cookbook ran by andrew@bullseye [admin]
20:19 <robh@cumin2002> START - Cookbook sre.hosts.provision for host lvs4008.mgmt.ulsfo.wmnet with reboot policy FORCED [production]
20:18 <wm-bot2> Safe reboot of 'cloudvirt1046.eqiad.wmnet' finished successfully. - cookbook ran by andrew@bullseye [admin]
20:18 <wm-bot2> Unset cloudvirt 'cloudvirt1046.eqiad.wmnet' maintenance. - cookbook ran by andrew@bullseye [admin]
20:17 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2175 (T318955)', diff saved to https://phabricator.wikimedia.org/P35737 and previous config saved to /var/cache/conftool/dbconfig/20221020-201748-ladsgroup.json [production]
20:16 <wm-bot2> Drained 'cloudvirt1044.eqiad.wmnet'. - cookbook ran by andrew@bullseye [admin]
20:15 <herron@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host thanos-fe1002.eqiad.wmnet [production]
20:15 <wm-bot2> Drained 'cloudvirt1047.eqiad.wmnet'. - cookbook ran by andrew@bullseye [admin]
20:15 <wm-bot2> Safe reboot of 'cloudvirt1045.eqiad.wmnet' finished successfully. - cookbook ran by andrew@bullseye [admin]
20:15 <wm-bot2> Unset cloudvirt 'cloudvirt1045.eqiad.wmnet' maintenance. - cookbook ran by andrew@bullseye [admin]
20:14 <robh@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host cp4049.mgmt.ulsfo.wmnet with reboot policy FORCED [production]
20:14 <robh@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host cp4047.mgmt.ulsfo.wmnet with reboot policy FORCED [production]
20:14 <wm-bot2> Drained 'cloudvirt1046.eqiad.wmnet'. - cookbook ran by andrew@bullseye [admin]
20:11 <wm-bot2> Drained 'cloudvirt1045.eqiad.wmnet'. - cookbook ran by andrew@bullseye [admin]
20:09 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2106 (T321312)', diff saved to https://phabricator.wikimedia.org/P35736 and previous config saved to /var/cache/conftool/dbconfig/20221020-200947-ladsgroup.json [production]
20:09 <sukhe@cumin2002> START - Cookbook sre.hosts.reimage for host cp4037.ulsfo.wmnet with OS buster [production]
20:08 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1146:3312', diff saved to https://phabricator.wikimedia.org/P35735 and previous config saved to /var/cache/conftool/dbconfig/20221020-200838-ladsgroup.json [production]
20:08 <thcipriani@deploy1002> thcipriani and jdlrobson: Backport for [[gerrit:843995|Updates to Wikipedia wordmark/taglines and project icons (T319223)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet [production]
20:08 <thcipriani@deploy1002> Started scap: Backport for [[gerrit:843995|Updates to Wikipedia wordmark/taglines and project icons (T319223)]] [production]
20:07 <herron@cumin1001> START - Cookbook sre.hosts.reboot-single for host thanos-fe1002.eqiad.wmnet [production]
20:05 <sukhe@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host cp4037.ulsfo.wmnet with OS buster [production]
20:05 <herron@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host thanos-fe1003.eqiad.wmnet [production]
20:03 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2106 (T321312)', diff saved to https://phabricator.wikimedia.org/P35734 and previous config saved to /var/cache/conftool/dbconfig/20221020-200321-ladsgroup.json [production]
20:03 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2106.codfw.wmnet with reason: Maintenance [production]
20:03 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2106.codfw.wmnet with reason: Maintenance [production]
20:02 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2175 (T318955)', diff saved to https://phabricator.wikimedia.org/P35733 and previous config saved to /var/cache/conftool/dbconfig/20221020-200205-ladsgroup.json [production]
20:01 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db2175.codfw.wmnet with reason: Maintenance [production]
20:01 <dzahn@cumin2002> END (PASS) - Cookbook sre.gitlab.reboot-runner (exit_code=0) rolling reboot on A:codfw and (A:gitlab-runner) [production]
20:01 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 8:00:00 on db2175.codfw.wmnet with reason: Maintenance [production]
20:01 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2170:3312 (T318955)', diff saved to https://phabricator.wikimedia.org/P35732 and previous config saved to /var/cache/conftool/dbconfig/20221020-200143-ladsgroup.json [production]
20:00 <robh@cumin2002> START - Cookbook sre.hosts.provision for host cp4049.mgmt.ulsfo.wmnet with reboot policy FORCED [production]
20:00 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2099.codfw.wmnet with reason: Maintenance [production]
20:00 <robh@cumin2002> START - Cookbook sre.hosts.provision for host cp4047.mgmt.ulsfo.wmnet with reboot policy FORCED [production]
19:59 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2099.codfw.wmnet with reason: Maintenance [production]
19:59 <wm-bot> <root> restart wikibugs (again) [tools.wikibugs]