1201-1250 of 10000 results (105ms)
2024-07-08 ยง
21:02 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host wdqs1023.eqiad.wmnet [production]
21:02 <fabfur@cumin1002> cookbooks.sre.cdn.roll-reboot finished rebooting cp3080.esams.wmnet [production]
21:01 <fabfur@cumin1002> cookbooks.sre.cdn.roll-reboot finished rebooting cp3072.esams.wmnet [production]
21:01 <arnaudb@cumin1002> dbctl commit (dc=all): 'Depooling db2158 (T367781)', diff saved to https://phabricator.wikimedia.org/P65997 and previous config saved to /var/cache/conftool/dbconfig/20240708-210144-arnaudb.json [production]
21:01 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db2187.codfw.wmnet with reason: Maintenance [production]
21:01 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 8:00:00 on db2187.codfw.wmnet with reason: Maintenance [production]
21:01 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2158.codfw.wmnet with reason: Maintenance [production]
21:01 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 4:00:00 on db2158.codfw.wmnet with reason: Maintenance [production]
21:01 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2151 (T367781)', diff saved to https://phabricator.wikimedia.org/P65996 and previous config saved to /var/cache/conftool/dbconfig/20240708-210106-arnaudb.json [production]
20:55 <bking@cumin2002> START - Cookbook sre.hosts.reboot-single for host wdqs1023.eqiad.wmnet [production]
20:52 <catrope@deploy1002> Finished scap: Backport for [[gerrit:1051853|Graph extension: Add tracking for data sources used in <graph> tags]] (duration: 13m 00s) [production]
20:50 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host wdqs1022.eqiad.wmnet [production]
20:47 <catrope@deploy1002> catrope: Continuing with sync [production]
20:45 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2151', diff saved to https://phabricator.wikimedia.org/P65995 and previous config saved to /var/cache/conftool/dbconfig/20240708-204559-arnaudb.json [production]
20:43 <bking@cumin2002> START - Cookbook sre.hosts.reboot-single for host wdqs1022.eqiad.wmnet [production]
20:42 <bking@cumin2002> END (FAIL) - Cookbook sre.wdqs.reboot (exit_code=99) [production]
20:42 <catrope@deploy1002> catrope: Backport for [[gerrit:1051853|Graph extension: Add tracking for data sources used in <graph> tags]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
20:40 <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db1230 (T367856)', diff saved to https://phabricator.wikimedia.org/P65994 and previous config saved to /var/cache/conftool/dbconfig/20240708-204042-marostegui.json [production]
20:40 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1230.eqiad.wmnet with reason: Maintenance [production]
20:40 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1230.eqiad.wmnet with reason: Maintenance [production]
20:39 <catrope@deploy1002> Started scap sync-world: Backport for [[gerrit:1051853|Graph extension: Add tracking for data sources used in <graph> tags]] [production]
20:38 <bking@cumin2002> START - Cookbook sre.wdqs.reboot [production]
20:35 <bking@cumin2002> END (FAIL) - Cookbook sre.wdqs.reboot (exit_code=99) [production]
20:30 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2151', diff saved to https://phabricator.wikimedia.org/P65993 and previous config saved to /var/cache/conftool/dbconfig/20240708-203052-arnaudb.json [production]
20:28 <bking@cumin2002> START - Cookbook sre.wdqs.reboot [production]
20:27 <bking@cumin2002> END (PASS) - Cookbook sre.wdqs.reboot (exit_code=0) [production]
20:15 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2151 (T367781)', diff saved to https://phabricator.wikimedia.org/P65992 and previous config saved to /var/cache/conftool/dbconfig/20240708-201545-arnaudb.json [production]
20:13 <arnaudb@cumin1002> dbctl commit (dc=all): 'Depooling db2151 (T367781)', diff saved to https://phabricator.wikimedia.org/P65991 and previous config saved to /var/cache/conftool/dbconfig/20240708-201318-arnaudb.json [production]
20:13 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2151.codfw.wmnet with reason: Maintenance [production]
20:13 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 4:00:00 on db2151.codfw.wmnet with reason: Maintenance [production]
20:12 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2129 (T367781)', diff saved to https://phabricator.wikimedia.org/P65990 and previous config saved to /var/cache/conftool/dbconfig/20240708-201256-arnaudb.json [production]
20:08 <bking@cumin2002> START - Cookbook sre.wdqs.reboot [production]
19:57 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2129', diff saved to https://phabricator.wikimedia.org/P65989 and previous config saved to /var/cache/conftool/dbconfig/20240708-195749-arnaudb.json [production]
19:44 <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db2149 (T367856)', diff saved to https://phabricator.wikimedia.org/P65988 and previous config saved to /var/cache/conftool/dbconfig/20240708-194435-marostegui.json [production]
19:44 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2149.codfw.wmnet with reason: Maintenance [production]
19:44 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2149.codfw.wmnet with reason: Maintenance [production]
19:42 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2129', diff saved to https://phabricator.wikimedia.org/P65987 and previous config saved to /var/cache/conftool/dbconfig/20240708-194242-arnaudb.json [production]
19:39 <bking@cumin2002> END (PASS) - Cookbook sre.wdqs.reboot (exit_code=0) [production]
19:27 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2129 (T367781)', diff saved to https://phabricator.wikimedia.org/P65986 and previous config saved to /var/cache/conftool/dbconfig/20240708-192735-arnaudb.json [production]
19:25 <arnaudb@cumin1002> dbctl commit (dc=all): 'Depooling db2129 (T367781)', diff saved to https://phabricator.wikimedia.org/P65985 and previous config saved to /var/cache/conftool/dbconfig/20240708-192508-arnaudb.json [production]
19:25 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2129.codfw.wmnet with reason: Maintenance [production]
19:24 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 4:00:00 on db2129.codfw.wmnet with reason: Maintenance [production]
19:24 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2124 (T367781)', diff saved to https://phabricator.wikimedia.org/P65984 and previous config saved to /var/cache/conftool/dbconfig/20240708-192444-arnaudb.json [production]
19:21 <fabfur@cumin1002> cookbooks.sre.cdn.roll-reboot finished rebooting cp3079.esams.wmnet [production]
19:21 <fabfur@cumin1002> cookbooks.sre.cdn.roll-reboot finished rebooting cp3071.esams.wmnet [production]
19:21 <btullis@deploy1002> helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. [production]
19:20 <btullis@deploy1002> helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. [production]
19:09 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2124', diff saved to https://phabricator.wikimedia.org/P65983 and previous config saved to /var/cache/conftool/dbconfig/20240708-190937-arnaudb.json [production]
19:02 <bking@cumin2002> START - Cookbook sre.wdqs.reboot [production]
18:54 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2124', diff saved to https://phabricator.wikimedia.org/P65982 and previous config saved to /var/cache/conftool/dbconfig/20240708-185430-arnaudb.json [production]