5101-5150 of 10000 results (90ms)
2023-08-29 ยง
07:28 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2162.codfw.wmnet with reason: Maintenance [production]
07:28 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2162.codfw.wmnet with reason: Maintenance [production]
07:28 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2154 (T343718)', diff saved to https://phabricator.wikimedia.org/P51835 and previous config saved to /var/cache/conftool/dbconfig/20230829-072832-ladsgroup.json [production]
07:28 <stevemunene@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1120.eqiad.wmnet with reason: host reimage [production]
07:27 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 30 days, 0:00:00 on debmonitor2003.codfw.wmnet with reason: WIP [production]
07:26 <jmm@cumin2002> START - Cookbook sre.hosts.downtime for 30 days, 0:00:00 on debmonitor2003.codfw.wmnet with reason: WIP [production]
07:26 <stevemunene@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1121.eqiad.wmnet with reason: host reimage [production]
07:25 <stevemunene@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1120.eqiad.wmnet with reason: host reimage [production]
07:24 <kartik@deploy1002> Finished scap: Backport for [[gerrit:952846|Enable Content and Section translation in Ligurian Wikipedia (T337669)]] (duration: 21m 02s) [production]
07:23 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2028.codfw.wmnet [production]
07:23 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2028.codfw.wmnet [production]
07:22 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1177 (T343718)', diff saved to https://phabricator.wikimedia.org/P51834 and previous config saved to /var/cache/conftool/dbconfig/20230829-072249-ladsgroup.json [production]
07:17 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host ganeti2028.codfw.wmnet [production]
07:14 <kartik@deploy1002> kartik: Continuing with sync [production]
07:13 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2154', diff saved to https://phabricator.wikimedia.org/P51833 and previous config saved to /var/cache/conftool/dbconfig/20230829-071326-ladsgroup.json [production]
07:12 <stevemunene@cumin1001> START - Cookbook sre.hosts.reimage for host an-worker1121.eqiad.wmnet with OS bullseye [production]
07:12 <stevemunene@cumin1001> START - Cookbook sre.hosts.reimage for host an-worker1120.eqiad.wmnet with OS bullseye [production]
07:07 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2028.codfw.wmnet [production]
07:07 <kartik@deploy1002> kartik: Backport for [[gerrit:952846|Enable Content and Section translation in Ligurian Wikipedia (T337669)]] synced to the testservers mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet, and mw-debug kubernetes deployment (accessible via k8s-experimental XWD option) [production]
07:04 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1177 (T343718)', diff saved to https://phabricator.wikimedia.org/P51832 and previous config saved to /var/cache/conftool/dbconfig/20230829-070443-ladsgroup.json [production]
07:04 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2027.codfw.wmnet [production]
07:04 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance [production]
07:04 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance [production]
07:04 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1172 (T343718)', diff saved to https://phabricator.wikimedia.org/P51831 and previous config saved to /var/cache/conftool/dbconfig/20230829-070422-ladsgroup.json [production]
07:04 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2027.codfw.wmnet [production]
07:03 <kartik@deploy1002> Started scap: Backport for [[gerrit:952846|Enable Content and Section translation in Ligurian Wikipedia (T337669)]] [production]
06:58 <stevemunene@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1119.eqiad.wmnet with OS bullseye [production]
06:58 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2154', diff saved to https://phabricator.wikimedia.org/P51830 and previous config saved to /var/cache/conftool/dbconfig/20230829-065819-ladsgroup.json [production]
06:57 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host ganeti2027.codfw.wmnet [production]
06:57 <stevemunene@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1118.eqiad.wmnet with OS bullseye [production]
06:55 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1147 (T343718)', diff saved to https://phabricator.wikimedia.org/P51829 and previous config saved to /var/cache/conftool/dbconfig/20230829-065525-ladsgroup.json [production]
06:55 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1147.eqiad.wmnet with reason: Maintenance [production]
06:55 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1147.eqiad.wmnet with reason: Maintenance [production]
06:55 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1146:3314 (T343718)', diff saved to https://phabricator.wikimedia.org/P51828 and previous config saved to /var/cache/conftool/dbconfig/20230829-065515-ladsgroup.json [production]
06:54 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2027.codfw.wmnet [production]
06:51 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2138:3314 (T343718)', diff saved to https://phabricator.wikimedia.org/P51827 and previous config saved to /var/cache/conftool/dbconfig/20230829-065059-ladsgroup.json [production]
06:50 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2138.codfw.wmnet with reason: Maintenance [production]
06:50 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2138.codfw.wmnet with reason: Maintenance [production]
06:50 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2137:3314 (T343718)', diff saved to https://phabricator.wikimedia.org/P51826 and previous config saved to /var/cache/conftool/dbconfig/20230829-065038-ladsgroup.json [production]
06:49 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P51825 and previous config saved to /var/cache/conftool/dbconfig/20230829-064916-ladsgroup.json [production]
06:43 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2154 (T343718)', diff saved to https://phabricator.wikimedia.org/P51824 and previous config saved to /var/cache/conftool/dbconfig/20230829-064313-ladsgroup.json [production]
06:40 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1146:3314', diff saved to https://phabricator.wikimedia.org/P51823 and previous config saved to /var/cache/conftool/dbconfig/20230829-064009-ladsgroup.json [production]
06:36 <stevemunene@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1119.eqiad.wmnet with reason: host reimage [production]
06:35 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2137:3314', diff saved to https://phabricator.wikimedia.org/P51822 and previous config saved to /var/cache/conftool/dbconfig/20230829-063531-ladsgroup.json [production]
06:34 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P51821 and previous config saved to /var/cache/conftool/dbconfig/20230829-063410-ladsgroup.json [production]
06:33 <stevemunene@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1118.eqiad.wmnet with reason: host reimage [production]
06:31 <stevemunene@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1119.eqiad.wmnet with reason: host reimage [production]
06:30 <stevemunene@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1118.eqiad.wmnet with reason: host reimage [production]
06:25 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2154 (T343718)', diff saved to https://phabricator.wikimedia.org/P51820 and previous config saved to /var/cache/conftool/dbconfig/20230829-062532-ladsgroup.json [production]
06:25 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2154.codfw.wmnet with reason: Maintenance [production]