2301-2350 of 10000 results (79ms)
2023-08-22 ยง
05:55 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1212 (T344589)', diff saved to https://phabricator.wikimedia.org/P50776 and previous config saved to /var/cache/conftool/dbconfig/20230822-055528-ladsgroup.json [production]
05:55 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on clouddb[1013,1017,1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance [production]
05:55 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 2 days, 0:00:00 on clouddb[1013,1017,1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance [production]
05:55 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1212.eqiad.wmnet with reason: Maintenance [production]
05:55 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1212.eqiad.wmnet with reason: Maintenance [production]
05:54 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1198 (T344589)', diff saved to https://phabricator.wikimedia.org/P50775 and previous config saved to /var/cache/conftool/dbconfig/20230822-055457-ladsgroup.json [production]
05:54 <stevemunene@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1113.eqiad.wmnet with reason: host reimage [production]
05:53 <stevemunene@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1114.eqiad.wmnet with reason: host reimage [production]
05:51 <stevemunene@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1113.eqiad.wmnet with reason: host reimage [production]
05:51 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2137:3315 (T343718)', diff saved to https://phabricator.wikimedia.org/P50774 and previous config saved to /var/cache/conftool/dbconfig/20230822-055101-ladsgroup.json [production]
05:51 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2137.codfw.wmnet with reason: Maintenance [production]
05:50 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2137.codfw.wmnet with reason: Maintenance [production]
05:50 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2128 (T343718)', diff saved to https://phabricator.wikimedia.org/P50773 and previous config saved to /var/cache/conftool/dbconfig/20230822-055050-ladsgroup.json [production]
05:50 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1185 (T343718)', diff saved to https://phabricator.wikimedia.org/P50772 and previous config saved to /var/cache/conftool/dbconfig/20230822-055040-ladsgroup.json [production]
05:50 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1185.eqiad.wmnet with reason: Maintenance [production]
05:50 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1185.eqiad.wmnet with reason: Maintenance [production]
05:50 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1161 (T343718)', diff saved to https://phabricator.wikimedia.org/P50771 and previous config saved to /var/cache/conftool/dbconfig/20230822-055030-ladsgroup.json [production]
05:49 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2177 (T344589)', diff saved to https://phabricator.wikimedia.org/P50770 and previous config saved to /var/cache/conftool/dbconfig/20230822-054920-ladsgroup.json [production]
05:49 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2177.codfw.wmnet with reason: Maintenance [production]
05:49 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2177.codfw.wmnet with reason: Maintenance [production]
05:48 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2156 (T344589)', diff saved to https://phabricator.wikimedia.org/P50769 and previous config saved to /var/cache/conftool/dbconfig/20230822-054855-ladsgroup.json [production]
05:39 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1198', diff saved to https://phabricator.wikimedia.org/P50768 and previous config saved to /var/cache/conftool/dbconfig/20230822-053951-ladsgroup.json [production]
05:38 <stevemunene@cumin1001> START - Cookbook sre.hosts.reimage for host an-worker1114.eqiad.wmnet with OS bullseye [production]
05:37 <stevemunene@cumin1001> START - Cookbook sre.hosts.reimage for host an-worker1113.eqiad.wmnet with OS bullseye [production]
05:35 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2128', diff saved to https://phabricator.wikimedia.org/P50767 and previous config saved to /var/cache/conftool/dbconfig/20230822-053544-ladsgroup.json [production]
05:35 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1161', diff saved to https://phabricator.wikimedia.org/P50766 and previous config saved to /var/cache/conftool/dbconfig/20230822-053524-ladsgroup.json [production]
05:33 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2156', diff saved to https://phabricator.wikimedia.org/P50765 and previous config saved to /var/cache/conftool/dbconfig/20230822-053349-ladsgroup.json [production]
05:24 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1198', diff saved to https://phabricator.wikimedia.org/P50764 and previous config saved to /var/cache/conftool/dbconfig/20230822-052445-ladsgroup.json [production]
05:20 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2128', diff saved to https://phabricator.wikimedia.org/P50763 and previous config saved to /var/cache/conftool/dbconfig/20230822-052038-ladsgroup.json [production]
05:20 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1161', diff saved to https://phabricator.wikimedia.org/P50762 and previous config saved to /var/cache/conftool/dbconfig/20230822-052018-ladsgroup.json [production]
05:18 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2156', diff saved to https://phabricator.wikimedia.org/P50761 and previous config saved to /var/cache/conftool/dbconfig/20230822-051843-ladsgroup.json [production]
05:13 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Set db1184 with weight 0 T344621', diff saved to https://phabricator.wikimedia.org/P50760 and previous config saved to /var/cache/conftool/dbconfig/20230822-051347-ladsgroup.json [production]
05:13 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 35 hosts with reason: Primary switchover s1 T344621 [production]
05:13 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1:00:00 on 35 hosts with reason: Primary switchover s1 T344621 [production]
05:12 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1184 (T344589)', diff saved to https://phabricator.wikimedia.org/P50759 and previous config saved to /var/cache/conftool/dbconfig/20230822-051204-ladsgroup.json [production]
05:09 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1198 (T344589)', diff saved to https://phabricator.wikimedia.org/P50758 and previous config saved to /var/cache/conftool/dbconfig/20230822-050938-ladsgroup.json [production]
05:05 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1184 (T344589)', diff saved to https://phabricator.wikimedia.org/P50757 and previous config saved to /var/cache/conftool/dbconfig/20230822-050543-ladsgroup.json [production]
05:05 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1184.eqiad.wmnet with reason: Maintenance [production]
05:05 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2128 (T343718)', diff saved to https://phabricator.wikimedia.org/P50756 and previous config saved to /var/cache/conftool/dbconfig/20230822-050532-ladsgroup.json [production]
05:05 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1184.eqiad.wmnet with reason: Maintenance [production]
05:05 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1161 (T343718)', diff saved to https://phabricator.wikimedia.org/P50755 and previous config saved to /var/cache/conftool/dbconfig/20230822-050511-ladsgroup.json [production]
05:03 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2156 (T344589)', diff saved to https://phabricator.wikimedia.org/P50754 and previous config saved to /var/cache/conftool/dbconfig/20230822-050337-ladsgroup.json [production]
04:57 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1198 (T344589)', diff saved to https://phabricator.wikimedia.org/P50753 and previous config saved to /var/cache/conftool/dbconfig/20230822-045716-ladsgroup.json [production]
04:57 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1198.eqiad.wmnet with reason: Maintenance [production]
04:57 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2156 (T344589)', diff saved to https://phabricator.wikimedia.org/P50752 and previous config saved to /var/cache/conftool/dbconfig/20230822-045707-ladsgroup.json [production]
04:57 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on db2186.codfw.wmnet with reason: Maintenance [production]
04:57 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 2 days, 0:00:00 on db2186.codfw.wmnet with reason: Maintenance [production]
04:56 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2156.codfw.wmnet with reason: Maintenance [production]
04:56 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1198.eqiad.wmnet with reason: Maintenance [production]
04:56 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1189 (T344589)', diff saved to https://phabricator.wikimedia.org/P50751 and previous config saved to /var/cache/conftool/dbconfig/20230822-045651-ladsgroup.json [production]