6451-6500 of 10000 results (86ms)
2023-08-22 ยง
05:13 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Set db1184 with weight 0 T344621', diff saved to https://phabricator.wikimedia.org/P50760 and previous config saved to /var/cache/conftool/dbconfig/20230822-051347-ladsgroup.json [production]
05:13 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on 35 hosts with reason: Primary switchover s1 T344621 [production]
05:13 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1:00:00 on 35 hosts with reason: Primary switchover s1 T344621 [production]
05:12 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1184 (T344589)', diff saved to https://phabricator.wikimedia.org/P50759 and previous config saved to /var/cache/conftool/dbconfig/20230822-051204-ladsgroup.json [production]
05:09 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1198 (T344589)', diff saved to https://phabricator.wikimedia.org/P50758 and previous config saved to /var/cache/conftool/dbconfig/20230822-050938-ladsgroup.json [production]
05:05 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1184 (T344589)', diff saved to https://phabricator.wikimedia.org/P50757 and previous config saved to /var/cache/conftool/dbconfig/20230822-050543-ladsgroup.json [production]
05:05 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1184.eqiad.wmnet with reason: Maintenance [production]
05:05 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2128 (T343718)', diff saved to https://phabricator.wikimedia.org/P50756 and previous config saved to /var/cache/conftool/dbconfig/20230822-050532-ladsgroup.json [production]
05:05 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1184.eqiad.wmnet with reason: Maintenance [production]
05:05 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1161 (T343718)', diff saved to https://phabricator.wikimedia.org/P50755 and previous config saved to /var/cache/conftool/dbconfig/20230822-050511-ladsgroup.json [production]
05:03 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2156 (T344589)', diff saved to https://phabricator.wikimedia.org/P50754 and previous config saved to /var/cache/conftool/dbconfig/20230822-050337-ladsgroup.json [production]
04:57 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1198 (T344589)', diff saved to https://phabricator.wikimedia.org/P50753 and previous config saved to /var/cache/conftool/dbconfig/20230822-045716-ladsgroup.json [production]
04:57 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1198.eqiad.wmnet with reason: Maintenance [production]
04:57 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2156 (T344589)', diff saved to https://phabricator.wikimedia.org/P50752 and previous config saved to /var/cache/conftool/dbconfig/20230822-045707-ladsgroup.json [production]
04:57 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on db2186.codfw.wmnet with reason: Maintenance [production]
04:57 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 2 days, 0:00:00 on db2186.codfw.wmnet with reason: Maintenance [production]
04:56 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2156.codfw.wmnet with reason: Maintenance [production]
04:56 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1198.eqiad.wmnet with reason: Maintenance [production]
04:56 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1189 (T344589)', diff saved to https://phabricator.wikimedia.org/P50751 and previous config saved to /var/cache/conftool/dbconfig/20230822-045651-ladsgroup.json [production]
04:56 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2156.codfw.wmnet with reason: Maintenance [production]
04:56 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2149 (T344589)', diff saved to https://phabricator.wikimedia.org/P50750 and previous config saved to /var/cache/conftool/dbconfig/20230822-045638-ladsgroup.json [production]
04:52 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2128 (T343718)', diff saved to https://phabricator.wikimedia.org/P50749 and previous config saved to /var/cache/conftool/dbconfig/20230822-045233-ladsgroup.json [production]
04:52 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on db2186.codfw.wmnet with reason: Maintenance [production]
04:52 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 2 days, 0:00:00 on db2186.codfw.wmnet with reason: Maintenance [production]
04:52 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2128.codfw.wmnet with reason: Maintenance [production]
04:52 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2128.codfw.wmnet with reason: Maintenance [production]
04:52 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2123 (T343718)', diff saved to https://phabricator.wikimedia.org/P50748 and previous config saved to /var/cache/conftool/dbconfig/20230822-045218-ladsgroup.json [production]
04:47 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1161 (T343718)', diff saved to https://phabricator.wikimedia.org/P50747 and previous config saved to /var/cache/conftool/dbconfig/20230822-044715-ladsgroup.json [production]
04:47 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on clouddb[1016,1020-1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance [production]
04:47 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 2 days, 0:00:00 on clouddb[1016,1020-1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance [production]
04:47 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1161.eqiad.wmnet with reason: Maintenance [production]
04:47 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1161.eqiad.wmnet with reason: Maintenance [production]
04:41 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1189', diff saved to https://phabricator.wikimedia.org/P50746 and previous config saved to /var/cache/conftool/dbconfig/20230822-044145-ladsgroup.json [production]
04:41 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2149', diff saved to https://phabricator.wikimedia.org/P50745 and previous config saved to /var/cache/conftool/dbconfig/20230822-044131-ladsgroup.json [production]
04:37 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2123', diff saved to https://phabricator.wikimedia.org/P50744 and previous config saved to /var/cache/conftool/dbconfig/20230822-043712-ladsgroup.json [production]
04:31 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1145.eqiad.wmnet with reason: Maintenance [production]
04:31 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1145.eqiad.wmnet with reason: Maintenance [production]
04:30 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1144:3315 (T343718)', diff saved to https://phabricator.wikimedia.org/P50743 and previous config saved to /var/cache/conftool/dbconfig/20230822-043058-ladsgroup.json [production]
04:26 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1189', diff saved to https://phabricator.wikimedia.org/P50742 and previous config saved to /var/cache/conftool/dbconfig/20230822-042639-ladsgroup.json [production]
04:26 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2149', diff saved to https://phabricator.wikimedia.org/P50741 and previous config saved to /var/cache/conftool/dbconfig/20230822-042625-ladsgroup.json [production]
04:22 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2123', diff saved to https://phabricator.wikimedia.org/P50740 and previous config saved to /var/cache/conftool/dbconfig/20230822-042206-ladsgroup.json [production]
04:15 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1144:3315', diff saved to https://phabricator.wikimedia.org/P50739 and previous config saved to /var/cache/conftool/dbconfig/20230822-041551-ladsgroup.json [production]
04:11 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1189 (T344589)', diff saved to https://phabricator.wikimedia.org/P50738 and previous config saved to /var/cache/conftool/dbconfig/20230822-041132-ladsgroup.json [production]
04:11 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2149 (T344589)', diff saved to https://phabricator.wikimedia.org/P50737 and previous config saved to /var/cache/conftool/dbconfig/20230822-041119-ladsgroup.json [production]
04:07 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2123 (T343718)', diff saved to https://phabricator.wikimedia.org/P50736 and previous config saved to /var/cache/conftool/dbconfig/20230822-040658-ladsgroup.json [production]
04:04 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2149 (T344589)', diff saved to https://phabricator.wikimedia.org/P50735 and previous config saved to /var/cache/conftool/dbconfig/20230822-040451-ladsgroup.json [production]
04:04 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2149.codfw.wmnet with reason: Maintenance [production]
04:04 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2149.codfw.wmnet with reason: Maintenance [production]
04:03 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1189 (T344589)', diff saved to https://phabricator.wikimedia.org/P50734 and previous config saved to /var/cache/conftool/dbconfig/20230822-040315-ladsgroup.json [production]
04:03 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1189.eqiad.wmnet with reason: Maintenance [production]