8501-8550 of 10000 results (111ms)
2023-05-05 §
01:18 <denisse@cumin1001> START - Cookbook sre.ganeti.reboot-vm for VM prometheus3002.esams.wmnet [production]
01:17 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2126', diff saved to https://phabricator.wikimedia.org/P47724 and previous config saved to /var/cache/conftool/dbconfig/20230505-011700-ladsgroup.json [production]
01:16 <denisse@cumin1001> START - Cookbook sre.hosts.reboot-single for host prometheus1005.eqiad.wmnet [production]
01:14 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1156', diff saved to https://phabricator.wikimedia.org/P47723 and previous config saved to /var/cache/conftool/dbconfig/20230505-011421-ladsgroup.json [production]
01:01 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2126', diff saved to https://phabricator.wikimedia.org/P47722 and previous config saved to /var/cache/conftool/dbconfig/20230505-010154-ladsgroup.json [production]
00:59 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1156', diff saved to https://phabricator.wikimedia.org/P47721 and previous config saved to /var/cache/conftool/dbconfig/20230505-005914-ladsgroup.json [production]
00:46 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2126 (T335845)', diff saved to https://phabricator.wikimedia.org/P47720 and previous config saved to /var/cache/conftool/dbconfig/20230505-004648-ladsgroup.json [production]
00:44 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1156 (T335845)', diff saved to https://phabricator.wikimedia.org/P47719 and previous config saved to /var/cache/conftool/dbconfig/20230505-004408-ladsgroup.json [production]
00:39 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2126 (T335845)', diff saved to https://phabricator.wikimedia.org/P47718 and previous config saved to /var/cache/conftool/dbconfig/20230505-003914-ladsgroup.json [production]
00:39 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on db2187.codfw.wmnet with reason: Maintenance [production]
00:39 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 2 days, 0:00:00 on db2187.codfw.wmnet with reason: Maintenance [production]
00:39 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2126.codfw.wmnet with reason: Maintenance [production]
00:38 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2126.codfw.wmnet with reason: Maintenance [production]
00:38 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2125 (T335845)', diff saved to https://phabricator.wikimedia.org/P47717 and previous config saved to /var/cache/conftool/dbconfig/20230505-003845-ladsgroup.json [production]
00:37 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1156 (T335845)', diff saved to https://phabricator.wikimedia.org/P47716 and previous config saved to /var/cache/conftool/dbconfig/20230505-003749-ladsgroup.json [production]
00:37 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on clouddb[1014,1018,1021].eqiad.wmnet,db1155.eqiad.wmnet with reason: Maintenance [production]
00:37 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 2 days, 0:00:00 on clouddb[1014,1018,1021].eqiad.wmnet,db1155.eqiad.wmnet with reason: Maintenance [production]
00:37 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1156.eqiad.wmnet with reason: Maintenance [production]
00:37 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1156.eqiad.wmnet with reason: Maintenance [production]
00:34 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1139.eqiad.wmnet with reason: Maintenance [production]
00:34 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1139.eqiad.wmnet with reason: Maintenance [production]
00:34 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1129 (T335845)', diff saved to https://phabricator.wikimedia.org/P47715 and previous config saved to /var/cache/conftool/dbconfig/20230505-003359-ladsgroup.json [production]
00:23 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2125', diff saved to https://phabricator.wikimedia.org/P47714 and previous config saved to /var/cache/conftool/dbconfig/20230505-002339-ladsgroup.json [production]
00:18 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1129', diff saved to https://phabricator.wikimedia.org/P47713 and previous config saved to /var/cache/conftool/dbconfig/20230505-001853-ladsgroup.json [production]
00:08 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2125', diff saved to https://phabricator.wikimedia.org/P47712 and previous config saved to /var/cache/conftool/dbconfig/20230505-000832-ladsgroup.json [production]
00:03 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1129', diff saved to https://phabricator.wikimedia.org/P47711 and previous config saved to /var/cache/conftool/dbconfig/20230505-000346-ladsgroup.json [production]
2023-05-04 §
23:53 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2125 (T335845)', diff saved to https://phabricator.wikimedia.org/P47710 and previous config saved to /var/cache/conftool/dbconfig/20230504-235326-ladsgroup.json [production]
23:48 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1129 (T335845)', diff saved to https://phabricator.wikimedia.org/P47709 and previous config saved to /var/cache/conftool/dbconfig/20230504-234840-ladsgroup.json [production]
23:45 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2125 (T335845)', diff saved to https://phabricator.wikimedia.org/P47708 and previous config saved to /var/cache/conftool/dbconfig/20230504-234544-ladsgroup.json [production]
23:45 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2125.codfw.wmnet with reason: Maintenance [production]
23:45 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2125.codfw.wmnet with reason: Maintenance [production]
23:45 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2104 (T335845)', diff saved to https://phabricator.wikimedia.org/P47707 and previous config saved to /var/cache/conftool/dbconfig/20230504-234520-ladsgroup.json [production]
23:43 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1129 (T335845)', diff saved to https://phabricator.wikimedia.org/P47706 and previous config saved to /var/cache/conftool/dbconfig/20230504-234330-ladsgroup.json [production]
23:43 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1129.eqiad.wmnet with reason: Maintenance [production]
23:43 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1129.eqiad.wmnet with reason: Maintenance [production]
23:43 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1122 (T335845)', diff saved to https://phabricator.wikimedia.org/P47705 and previous config saved to /var/cache/conftool/dbconfig/20230504-234306-ladsgroup.json [production]
23:30 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2104', diff saved to https://phabricator.wikimedia.org/P47704 and previous config saved to /var/cache/conftool/dbconfig/20230504-233013-ladsgroup.json [production]
23:28 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1122', diff saved to https://phabricator.wikimedia.org/P47703 and previous config saved to /var/cache/conftool/dbconfig/20230504-232800-ladsgroup.json [production]
23:15 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2104', diff saved to https://phabricator.wikimedia.org/P47702 and previous config saved to /var/cache/conftool/dbconfig/20230504-231507-ladsgroup.json [production]
23:12 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1122', diff saved to https://phabricator.wikimedia.org/P47701 and previous config saved to /var/cache/conftool/dbconfig/20230504-231254-ladsgroup.json [production]
23:00 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2104 (T335845)', diff saved to https://phabricator.wikimedia.org/P47700 and previous config saved to /var/cache/conftool/dbconfig/20230504-230001-ladsgroup.json [production]
22:57 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1122 (T335845)', diff saved to https://phabricator.wikimedia.org/P47699 and previous config saved to /var/cache/conftool/dbconfig/20230504-225747-ladsgroup.json [production]
22:53 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2104 (T335845)', diff saved to https://phabricator.wikimedia.org/P47698 and previous config saved to /var/cache/conftool/dbconfig/20230504-225336-ladsgroup.json [production]
22:53 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2104.codfw.wmnet with reason: Maintenance [production]
22:53 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2104.codfw.wmnet with reason: Maintenance [production]
22:50 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1122 (T335845)', diff saved to https://phabricator.wikimedia.org/P47697 and previous config saved to /var/cache/conftool/dbconfig/20230504-225013-ladsgroup.json [production]
22:50 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1122.eqiad.wmnet with reason: Maintenance [production]
22:49 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1122.eqiad.wmnet with reason: Maintenance [production]
22:49 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2097.codfw.wmnet with reason: Maintenance [production]
22:49 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2097.codfw.wmnet with reason: Maintenance [production]