7101-7150 of 10000 results (104ms)
2024-04-27 §
19:55 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2166', diff saved to https://phabricator.wikimedia.org/P61278 and previous config saved to /var/cache/conftool/dbconfig/20240427-195524-ladsgroup.json [production]
19:46 <cmooney@cumin1002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
19:46 <cmooney@cumin1002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add DNS for magru PDUs - cmooney@cumin1002" [production]
19:45 <cmooney@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add DNS for magru PDUs - cmooney@cumin1002" [production]
19:43 <cmooney@cumin1002> START - Cookbook sre.dns.netbox [production]
19:40 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2166 (T352010)', diff saved to https://phabricator.wikimedia.org/P61277 and previous config saved to /var/cache/conftool/dbconfig/20240427-194017-ladsgroup.json [production]
19:39 <cmooney@cumin1002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
19:39 <cmooney@cumin1002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add DNS for magru PDUs - cmooney@cumin1002" [production]
19:38 <cmooney@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add DNS for magru PDUs - cmooney@cumin1002" [production]
19:34 <cmooney@cumin1002> START - Cookbook sre.dns.netbox [production]
17:55 <ayounsi@cumin1002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
17:54 <ayounsi@cumin1002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: magru oob - ayounsi@cumin1002" [production]
17:42 <ayounsi@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: magru oob - ayounsi@cumin1002" [production]
17:39 <ayounsi@cumin1002> START - Cookbook sre.dns.netbox [production]
17:06 <ayounsi@cumin1002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
17:06 <ayounsi@cumin1002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: magru - ayounsi@cumin1002" [production]
17:03 <ayounsi@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: magru - ayounsi@cumin1002" [production]
17:01 <ayounsi@cumin1002> START - Cookbook sre.dns.netbox [production]
14:47 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db2212 (T352010)', diff saved to https://phabricator.wikimedia.org/P61275 and previous config saved to /var/cache/conftool/dbconfig/20240427-144642-ladsgroup.json [production]
14:46 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2212.codfw.wmnet with reason: Maintenance [production]
14:46 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2212.codfw.wmnet with reason: Maintenance [production]
09:51 <volans> manually upgraded wmflib in netbox1002/2002's Netbox's venv [production]
08:58 <volans> restarted uwsgi on netbox1002 to pickup the latest wmflib with magru [production]
07:52 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db2166 (T352010)', diff saved to https://phabricator.wikimedia.org/P61274 and previous config saved to /var/cache/conftool/dbconfig/20240427-075233-ladsgroup.json [production]
07:52 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2166.codfw.wmnet with reason: Maintenance [production]
07:52 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2166.codfw.wmnet with reason: Maintenance [production]
07:52 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2164 (T352010)', diff saved to https://phabricator.wikimedia.org/P61273 and previous config saved to /var/cache/conftool/dbconfig/20240427-075210-ladsgroup.json [production]
07:43 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2202.codfw.wmnet with reason: Maintenance [production]
07:43 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2202.codfw.wmnet with reason: Maintenance [production]
07:42 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2188 (T352010)', diff saved to https://phabricator.wikimedia.org/P61272 and previous config saved to /var/cache/conftool/dbconfig/20240427-074250-ladsgroup.json [production]
07:37 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2164', diff saved to https://phabricator.wikimedia.org/P61271 and previous config saved to /var/cache/conftool/dbconfig/20240427-073703-ladsgroup.json [production]
07:27 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2188', diff saved to https://phabricator.wikimedia.org/P61270 and previous config saved to /var/cache/conftool/dbconfig/20240427-072742-ladsgroup.json [production]
07:21 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2164', diff saved to https://phabricator.wikimedia.org/P61269 and previous config saved to /var/cache/conftool/dbconfig/20240427-072155-ladsgroup.json [production]
07:12 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2188', diff saved to https://phabricator.wikimedia.org/P61268 and previous config saved to /var/cache/conftool/dbconfig/20240427-071235-ladsgroup.json [production]
07:06 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2164 (T352010)', diff saved to https://phabricator.wikimedia.org/P61267 and previous config saved to /var/cache/conftool/dbconfig/20240427-070648-ladsgroup.json [production]
06:57 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2188 (T352010)', diff saved to https://phabricator.wikimedia.org/P61266 and previous config saved to /var/cache/conftool/dbconfig/20240427-065728-ladsgroup.json [production]
00:51 <urandom> rebooting puppetserver1001.eqiad.wmnet via drac [production]
2024-04-26 §
23:13 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db2188 (T352010)', diff saved to https://phabricator.wikimedia.org/P61265 and previous config saved to /var/cache/conftool/dbconfig/20240426-231316-ladsgroup.json [production]
23:13 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2188.codfw.wmnet with reason: Maintenance [production]
23:13 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2188.codfw.wmnet with reason: Maintenance [production]
23:12 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2176 (T352010)', diff saved to https://phabricator.wikimedia.org/P61264 and previous config saved to /var/cache/conftool/dbconfig/20240426-231252-ladsgroup.json [production]
22:57 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2176', diff saved to https://phabricator.wikimedia.org/P61263 and previous config saved to /var/cache/conftool/dbconfig/20240426-225744-ladsgroup.json [production]
22:42 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2176', diff saved to https://phabricator.wikimedia.org/P61262 and previous config saved to /var/cache/conftool/dbconfig/20240426-224235-ladsgroup.json [production]
22:27 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2176 (T352010)', diff saved to https://phabricator.wikimedia.org/P61261 and previous config saved to /var/cache/conftool/dbconfig/20240426-222728-ladsgroup.json [production]
22:24 <dzahn@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host lists2001.wikimedia.org with OS bookworm [production]
22:07 <dzahn@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on lists2001.wikimedia.org with reason: host reimage [production]
22:04 <dzahn@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on lists2001.wikimedia.org with reason: host reimage [production]
21:43 <dzahn@cumin2002> START - Cookbook sre.hosts.reimage for host lists2001.wikimedia.org with OS bookworm [production]
21:38 <dzahn@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host lists2001.wikimedia.org with OS bullseye [production]
21:38 <dzahn@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - dzahn@cumin2002" [production]