651-700 of 10000 results (27ms)
2024-10-01 ยง
10:23 <elukey@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host dbproxy2005.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
10:23 <elukey@cumin2002> START - Cookbook sre.hosts.provision for host dbproxy2005.mgmt.codfw.wmnet with chassis set policy GRACEFUL_RESTART [production]
10:21 <elukey@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host dbproxy1029.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
10:21 <elukey@cumin2002> START - Cookbook sre.hosts.provision for host dbproxy1029.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
10:17 <wmbot~dcaro@urcuchillay> START - Cookbook wmcs.toolforge.component.deploy for component api-gateway [toolsbeta]
10:17 <elukey@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host dbproxy1028.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
10:17 <elukey@cumin2002> START - Cookbook sre.hosts.provision for host dbproxy1028.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
10:16 <elukey@cumin1002> START - Cookbook sre.hosts.reimage for host irc2003.wikimedia.org with OS bookworm [production]
10:15 <elukey@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host an-conf1006.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
10:15 <elukey@cumin1002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM irc2003.wikimedia.org - elukey@cumin1002" [production]
10:15 <elukey@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM irc2003.wikimedia.org - elukey@cumin1002" [production]
10:15 <elukey@cumin2002> START - Cookbook sre.hosts.provision for host an-conf1006.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
10:15 <elukey@cumin1002> END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) irc2003.wikimedia.org on all recursors [production]
10:15 <elukey@cumin1002> START - Cookbook sre.dns.wipe-cache irc2003.wikimedia.org on all recursors [production]
10:15 <elukey@cumin1002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
10:15 <elukey@cumin1002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM irc2003.wikimedia.org - elukey@cumin1002" [production]
10:15 <elukey@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM irc2003.wikimedia.org - elukey@cumin1002" [production]
10:13 <wmbot~dcaro@urcuchillay> END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component api-gateway [toolsbeta]
10:13 <elukey@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host an-conf1005.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
10:13 <elukey@cumin2002> START - Cookbook sre.hosts.provision for host an-conf1005.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
10:11 <elukey@cumin1002> START - Cookbook sre.dns.netbox [production]
10:11 <elukey@cumin1002> START - Cookbook sre.ganeti.makevm for new host irc2003.wikimedia.org [production]
10:08 <wmbot~dcaro@urcuchillay> START - Cookbook wmcs.toolforge.component.deploy for component api-gateway [toolsbeta]
10:07 <elukey@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host an-conf1004.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
10:06 <elukey@cumin2002> START - Cookbook sre.hosts.provision for host an-conf1004.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
10:06 <dcaro@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component api-gateway [toolsbeta]
10:04 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host irc1003.wikimedia.org [production]
10:02 <elukey@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host an-conf1004.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
10:01 <elukey@cumin2002> START - Cookbook sre.hosts.provision for host an-conf1004.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
10:00 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host irc1003.wikimedia.org [production]
10:00 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component api-gateway [toolsbeta]
09:59 <elukey@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host an-conf1004.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
09:57 <elukey@cumin2002> START - Cookbook sre.hosts.provision for host an-conf1004.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
09:57 <aborrero@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.tofu (exit_code=0) running tofu plan+apply for main branch [admin]
09:53 <aborrero@cloudcumin1001> START - Cookbook wmcs.openstack.tofu running tofu plan+apply for main branch [admin]
09:52 <dcaro@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component ingress-admission [tools]
09:47 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component ingress-admission [tools]
09:40 <dcaro@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component ingress-admission [toolsbeta]
09:34 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component ingress-admission [toolsbeta]
09:24 <jmm@deploy2002> Finished scap sync-world: Backport for [[gerrit:1050261|Remove irc1001/irc2001 from mediawiki-config and add irc1003 (T331702 T376014)]] (duration: 08m 07s) [production]
09:19 <jmm@deploy2002> jmm: Continuing with sync [production]
09:19 <fnegri@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 14 days, 0:00:00 on cloudvirt1063.eqiad.wmnet with reason: cloudvirt1063 needs maintenance T375223 [production]
09:18 <fnegri@cumin1002> START - Cookbook sre.hosts.downtime for 14 days, 0:00:00 on cloudvirt1063.eqiad.wmnet with reason: cloudvirt1063 needs maintenance T375223 [production]
09:18 <jmm@deploy2002> jmm: Backport for [[gerrit:1050261|Remove irc1001/irc2001 from mediawiki-config and add irc1003 (T331702 T376014)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
09:15 <jmm@deploy2002> Started scap sync-world: Backport for [[gerrit:1050261|Remove irc1001/irc2001 from mediawiki-config and add irc1003 (T331702 T376014)]] [production]
09:11 <wmbot~jeanfred@tools-sgebastion-10> Deploy afb66f1, a2be205, 47e99e3 [tools.wikiloves]
09:07 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db1177 (T367856)', diff saved to https://phabricator.wikimedia.org/P69437 and previous config saved to /var/cache/conftool/dbconfig/20241001-090708-ladsgroup.json [production]
09:07 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 12:00:00 on db1177.eqiad.wmnet with reason: Maintenance [production]
09:06 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 2 days, 12:00:00 on db1177.eqiad.wmnet with reason: Maintenance [production]
09:06 <ladsgroup@cumin1002> END (ERROR) - Cookbook sre.hosts.downtime (exit_code=97) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance [production]