1-50 of 10000 results (20ms)
2023-01-14 §
21:45 <AntiComposite> move cvn-clerkbot from cvn-app9 to cvn-app12 (deploy 4cee27a) [cvn]
21:22 <AntiComposite> move cvn-clerbot back to cvn-app9 (deploy 371ba2a) [cvn]
21:10 <AntiComposite> move cvn-clerkbot from cvn-app9 to cvn-app12 (deploy 3f3f40f) [cvn]
09:46 <godog> issue 'request system reboot member 2' - T327001 [production]
09:20 <mvernon@cumin2002> conftool action : set/pooled=no; selector: name=thanos-fe2002.codfw.wmnet [production]
09:19 <Emperor> depool thanos-fe2002 T327001 [production]
09:19 <mvernon@cumin2002> conftool action : set/pooled=no; selector: name=ms-fe2010.codfw.wmnet [production]
09:19 <Emperor> depool ms-fe2010 T327001 [production]
2023-01-13 §
23:39 <mutante> people2002 - systemctl reset-failed after removing auto_restart_rsync timers [production]
22:26 <mutante> mirror1001 - systemctl start update-ubuntu-mirror (sometimes sync fails) [production]
20:58 <pt1979@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=False) upgrade firmware for hosts ['druid1011'] [production]
20:58 <pt1979@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['druid1011'] [production]
20:56 <pt1979@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=True) upgrade firmware for hosts ['druid1011'] [production]
20:49 <pt1979@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['druid1011'] [production]
20:47 <pt1979@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts ['druid1011'] [production]
20:37 <pt1979@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['druid1011'] [production]
20:37 <pt1979@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=False) upgrade firmware for hosts ['druid1010'] [production]
20:36 <pt1979@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['druid1010'] [production]
20:36 <pt1979@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=True) upgrade firmware for hosts ['druid1010'] [production]
20:35 <pt1979@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=False) upgrade firmware for hosts ['druid1009'] [production]
20:35 <pt1979@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['druid1009'] [production]
20:35 <pt1979@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=True) upgrade firmware for hosts ['druid1009'] [production]
20:16 <pt1979@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['druid1010'] [production]
20:16 <pt1979@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['druid1009'] [production]
20:04 <dzahn@cumin2002> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host aphlict2001.codfw.wmnet [production]
19:58 <pt1979@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-mariadb1002.eqiad.wmnet with OS bullseye [production]
19:58 <pt1979@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - pt1979@cumin2002" [production]
19:54 <dzahn@cumin2002> END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) aphlict2001.codfw.wmnet on all recursors [production]
19:54 <dzahn@cumin2002> START - Cookbook sre.dns.wipe-cache aphlict2001.codfw.wmnet on all recursors [production]
19:54 <dzahn@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
19:54 <dzahn@cumin2002> END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM aphlict2001.codfw.wmnet - dzahn@cumin2002" [production]
19:52 <dzahn@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM aphlict2001.codfw.wmnet - dzahn@cumin2002" [production]
19:51 <pt1979@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - pt1979@cumin2002" [production]
19:49 <dzahn@cumin2002> START - Cookbook sre.dns.netbox [production]
19:49 <dzahn@cumin2002> START - Cookbook sre.ganeti.makevm for new host aphlict2001.codfw.wmnet [production]
19:40 <pt1979@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-mariadb1001.eqiad.wmnet with OS bullseye [production]
19:40 <pt1979@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - pt1979@cumin2002" [production]
19:38 <pt1979@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - pt1979@cumin2002" [production]
19:37 <pt1979@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-mariadb1002.eqiad.wmnet with reason: host reimage [production]
19:34 <pt1979@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on an-mariadb1002.eqiad.wmnet with reason: host reimage [production]
19:26 <pt1979@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-mariadb1001.eqiad.wmnet with reason: host reimage [production]
19:22 <pt1979@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on an-mariadb1001.eqiad.wmnet with reason: host reimage [production]
19:22 <pt1979@cumin2002> START - Cookbook sre.hosts.reimage for host an-mariadb1002.eqiad.wmnet with OS bullseye [production]
19:20 <wm-bot> <samtar> Recreated refill-api deployment [tools.refill-api]
18:25 <zabe> mwscript extensions/GlobalBlocking/maintenance/FixBlockerUsername.php --wiki metawiki "Green Giant" "Cromium" # T298707 [production]
17:36 <wm-bot2> Restarting openstack services on cloudcontrol2005-dev: ['nova-conductor', 'nova-scheduler', 'nova-api', 'nova-api-metadata'] - cookbook ran by andrew@bullseye [admin]
17:36 <wm-bot2> Restarting openstack services on cloudvirt2003-dev: ['nova-compute'] - cookbook ran by andrew@bullseye [admin]
17:36 <wm-bot2> Restarting openstack services on cloudvirt2002-dev: ['nova-compute'] - cookbook ran by andrew@bullseye [admin]
17:35 <wm-bot2> Restarting openstack services on cloudvirt2001-dev: ['nova-compute'] - cookbook ran by andrew@bullseye [admin]
17:35 <wm-bot2> Restarting openstack services on cloudcontrol2004-dev: ['nova-conductor', 'nova-scheduler', 'nova-api', 'nova-api-metadata'] - cookbook ran by andrew@bullseye [admin]