5551-5600 of 10000 results (16ms)
2022-12-16 ยง
22:29 <bd808> Updated to 64a78ad (T325381) [tools.stashbot]
21:22 <wm-bot> <lucaswerkmeister> deployed 4802902384 (l10n updates: yue) [tools.lexeme-forms]
19:57 <andrewbogott> deleted instance 'diegoTest' because it had been stuck in scheduling state for 6 months [wmf-research-tools]
19:55 <robh@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ganeti4007.ulsfo.wmnet with OS bullseye [production]
19:55 <robh@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - robh@cumin2002" [production]
19:36 <volans> restarted sshd twice on bastion-restricted-eqiad1-02 to debug SSH connections for T319401 [admin]
19:00 <robh@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - robh@cumin2002" [production]
18:44 <robh@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ganeti4007.ulsfo.wmnet with reason: host reimage [production]
18:41 <robh@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on ganeti4007.ulsfo.wmnet with reason: host reimage [production]
18:19 <robh@cumin2002> START - Cookbook sre.hosts.reimage for host ganeti4007.ulsfo.wmnet with OS bullseye [production]
18:08 <robh@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['ganeti4007'] [production]
17:56 <robh@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['ganeti4007'] [production]
17:53 <robh@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['ganeti4007'] [production]
17:52 <robh@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['ganeti4007'] [production]
17:28 <mutante> etherpad1003 - testing new: sudo systemctl start wmf_auto_restart_envoyproxy [production]
16:46 <robh@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti4007.mgmt.ulsfo.wmnet with reboot policy FORCED [production]
16:24 <robh@cumin2002> START - Cookbook sre.hosts.provision for host ganeti4007.mgmt.ulsfo.wmnet with reboot policy FORCED [production]
16:24 <robh@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
16:23 <robh@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: gaenti4007 - robh@cumin2002" [production]
16:22 <robh@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: gaenti4007 - robh@cumin2002" [production]
16:20 <robh@cumin2002> START - Cookbook sre.dns.netbox [production]
16:19 <robh@cumin2002> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host ganeti4007 [production]
16:19 <robh@cumin2002> START - Cookbook sre.network.configure-switch-interfaces for host ganeti4007 [production]
16:08 <dcaro> removing coe cluster rook3 (T325373) [paws]
15:36 <xcollazo> deploying 'Fix subtle bug on image_suggestions when resolving varprop.' on platform_eng Airflow instance. [analytics]
15:34 <xcollazo@deploy1002> Finished deploy [airflow-dags/platform_eng@44d4e81]: Fix subtle bug on image_suggestions when resolving varprop. (duration: 00m 09s) [production]
15:34 <xcollazo@deploy1002> Started deploy [airflow-dags/platform_eng@44d4e81]: Fix subtle bug on image_suggestions when resolving varprop. [production]
13:48 <aborrero@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudgw2001-dev.codfw.wmnet with OS bullseye [production]
12:32 <jbond> deploy update to webproxy https://gerrit.wikimedia.org/r/c/operations/puppet/+/868372 [production]
12:09 <volans> run homer on cr[1-2]-{eqiad,codfw} to allow SSH from cloudcumin hosts to cloud hosts [production]
12:00 <ayounsi@cumin1001> END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'email' for AS: 28398 [production]
12:00 <ayounsi@cumin1001> START - Cookbook sre.network.peering with action 'email' for AS: 28398 [production]
11:19 <aborrero@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudgw2001-dev.codfw.wmnet with reason: host reimage [production]
11:16 <aborrero@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on cloudgw2001-dev.codfw.wmnet with reason: host reimage [production]
10:51 <aborrero@cumin2002> START - Cookbook sre.hosts.reimage for host cloudgw2001-dev.codfw.wmnet with OS bullseye [production]
09:44 <moritzm> import terraform 1.3.6 to thirdparty/terraform for buster/bullseye T322344 [production]
09:19 <jmm@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
09:18 <jmm@cumin2002> START - Cookbook sre.dns.netbox [production]
09:13 <jmm@cumin2002> END (FAIL) - Cookbook sre.hosts.decommission (exit_code=1) for hosts ganeti5003.eqsin.wmnet [production]
09:13 <jmm@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
09:13 <jmm@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: ganeti5003.eqsin.wmnet decommissioned, removing all IPs except the asset tag one - jmm@cumin2002" [production]
09:11 <jmm@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: ganeti5003.eqsin.wmnet decommissioned, removing all IPs except the asset tag one - jmm@cumin2002" [production]
08:53 <jmm@cumin2002> START - Cookbook sre.dns.netbox [production]
08:46 <dcaro> restart designate-sink on both cloudservice hosts (T322279) [admin]
08:45 <dcaro> restart designate-sink on both cloudservice hosts [admin]
08:45 <jayme@deploy1002> helmfile [staging] DONE helmfile.d/services/sessionstore: apply [production]
08:45 <jayme@deploy1002> helmfile [staging] START helmfile.d/services/sessionstore: apply [production]
08:45 <jmm@cumin2002> START - Cookbook sre.hosts.decommission for hosts ganeti5003.eqsin.wmnet [production]
08:43 <jayme@deploy1002> helmfile [staging] DONE helmfile.d/services/echostore: apply [production]
08:42 <jayme@deploy1002> helmfile [staging] START helmfile.d/services/echostore: apply [production]