151-200 of 10000 results (92ms)
2024-11-07 ยง
18:58 <herron@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM aux-k8s-worker1004.eqiad.wmnet - herron@cumin1002" [production]
18:50 <herron@cumin1002> START - Cookbook sre.dns.netbox [production]
18:50 <herron@cumin1002> START - Cookbook sre.ganeti.makevm for new host aux-k8s-worker1004.eqiad.wmnet [production]
18:43 <jhancock@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
18:43 <jhancock@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2138 to codfw - jhancock@cumin2002" [production]
18:43 <jhancock@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: adding wikikube-worker2138 to codfw - jhancock@cumin2002" [production]
18:14 <swfrench-wmf> updated changeprop-jobqueue to 2024-11-05-170900-production - T356241 [production]
18:13 <swfrench@deploy2002> helmfile [codfw] DONE helmfile.d/services/changeprop-jobqueue: apply [production]
18:11 <swfrench@deploy2002> helmfile [codfw] START helmfile.d/services/changeprop-jobqueue: apply [production]
18:01 <swfrench@deploy2002> helmfile [eqiad] DONE helmfile.d/services/changeprop-jobqueue: apply [production]
17:59 <swfrench@deploy2002> helmfile [eqiad] START helmfile.d/services/changeprop-jobqueue: apply [production]
17:58 <swfrench@deploy2002> helmfile [staging] DONE helmfile.d/services/changeprop-jobqueue: apply [production]
17:57 <swfrench@deploy2002> helmfile [staging] START helmfile.d/services/changeprop-jobqueue: apply [production]
17:55 <fnegri@cumin1002> END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for cloudvirt1063.eqiad.wmnet [production]
17:55 <fnegri@cumin1002> START - Cookbook sre.hosts.remove-downtime for cloudvirt1063.eqiad.wmnet [production]
17:48 <swfrench@deploy2002> helmfile [codfw] DONE helmfile.d/services/changeprop: apply [production]
17:48 <swfrench@deploy2002> helmfile [codfw] START helmfile.d/services/changeprop: apply [production]
17:44 <swfrench@deploy2002> helmfile [eqiad] DONE helmfile.d/services/changeprop: apply [production]
17:43 <swfrench@deploy2002> helmfile [eqiad] START helmfile.d/services/changeprop: apply [production]
17:42 <swfrench@deploy2002> helmfile [staging] DONE helmfile.d/services/changeprop: apply [production]
17:41 <swfrench@deploy2002> helmfile [staging] START helmfile.d/services/changeprop: apply [production]
17:29 <fnegri@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt1063.eqiad.wmnet with OS bookworm [production]
17:29 <fnegri@cumin1002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - fnegri@cumin1002" [production]
17:27 <fnegri@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - fnegri@cumin1002" [production]
17:18 <cmooney@cumin1002> END (PASS) - Cookbook sre.network.tls (exit_code=0) for network device fasw2-c1a-eqiad [production]
17:16 <cmooney@cumin1002> START - Cookbook sre.network.tls for network device fasw2-c1a-eqiad [production]
17:12 <rzl> manually run mediawiki_job_wikimediaevents-UpdatePeriodicMetrics-global # T375508 [production]
17:09 <arlolra@deploy2002> helmfile [codfw] DONE helmfile.d/services/mobileapps: apply [production]
17:08 <arlolra@deploy2002> helmfile [codfw] START helmfile.d/services/mobileapps: apply [production]
17:06 <rzl> manually run mediawiki_job_wikimediaevents-UpdatePeriodicMetrics-per-wiki # T375508 [production]
17:03 <arlolra@deploy2002> helmfile [eqiad] DONE helmfile.d/services/mobileapps: apply [production]
17:02 <arlolra@deploy2002> helmfile [eqiad] START helmfile.d/services/mobileapps: apply [production]
17:01 <fnegri@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirt1063.eqiad.wmnet with reason: host reimage [production]
16:57 <elukey@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ms-be2082.codfw.wmnet with OS bullseye [production]
16:57 <elukey@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - elukey@cumin2002" [production]
16:57 <elukey@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host ms-be2084.codfw.wmnet with OS bullseye [production]
16:57 <arlolra@deploy2002> helmfile [codfw] DONE helmfile.d/services/mobileapps: apply [production]
16:56 <arlolra@deploy2002> helmfile [codfw] START helmfile.d/services/mobileapps: apply [production]
16:56 <arlolra@deploy2002> helmfile [eqiad] DONE helmfile.d/services/mobileapps: apply [production]
16:56 <fnegri@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on cloudvirt1063.eqiad.wmnet with reason: host reimage [production]
16:54 <arlolra@deploy2002> helmfile [eqiad] START helmfile.d/services/mobileapps: apply [production]
16:54 <elukey@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ms-be2083.codfw.wmnet with OS bullseye [production]
16:48 <elukey@cumin1002> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host fransc1001.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED [production]
16:48 <elukey@cumin1002> START - Cookbook sre.hosts.provision for host fransc1001.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED [production]
16:46 <elukey@cumin2002> START - Cookbook sre.hosts.reimage for host ms-be2084.codfw.wmnet with OS bullseye [production]
16:45 <elukey@cumin1002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ms-be2084.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART [production]
16:41 <fnegri@cumin1002> START - Cookbook sre.hosts.reimage for host cloudvirt1063.eqiad.wmnet with OS bookworm [production]
16:34 <elukey@cumin1002> START - Cookbook sre.hosts.provision for host ms-be2084.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART [production]
16:32 <elukey@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ms-be2083.codfw.wmnet with reason: host reimage [production]
16:28 <elukey@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - elukey@cumin2002" [production]