1-50 of 10000 results (52ms)
2026-09-01 ยง
19:45 <eevans@cumin1003> START - Cookbook sre.hosts.reimage for host aqs1016.eqiad.wmnet with OS bookworm [production]
19:42 <eevans@cumin1003> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host aqs1016.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL [production]
19:41 <jhancock@cumin1003> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudcephosd1055.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
19:40 <eevans@cumin1003> START - Cookbook sre.hosts.provision for host aqs1016.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL [production]
19:39 <eevans@cumin1003> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host aqs1016.eqiad.wmnet with OS bookworm [production]
19:35 <jhancock@cumin1003> START - Cookbook sre.hosts.provision for host cloudcephosd1055.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
19:32 <jhancock@cumin1003> END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host cloudcephosd1055 [production]
19:32 <jhancock@cumin1003> START - Cookbook sre.network.configure-switch-interfaces for host cloudcephosd1055 [production]
19:30 <cdobbins@cumin1003> conftool action : set/pooled=yes; selector: name=cp5022.* [reason: update IP addrs] [production]
19:30 <cdobbins@cumin1003> END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for cp5022.eqsin.wmnet [production]
19:30 <cdobbins@cumin1003> START - Cookbook sre.hosts.remove-downtime for cp5022.eqsin.wmnet [production]
19:25 <jclark@cumin1003> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudcephosd1055.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
19:25 <jclark@cumin1003> START - Cookbook sre.hosts.provision for host cloudcephosd1055.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
19:25 <jclark@cumin1003> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudcephosd1055.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
19:25 <jclark@cumin1003> START - Cookbook sre.hosts.provision for host cloudcephosd1055.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
19:24 <jclark@cumin1003> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudcephosd1055.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
19:24 <jclark@cumin1003> START - Cookbook sre.hosts.provision for host cloudcephosd1055.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
19:23 <jclark@cumin1003> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudcephosd1056.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
19:22 <jclark@cumin1003> START - Cookbook sre.hosts.provision for host cloudcephosd1056.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
19:21 <eevans@cumin1003> START - Cookbook sre.hosts.reimage for host aqs1016.eqiad.wmnet with OS bookworm [production]
19:16 <jclark@cumin1003> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudcephosd1056.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
19:14 <jclark@cumin1003> START - Cookbook sre.hosts.provision for host cloudcephosd1056.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
19:14 <jclark@cumin1003> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host cloudcephosd1055.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
19:13 <jclark@cumin1003> START - Cookbook sre.hosts.provision for host cloudcephosd1055.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
19:12 <jclark@cumin1003> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudcephosd1056 [production]
19:12 <jclark@cumin1003> START - Cookbook sre.network.configure-switch-interfaces for host cloudcephosd1056 [production]
19:12 <jclark@cumin1003> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudcephosd1055 [production]
19:12 <jclark@cumin1003> START - Cookbook sre.network.configure-switch-interfaces for host cloudcephosd1055 [production]
19:12 <jclark@cumin1003> END (ERROR) - Cookbook sre.network.configure-switch-interfaces (exit_code=97) for host cloudcephosd1054 [production]
19:12 <jclark@cumin1003> START - Cookbook sre.network.configure-switch-interfaces for host cloudcephosd1054 [production]
19:11 <jclark@cumin1003> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
19:11 <jclark@cumin1003> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: adding cloudcephosd1055 to eqiad - jclark@cumin1003" [production]
19:11 <jclark@cumin1003> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: adding cloudcephosd1055 to eqiad - jclark@cumin1003" [production]
19:06 <jclark@cumin1003> START - Cookbook sre.dns.netbox [production]
19:06 <sukhe@dns1004> END - running authdns-update [production]
19:03 <sukhe@dns1004> START - running authdns-update [production]
18:14 <dancy@deploy1003> rebuilt and synchronized wikiversions files: group0 to 1.47.0-wmf.18 refs T430837 [production]
17:04 <bking@cumin2003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on matomo1003.eqiad.wmnet with reason: Matomo version upgrade T431608 [production]
17:00 <dancy@deploy1003> Finished scap sync-world: testing (duration: 08m 07s) [production]
16:52 <dancy@deploy1003> Started scap sync-world: testing [production]
16:48 <dancy@deploy1003> sync-world aborted: testing (duration: 00m 05s) [production]
16:48 <dancy@deploy1003> Started scap sync-world: testing [production]
16:47 <dancy@deploy1003> Installation of scap version "4.287.0" completed for 156 hosts [production]
16:42 <dancy@deploy1003> Installing scap version "4.287.0" for 156 host(s) [production]
16:24 <atsuko@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/blunderbuss: apply [production]
16:23 <atsuko@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/blunderbuss: apply [production]
15:51 <moritzm> installing mesa security updates [production]
15:29 <aokoth@cumin1003> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts phab1004.eqiad.wmnet [production]
15:29 <aokoth@cumin1003> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
15:29 <aokoth@cumin1003> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: phab1004.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - aokoth@cumin1003" [production]