|
2025-04-21
§
|
| 17:24 |
<jhancock@cumin2002> |
START - Cookbook sre.hosts.reimage for host ganeti2050.codfw.wmnet with OS bookworm |
[production] |
| 17:24 |
<bking@cumin2002> |
START - Cookbook sre.dns.netbox |
[production] |
| 17:24 |
<bking@cumin2002> |
START - Cookbook sre.hosts.move-vlan for host cirrussearch2094 |
[production] |
| 17:24 |
<bking@cumin2002> |
START - Cookbook sre.hosts.reimage for host cirrussearch2094.codfw.wmnet with OS bullseye |
[production] |
| 17:23 |
<bking@cumin2002> |
END (PASS) - Cookbook sre.hosts.rename (exit_code=0) from elastic2094 to cirrussearch2094 |
[production] |
| 17:23 |
<bking@cumin2002> |
END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cirrussearch2094 |
[production] |
| 17:23 |
<bking@cumin2002> |
START - Cookbook sre.network.configure-switch-interfaces for host cirrussearch2094 |
[production] |
| 17:23 |
<bking@cumin2002> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 17:22 |
<bking@cumin2002> |
END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Renaming elastic2094 to cirrussearch2094 - bking@cumin2002" |
[production] |
| 17:22 |
<AntiComposite> |
Hard reboot cvn-app10, flapping and not responsive to ssh |
[cvn] |
| 17:22 |
<bking@cumin2002> |
START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Renaming elastic2094 to cirrussearch2094 - bking@cumin2002" |
[production] |
| 17:18 |
<bking@cumin2002> |
START - Cookbook sre.dns.netbox |
[production] |
| 17:18 |
<bking@cumin2002> |
START - Cookbook sre.hosts.rename from elastic2094 to cirrussearch2094 |
[production] |
| 16:46 |
<wmbot~lucaswerkmeister@tools-bastion-13> |
webservice restart # clear out a stuck broken variable in toolforge-i18n |
[tools.wd-image-positions] |
| 16:43 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.ceph.osd.bootstrap_and_add |
[admin] |
| 16:31 |
<bking@cumin2002> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch2064.codfw.wmnet with OS bullseye |
[production] |
| 16:16 |
<andrew@cloudcumin1001> |
END (FAIL) - Cookbook wmcs.ceph.osd.bootstrap_and_add (exit_code=99) |
[admin] |
| 16:13 |
<urandom> |
decommissioning Cassandra/restbase1030-{a,b,c} — T389423 |
[production] |
| 16:12 |
<bking@cumin2002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cirrussearch2064.codfw.wmnet with reason: host reimage |
[production] |
| 16:11 |
<eevans@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 30 days, 0:00:00 on restbase1030.eqiad.wmnet with reason: Decommissioning — T378725 |
[production] |
| 16:10 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.ceph.osd.bootstrap_and_add |
[admin] |
| 16:09 |
<bking@cumin2002> |
START - Cookbook sre.hosts.downtime for 2:00:00 on cirrussearch2064.codfw.wmnet with reason: host reimage |
[production] |
| 16:03 |
<bking@cumin2002> |
conftool action : set/pooled=yes:weight=10; selector: name=cirrussearch2103.codfw.wmnet|cirrussearch2104.codfw.wmnet|cirrussearch2105.codfw.wmnet|cirrussearch2107.codfw.wmnet|cirrussearch2109.codfw.wmnet|cirrussearch2111.codfw.wmnet|cirrussearch2112.codfw.wmnet|cirrussearch2114.codfw.wmnet|cirrussearch2115.codfw.wmnet |
[production] |
| 16:00 |
<jhancock@cumin2002> |
END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti2050.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART |
[production] |
| 16:00 |
<bking@cumin2002> |
conftool action : set/weight=10:pooled=yes; selector: name=cirrussearch2097\.codfw\.wmnet |
[production] |
| 16:00 |
<bking@cumin2002> |
conftool action : set/weight=10:pooled=yes; selector: name=cirrussearch2091\.codfw\.wmnet |
[production] |
| 16:00 |
<bking@cumin2002> |
conftool action : set/weight=10:pooled=yes; selector: name=cirrussearch2090\.codfw\.wmnet |
[production] |
| 16:00 |
<bking@cumin2002> |
conftool action : set/weight=10:pooled=yes; selector: name=cirrussearch2089\.codfw\.wmnet |
[production] |
| 16:00 |
<bking@cumin2002> |
conftool action : set/weight=10:pooled=yes; selector: name=cirrussearch2088\.codfw\.wmnet |
[production] |
| 16:00 |
<bking@cumin2002> |
conftool action : set/weight=10:pooled=yes; selector: name=cirrussearch2087\.codfw\.wmnet |
[production] |
| 16:00 |
<bking@cumin2002> |
conftool action : set/weight=10:pooled=yes; selector: name=cirrussearch2085\.codfw\.wmnet |
[production] |
| 16:00 |
<bking@cumin2002> |
conftool action : set/weight=10:pooled=yes; selector: name=cirrussearch2082\.codfw\.wmnet |
[production] |
| 16:00 |
<bking@cumin2002> |
conftool action : set/weight=10:pooled=yes; selector: name=cirrussearch2079\.codfw\.wmnet |
[production] |
| 16:00 |
<bking@cumin2002> |
conftool action : set/weight=10:pooled=yes; selector: name=cirrussearch2077\.codfw\.wmnet |
[production] |
| 16:00 |
<bking@cumin2002> |
conftool action : set/weight=10:pooled=yes; selector: name=cirrussearch2075\.codfw\.wmnet |
[production] |
| 16:00 |
<bking@cumin2002> |
conftool action : set/weight=10:pooled=yes; selector: name=cirrussearch2074\.codfw\.wmnet |
[production] |
| 15:59 |
<bking@cumin2002> |
conftool action : set/weight=10:pooled=yes; selector: name=cirrussearch2072\.codfw\.wmnet |
[production] |
| 15:59 |
<bking@cumin2002> |
conftool action : set/weight=10:pooled=yes; selector: name=cirrussearch2070\.codfw\.wmnet |
[production] |
| 15:59 |
<bking@cumin2002> |
conftool action : set/weight=10:pooled=yes; selector: name=cirrussearch2069\.codfw\.wmnet |
[production] |
| 15:59 |
<bking@cumin2002> |
conftool action : set/weight=10:pooled=yes; selector: name=cirrussearch2068\.codfw\.wmnet |
[production] |
| 15:59 |
<bking@cumin2002> |
conftool action : set/weight=10:pooled=yes; selector: name=cirrussearch2067\.codfw\.wmnet |
[production] |
| 15:59 |
<bking@cumin2002> |
conftool action : set/weight=10:pooled=yes; selector: name=cirrussearch2066\.codfw\.wmnet |
[production] |
| 15:59 |
<bking@cumin2002> |
conftool action : set/weight=10:pooled=yes; selector: name=cirrussearch2065\.codfw\.wmnet |
[production] |
| 15:59 |
<bking@cumin2002> |
conftool action : set/weight=10:pooled=yes; selector: name=cirrussearch2063\.codfw\.wmnet |
[production] |
| 15:59 |
<bking@cumin2002> |
conftool action : set/weight=10:pooled=yes; selector: name=cirrussearch2062\.codfw\.wmnet |
[production] |
| 15:59 |
<bking@cumin2002> |
conftool action : set/weight=10:pooled=yes; selector: name=cirrussearch2060\.codfw\.wmnet |
[production] |
| 15:58 |
<bking@cumin2002> |
conftool action : set/weight=10:pooled=yes; selector: name=cirrussearch2059\.codfw\.wmnet |
[production] |
| 15:58 |
<bking@cumin2002> |
conftool action : set/weight=10:pooled=yes; selector: name=cirrussearch2058\.codfw\.wmnet |
[production] |
| 15:58 |
<bking@cumin2002> |
conftool action : set/weight=10:pooled=yes; selector: name=cirrussearch2057\.codfw\.wmnet |
[production] |
| 15:58 |
<bking@cumin2002> |
conftool action : set/weight=10:pooled=yes; selector: name=cirrussearch2056\.codfw\.wmnet |
[production] |