|
2026-03-13
ยง
|
| 21:40 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.openstack.cloudvirt.safe_reboot on hosts matched by 'D{cloudvirt1061.eqiad.wmnet}' (T419948) |
[admin] |
| 21:40 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.openstack.cloudvirt.safe_reboot (exit_code=0) on hosts matched by 'D{cloudvirt1062.eqiad.wmnet}' (T419948) |
[admin] |
| 21:21 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.openstack.cloudvirt.safe_reboot on hosts matched by 'D{cloudvirt1062.eqiad.wmnet}' (T419948) |
[admin] |
| 21:21 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.openstack.cloudvirt.safe_reboot (exit_code=0) on hosts matched by 'D{cloudvirt1063.eqiad.wmnet}' (T419948) |
[admin] |
| 21:03 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.openstack.cloudvirt.safe_reboot on hosts matched by 'D{cloudvirt1063.eqiad.wmnet}' (T419948) |
[admin] |
| 21:03 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.openstack.cloudvirt.safe_reboot (exit_code=0) on hosts matched by 'D{cloudvirt1064.eqiad.wmnet}' (T419948) |
[admin] |
| 20:47 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.openstack.cloudvirt.safe_reboot on hosts matched by 'D{cloudvirt1064.eqiad.wmnet}' (T419948) |
[admin] |
| 20:47 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.openstack.cloudvirt.safe_reboot (exit_code=0) on hosts matched by 'D{cloudvirt1065.eqiad.wmnet}' (T419948) |
[admin] |
| 20:37 |
<andrew@cloudcumin1001> |
END (FAIL) - Cookbook wmcs.vps.instance.stop_start (exit_code=99) vm zuul-k8s-v128-qnv6fefra4wd-node-2 (cluster eqiad1) |
[zuul] |
| 20:37 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.vps.instance.stop_start vm zuul-k8s-v128-qnv6fefra4wd-node-2 (cluster eqiad1) |
[zuul] |
| 20:33 |
<wmbot~jeanfred@tools-bastion-15> |
Deploy a39f1d9 (Add proper handling for transient Wikidata server errors) for T415439 |
[tools.integraality] |
| 20:31 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.openstack.cloudvirt.safe_reboot on hosts matched by 'D{cloudvirt1065.eqiad.wmnet}' (T419948) |
[admin] |
| 20:31 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.openstack.cloudvirt.safe_reboot (exit_code=0) on hosts matched by 'D{cloudvirt1066.eqiad.wmnet}' (T419948) |
[admin] |
| 20:27 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.vps.instance.stop_start (exit_code=0) vm paws-127c-uwce57bvcgrt-node-3 (cluster eqiad1) |
[paws] |
| 20:26 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.vps.instance.stop_start vm paws-127c-uwce57bvcgrt-node-3 (cluster eqiad1) |
[paws] |
| 20:19 |
<wmbot~jeanfred@tools-bastion-15> |
Deploy 0c418a5 (Fix tests hitting live Wikidata API (and potential rate-limits)) for T420054 |
[tools.integraality] |
| 20:13 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.openstack.cloudvirt.safe_reboot on hosts matched by 'D{cloudvirt1066.eqiad.wmnet}' (T419948) |
[admin] |
| 20:13 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.openstack.cloudvirt.safe_reboot (exit_code=0) on hosts matched by 'D{cloudvirt1067.eqiad.wmnet}' (T419948) |
[admin] |
| 20:08 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.vps.instance.stop_start (exit_code=0) vm phi-syslog-01 (cluster eqiad1) |
[o11y] |
| 20:08 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.vps.instance.stop_start vm phi-syslog-01 (cluster eqiad1) |
[o11y] |
| 20:07 |
<andrew@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudservices2004-dev.codfw.wmnet |
[production] |
| 20:06 |
<andrewbogott> |
restarting phi-syslog-01 to fix incorrect MTU |
[o11y] |
| 20:05 |
<andrewbogott> |
restarting k3s-envDB to fix incorrect MTU |
[catalyst-dev] |
| 20:01 |
<andrew@cloudcumin1001> |
END (FAIL) - Cookbook wmcs.vps.instance.stop_start (exit_code=99) vm zuul-k8s-v128-qnv6fefra4wd-node-9 (cluster eqiad1) |
[zuul] |
| 20:01 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.vps.instance.stop_start vm zuul-k8s-v128-qnv6fefra4wd-node-9 (cluster eqiad1) |
[zuul] |
| 20:01 |
<andrewbogott> |
stop/starting several k8s nodes to fix incorrect MTU |
[zuul] |
| 20:01 |
<andrew@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host cloudservices2004-dev.codfw.wmnet |
[production] |
| 20:01 |
<andrew@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudservices2004-dev.codfw.wmnet |
[production] |
| 20:00 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.vps.instance.stop_start (exit_code=0) vm paws-127c-uwce57bvcgrt-master-0 (cluster eqiad1) |
[paws] |
| 20:00 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.vps.instance.stop_start vm paws-127c-uwce57bvcgrt-master-0 (cluster eqiad1) |
[paws] |
| 19:55 |
<brett@puppetserver1001> |
conftool action : set/pooled=yes; selector: name=cp4052.* |
[production] |
| 19:54 |
<andrew@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host cloudservices2004-dev.codfw.wmnet |
[production] |
| 19:54 |
<brett@cumin2002> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp4052.ulsfo.wmnet with OS trixie |
[production] |
| 19:53 |
<andrew@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudservices2005-dev.codfw.wmnet |
[production] |
| 19:46 |
<andrew@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host cloudservices2005-dev.codfw.wmnet |
[production] |
| 19:46 |
<andrew@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudservices2004-dev.codfw.wmnet |
[production] |
| 19:40 |
<brett@puppetserver1001> |
conftool action : set/pooled=yes; selector: name=cp4050.* |
[production] |
| 19:40 |
<andrew@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host cloudservices2004-dev.codfw.wmnet |
[production] |
| 19:34 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.openstack.cloudvirt.safe_reboot on hosts matched by 'D{cloudvirt1067.eqiad.wmnet}' (T419948) |
[admin] |
| 19:34 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.openstack.cloudvirt.safe_reboot (exit_code=0) on hosts matched by 'D{cloudvirt1068.eqiad.wmnet}' (T419948) |
[admin] |
| 19:29 |
<brett@cumin2002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cp4052.ulsfo.wmnet with reason: host reimage |
[production] |
| 19:28 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.openstack.cloudvirt.safe_reboot (exit_code=0) on hosts matched by 'D{cloudvirt2004-dev.codfw.wmnet}' (T419948) |
[admin] |
| 19:24 |
<brett@cumin2002> |
START - Cookbook sre.hosts.downtime for 2:00:00 on cp4052.ulsfo.wmnet with reason: host reimage |
[production] |
| 19:23 |
<brett@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cp4050.ulsfo.wmnet |
[production] |
| 19:19 |
<jclark@cumin1003> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wdqs1035.eqiad.wmnet with OS trixie |
[production] |
| 19:19 |
<jclark@cumin1003> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wdqs1034.eqiad.wmnet with OS trixie |
[production] |
| 19:18 |
<jclark@cumin1003> |
END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1003" |
[production] |
| 19:18 |
<jclark@cumin1003> |
END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1003" |
[production] |
| 19:18 |
<jclark@cumin1003> |
START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1003" |
[production] |
| 19:16 |
<brett@puppetserver1001> |
conftool action : set/pooled=yes; selector: name=cp4051.* |
[production] |