|
2024-04-17
§
|
| 02:48 |
<ryankemper> |
T361525 Trying to powercycle `elastic2088` thru mgmt port (host not responding to ssh) |
[production] |
| 02:43 |
<dani@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/miscweb: apply |
[production] |
| 02:43 |
<dani@deploy1002> |
helmfile [codfw] START helmfile.d/services/miscweb: apply |
[production] |
| 02:43 |
<dani@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/miscweb: apply |
[production] |
| 02:43 |
<dani@deploy1002> |
helmfile [eqiad] START helmfile.d/services/miscweb: apply |
[production] |
| 02:43 |
<dani@deploy1002> |
helmfile [staging] DONE helmfile.d/services/miscweb: apply |
[production] |
| 02:42 |
<dani@deploy1002> |
helmfile [staging] START helmfile.d/services/miscweb: apply |
[production] |
| 01:58 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) |
[admin] |
| 01:54 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.openstack.restart_openstack |
[admin] |
| 01:48 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.openstack.cloudvirt.live_upgrade_openstack (exit_code=0) on host 'cloudvirt2006-dev.codfw.wmnet' (T356287) |
[admin] |
| 01:43 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.openstack.cloudvirt.live_upgrade_openstack on host 'cloudvirt2006-dev.codfw.wmnet' (T356287) |
[admin] |
| 01:40 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.openstack.cloudvirt.live_upgrade_openstack (exit_code=0) on host 'cloudvirt2005-dev.codfw.wmnet' (T356287) |
[admin] |
| 01:35 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.openstack.cloudvirt.live_upgrade_openstack on host 'cloudvirt2005-dev.codfw.wmnet' (T356287) |
[admin] |
| 01:05 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.openstack.cloudvirt.live_upgrade_openstack (exit_code=0) on host 'cloudvirt2004-dev.codfw.wmnet' (T356287) |
[admin] |
| 01:00 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.openstack.cloudvirt.live_upgrade_openstack on host 'cloudvirt2004-dev.codfw.wmnet' (T356287) |
[admin] |
|
2024-04-16
§
|
| 23:45 |
<wmbot~anticomposite@tools-sgebastion-10> |
./stewardbots/StewardBot/manage.sh restart # RC reader not reading RC |
[tools.stewardbots] |
| 23:25 |
<hmonroy@deploy1002> |
Finished scap: Backport for [[gerrit:1019893|[mediawikiwiki] enable CodeMirror V6 (T357795)]] (duration: 17m 29s) |
[production] |
| 23:12 |
<hmonroy@deploy1002> |
musikanimal and hmonroy: Continuing with sync |
[production] |
| 23:11 |
<hmonroy@deploy1002> |
musikanimal and hmonroy: Backport for [[gerrit:1019893|[mediawikiwiki] enable CodeMirror V6 (T357795)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 23:08 |
<hmonroy@deploy1002> |
Started scap: Backport for [[gerrit:1019893|[mediawikiwiki] enable CodeMirror V6 (T357795)]] |
[production] |
| 23:06 |
<pt1979@cumin2002> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudcontrol2009-dev.codfw.wmnet with OS bookworm |
[production] |
| 23:06 |
<pt1979@cumin2002> |
END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - pt1979@cumin2002" |
[production] |
| 23:03 |
<pt1979@cumin2002> |
START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - pt1979@cumin2002" |
[production] |
| 22:46 |
<pt1979@cumin2002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudcontrol2009-dev.codfw.wmnet with reason: host reimage |
[production] |
| 22:43 |
<pt1979@cumin2002> |
START - Cookbook sre.hosts.downtime for 2:00:00 on cloudcontrol2009-dev.codfw.wmnet with reason: host reimage |
[production] |
| 22:32 |
<JJMC89> |
`update diffs set status=0, status_user_text=NULL where status_user_text='WizardGamer775';` # mass unreview |
[copypatrol] |
| 22:25 |
<pt1979@cumin2002> |
START - Cookbook sre.hosts.reimage for host cloudcontrol2009-dev.codfw.wmnet with OS bookworm |
[production] |
| 21:54 |
<pt1979@cumin2002> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudcontrol2009-dev.codfw.wmnet with OS bookworm |
[production] |
| 21:52 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.openstack.cloudcontrol.upgrade_openstack_node (exit_code=0) on host 'cloudnet2006-dev.codfw.wmnet' (T356287) |
[admin] |
| 21:48 |
<pfischer@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 21:47 |
<pfischer@deploy1002> |
helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 21:47 |
<pfischer@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 21:47 |
<pfischer@deploy1002> |
helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 21:46 |
<pfischer@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 21:46 |
<pfischer@deploy1002> |
helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 21:46 |
<pfischer@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 21:45 |
<pfischer@deploy1002> |
helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 21:45 |
<pfischer@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 21:45 |
<pfischer@deploy1002> |
helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 21:44 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.openstack.cloudcontrol.upgrade_openstack_node on host 'cloudnet2006-dev.codfw.wmnet' (T356287) |
[admin] |
| 21:44 |
<pfischer@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 21:43 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.openstack.cloudcontrol.upgrade_openstack_node (exit_code=0) on host 'cloudnet2005-dev.codfw.wmnet' (T356287) |
[admin] |
| 21:42 |
<pfischer@deploy1002> |
helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply |
[production] |
| 21:38 |
<cjming> |
end of UTC late backport window |
[production] |
| 21:38 |
<cjming@deploy1002> |
Finished scap: Backport for [[gerrit:1019941|Use WikimediaMessages for template overrides (T361589)]] (duration: 19m 30s) |
[production] |
| 21:35 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.openstack.cloudcontrol.upgrade_openstack_node on host 'cloudnet2005-dev.codfw.wmnet' (T356287) |
[admin] |
| 21:31 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.openstack.cloudcontrol.upgrade_openstack_node (exit_code=0) on host 'cloudcontrol2005-dev.codfw.wmnet' (T356287) |
[admin] |
| 21:25 |
<cjming@deploy1002> |
jdlrobson and cjming: Continuing with sync |
[production] |
| 21:21 |
<cjming@deploy1002> |
jdlrobson and cjming: Backport for [[gerrit:1019941|Use WikimediaMessages for template overrides (T361589)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) |
[production] |
| 21:18 |
<cjming@deploy1002> |
Started scap: Backport for [[gerrit:1019941|Use WikimediaMessages for template overrides (T361589)]] |
[production] |