9201-9250 of 10000 results (50ms)
2024-04-03 ยง
14:34 <aborrero@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirt1040.eqiad.wmnet with reason: host reimage [production]
14:32 <wmbot~anticomposite@tools-sgebastion-10> ./stewardbots/StewardBot/manage.sh restart # RC reader not reading RC [tools.stewardbots]
14:31 <aborrero@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on cloudvirt1040.eqiad.wmnet with reason: host reimage [production]
14:31 <wmbot~anticomposite@tools-sgebastion-10> SULWatcher/manage.sh restart # SULWatchers disconnected [tools.stewardbots]
14:30 <wmbot~raymond@ubuntu> END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component builds-api [toolsbeta]
14:29 <wmbot~raymond@ubuntu> START - Cookbook wmcs.toolforge.k8s.component.deploy for component builds-api [toolsbeta]
14:27 <hnowlan@deploy1002> helmfile [eqiad] DONE helmfile.d/admin 'apply'. [production]
14:27 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1231 (T360332)', diff saved to https://phabricator.wikimedia.org/P59346 and previous config saved to /var/cache/conftool/dbconfig/20240403-142709-arnaudb.json [production]
14:27 <hnowlan@deploy1002> helmfile [eqiad] START helmfile.d/admin 'apply'. [production]
14:26 <hnowlan@deploy1002> helmfile [codfw] DONE helmfile.d/admin 'apply'. [production]
14:26 <hnowlan@deploy1002> helmfile [codfw] START helmfile.d/admin 'apply'. [production]
14:24 <aborrero@cumin1002> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1040 [production]
14:24 <aborrero@cumin1002> START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1040 [production]
14:21 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on dbstore1009.eqiad.wmnet with reason: Maintenance [production]
14:21 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 8:00:00 on dbstore1009.eqiad.wmnet with reason: Maintenance [production]
14:21 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1226 (T356166)', diff saved to https://phabricator.wikimedia.org/P59345 and previous config saved to /var/cache/conftool/dbconfig/20240403-142142-marostegui.json [production]
14:17 <jmm@cumin2002> END (PASS) - Cookbook sre.maps.roll-restart-reboot (exit_code=0) rolling restart_daemons on A:maps-replica-eqiad [production]
14:16 <aborrero@cumin1002> START - Cookbook sre.hosts.reimage for host cloudvirt1040.eqiad.wmnet with OS bookworm [production]
14:16 <aborrero@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.cloudvirt.drain (exit_code=0) on host 'cloudvirt1040.eqiad.wmnet' (T319184) [admin]
14:11 <jmm@cumin2002> START - Cookbook sre.maps.roll-restart-reboot rolling restart_daemons on A:maps-replica-eqiad [production]
14:11 <jmm@cumin2002> END (PASS) - Cookbook sre.maps.roll-restart-reboot (exit_code=0) rolling restart_daemons on A:maps-replica-codfw [production]
14:09 <pfischer@deploy1002> helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
14:09 <pfischer@deploy1002> helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
14:09 <aborrero@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.drain on host 'cloudvirt1040.eqiad.wmnet' (T319184) [admin]
14:08 <ayounsi@cumin1002> END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'configure' for AS: 2527 [production]
14:07 <ayounsi@cumin1002> START - Cookbook sre.network.peering with action 'configure' for AS: 2527 [production]
14:06 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1226', diff saved to https://phabricator.wikimedia.org/P59344 and previous config saved to /var/cache/conftool/dbconfig/20240403-140634-marostegui.json [production]
14:06 <jmm@cumin2002> START - Cookbook sre.maps.roll-restart-reboot rolling restart_daemons on A:maps-replica-codfw [production]
13:54 <wmbot~taavi@tools-bastion-12> toolforge webservice restart [tools.sal]
13:51 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1226', diff saved to https://phabricator.wikimedia.org/P59343 and previous config saved to /var/cache/conftool/dbconfig/20240403-135126-marostegui.json [production]
13:46 <James_F> Zuul: [mediawiki/skins/2018] Add basic CI [releng]
13:41 <arnaudb@cumin1002> dbctl commit (dc=all): 'Depooling db1231 (T360332)', diff saved to https://phabricator.wikimedia.org/P59342 and previous config saved to /var/cache/conftool/dbconfig/20240403-134136-arnaudb.json [production]
13:41 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1231.eqiad.wmnet with reason: Maintenance [production]
13:41 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 12:00:00 on db1231.eqiad.wmnet with reason: Maintenance [production]
13:40 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1225.eqiad.wmnet with reason: Maintenance [production]
13:40 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 12:00:00 on db1225.eqiad.wmnet with reason: Maintenance [production]
13:40 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1224 (T360332)', diff saved to https://phabricator.wikimedia.org/P59341 and previous config saved to /var/cache/conftool/dbconfig/20240403-134044-arnaudb.json [production]
13:36 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1226 (T356166)', diff saved to https://phabricator.wikimedia.org/P59340 and previous config saved to /var/cache/conftool/dbconfig/20240403-133619-marostegui.json [production]
13:32 <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db1226 (T356166)', diff saved to https://phabricator.wikimedia.org/P59339 and previous config saved to /var/cache/conftool/dbconfig/20240403-133200-marostegui.json [production]
13:31 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1226.eqiad.wmnet with reason: Maintenance [production]
13:31 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 8:00:00 on db1226.eqiad.wmnet with reason: Maintenance [production]
13:31 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1216.eqiad.wmnet with reason: Maintenance [production]
13:31 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 8:00:00 on db1216.eqiad.wmnet with reason: Maintenance [production]
13:31 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1214 (T356166)', diff saved to https://phabricator.wikimedia.org/P59338 and previous config saved to /var/cache/conftool/dbconfig/20240403-133113-marostegui.json [production]
13:27 <hashar> Reloaded Zuul for https://gerrit.wikimedia.org/r/1016788 [releng]
13:25 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1224', diff saved to https://phabricator.wikimedia.org/P59337 and previous config saved to /var/cache/conftool/dbconfig/20240403-132536-arnaudb.json [production]
13:16 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1214', diff saved to https://phabricator.wikimedia.org/P59336 and previous config saved to /var/cache/conftool/dbconfig/20240403-131606-marostegui.json [production]
13:15 <moritzm> installing tiff security updates [production]
13:10 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1224', diff saved to https://phabricator.wikimedia.org/P59335 and previous config saved to /var/cache/conftool/dbconfig/20240403-131029-arnaudb.json [production]
13:08 <hashar> Reloaded Zuul for https://gerrit.wikimedia.org/r/1016761 https://gerrit.wikimedia.org/r/1016762 https://gerrit.wikimedia.org/r/1016765 [releng]