601-650 of 10000 results (83ms)
2024-04-03 ยง
15:01 <aborrero@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt1040.eqiad.wmnet with OS bookworm [production]
15:01 <hnowlan@deploy1002> helmfile [eqiad] START helmfile.d/services/wikifeeds: sync [production]
14:57 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1231', diff saved to https://phabricator.wikimedia.org/P59349 and previous config saved to /var/cache/conftool/dbconfig/20240403-145725-arnaudb.json [production]
14:44 <dreamyjazz@deploy1002> Started scap: (no justification provided) [production]
14:42 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1231', diff saved to https://phabricator.wikimedia.org/P59347 and previous config saved to /var/cache/conftool/dbconfig/20240403-144217-arnaudb.json [production]
14:34 <aborrero@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirt1040.eqiad.wmnet with reason: host reimage [production]
14:31 <aborrero@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on cloudvirt1040.eqiad.wmnet with reason: host reimage [production]
14:27 <hnowlan@deploy1002> helmfile [eqiad] DONE helmfile.d/admin 'apply'. [production]
14:27 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1231 (T360332)', diff saved to https://phabricator.wikimedia.org/P59346 and previous config saved to /var/cache/conftool/dbconfig/20240403-142709-arnaudb.json [production]
14:27 <hnowlan@deploy1002> helmfile [eqiad] START helmfile.d/admin 'apply'. [production]
14:26 <hnowlan@deploy1002> helmfile [codfw] DONE helmfile.d/admin 'apply'. [production]
14:26 <hnowlan@deploy1002> helmfile [codfw] START helmfile.d/admin 'apply'. [production]
14:24 <aborrero@cumin1002> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1040 [production]
14:24 <aborrero@cumin1002> START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1040 [production]
14:21 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on dbstore1009.eqiad.wmnet with reason: Maintenance [production]
14:21 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 8:00:00 on dbstore1009.eqiad.wmnet with reason: Maintenance [production]
14:21 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1226 (T356166)', diff saved to https://phabricator.wikimedia.org/P59345 and previous config saved to /var/cache/conftool/dbconfig/20240403-142142-marostegui.json [production]
14:17 <jmm@cumin2002> END (PASS) - Cookbook sre.maps.roll-restart-reboot (exit_code=0) rolling restart_daemons on A:maps-replica-eqiad [production]
14:16 <aborrero@cumin1002> START - Cookbook sre.hosts.reimage for host cloudvirt1040.eqiad.wmnet with OS bookworm [production]
14:11 <jmm@cumin2002> START - Cookbook sre.maps.roll-restart-reboot rolling restart_daemons on A:maps-replica-eqiad [production]
14:11 <jmm@cumin2002> END (PASS) - Cookbook sre.maps.roll-restart-reboot (exit_code=0) rolling restart_daemons on A:maps-replica-codfw [production]
14:09 <pfischer@deploy1002> helmfile [eqiad] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
14:09 <pfischer@deploy1002> helmfile [eqiad] START helmfile.d/services/cirrus-streaming-updater: apply [production]
14:08 <ayounsi@cumin1002> END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'configure' for AS: 2527 [production]
14:07 <ayounsi@cumin1002> START - Cookbook sre.network.peering with action 'configure' for AS: 2527 [production]
14:06 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1226', diff saved to https://phabricator.wikimedia.org/P59344 and previous config saved to /var/cache/conftool/dbconfig/20240403-140634-marostegui.json [production]
14:06 <jmm@cumin2002> START - Cookbook sre.maps.roll-restart-reboot rolling restart_daemons on A:maps-replica-codfw [production]
13:51 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1226', diff saved to https://phabricator.wikimedia.org/P59343 and previous config saved to /var/cache/conftool/dbconfig/20240403-135126-marostegui.json [production]
13:41 <arnaudb@cumin1002> dbctl commit (dc=all): 'Depooling db1231 (T360332)', diff saved to https://phabricator.wikimedia.org/P59342 and previous config saved to /var/cache/conftool/dbconfig/20240403-134136-arnaudb.json [production]
13:41 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1231.eqiad.wmnet with reason: Maintenance [production]
13:41 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 12:00:00 on db1231.eqiad.wmnet with reason: Maintenance [production]
13:40 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1225.eqiad.wmnet with reason: Maintenance [production]
13:40 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 12:00:00 on db1225.eqiad.wmnet with reason: Maintenance [production]
13:40 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1224 (T360332)', diff saved to https://phabricator.wikimedia.org/P59341 and previous config saved to /var/cache/conftool/dbconfig/20240403-134044-arnaudb.json [production]
13:36 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1226 (T356166)', diff saved to https://phabricator.wikimedia.org/P59340 and previous config saved to /var/cache/conftool/dbconfig/20240403-133619-marostegui.json [production]
13:32 <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db1226 (T356166)', diff saved to https://phabricator.wikimedia.org/P59339 and previous config saved to /var/cache/conftool/dbconfig/20240403-133200-marostegui.json [production]
13:31 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1226.eqiad.wmnet with reason: Maintenance [production]
13:31 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 8:00:00 on db1226.eqiad.wmnet with reason: Maintenance [production]
13:31 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1216.eqiad.wmnet with reason: Maintenance [production]
13:31 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 8:00:00 on db1216.eqiad.wmnet with reason: Maintenance [production]
13:31 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1214 (T356166)', diff saved to https://phabricator.wikimedia.org/P59338 and previous config saved to /var/cache/conftool/dbconfig/20240403-133113-marostegui.json [production]
13:25 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1224', diff saved to https://phabricator.wikimedia.org/P59337 and previous config saved to /var/cache/conftool/dbconfig/20240403-132536-arnaudb.json [production]
13:16 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1214', diff saved to https://phabricator.wikimedia.org/P59336 and previous config saved to /var/cache/conftool/dbconfig/20240403-131606-marostegui.json [production]
13:15 <moritzm> installing tiff security updates [production]
13:10 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1224', diff saved to https://phabricator.wikimedia.org/P59335 and previous config saved to /var/cache/conftool/dbconfig/20240403-131029-arnaudb.json [production]
13:00 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1214', diff saved to https://phabricator.wikimedia.org/P59334 and previous config saved to /var/cache/conftool/dbconfig/20240403-130058-marostegui.json [production]
12:55 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1224 (T360332)', diff saved to https://phabricator.wikimedia.org/P59333 and previous config saved to /var/cache/conftool/dbconfig/20240403-125521-arnaudb.json [production]
12:45 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1214 (T356166)', diff saved to https://phabricator.wikimedia.org/P59332 and previous config saved to /var/cache/conftool/dbconfig/20240403-124550-marostegui.json [production]
12:42 <hashar> Upgrading CI Jenkins # T360759 [production]
12:34 <aborrero@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt1039.eqiad.wmnet with OS bookworm [production]