6001-6050 of 10000 results (103ms)
2024-05-30 ยง
13:43 <logmsgbot> @deploy1002 helmfile [codfw] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
13:43 <logmsgbot> @deploy1002 helmfile [codfw] START helmfile.d/services/cirrus-streaming-updater: apply [production]
13:43 <jiji@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mc2044.codfw.wmnet with OS bookworm [production]
13:40 <logmsgbot> @deploy1002 helmfile [codfw] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
13:40 <logmsgbot> @deploy1002 helmfile [codfw] START helmfile.d/services/cirrus-streaming-updater: apply [production]
13:39 <logmsgbot> @deploy1002 helmfile [codfw] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
13:38 <logmsgbot> @deploy1002 helmfile [codfw] START helmfile.d/services/cirrus-streaming-updater: apply [production]
13:38 <jiji@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mc1044.eqiad.wmnet with OS bookworm [production]
13:37 <logmsgbot> @deploy1002 helmfile [codfw] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
13:36 <logmsgbot> @deploy1002 helmfile [codfw] START helmfile.d/services/cirrus-streaming-updater: apply [production]
13:33 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2220 (T366123)', diff saved to https://phabricator.wikimedia.org/P63697 and previous config saved to /var/cache/conftool/dbconfig/20240530-133348-marostegui.json [production]
13:28 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1173.eqiad.wmnet with reason: upgrade [production]
13:27 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on db1173.eqiad.wmnet with reason: upgrade [production]
13:27 <logmsgbot> @deploy1002 helmfile [codfw] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
13:27 <logmsgbot> @deploy1002 helmfile [codfw] START helmfile.d/services/cirrus-streaming-updater: apply [production]
13:25 <logmsgbot> @deploy1002 helmfile [codfw] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
13:25 <logmsgbot> @deploy1002 helmfile [codfw] START helmfile.d/services/cirrus-streaming-updater: apply [production]
13:25 <jiji@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mc2044.codfw.wmnet with reason: host reimage [production]
13:23 <logmsgbot> @deploy1002 helmfile [codfw] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
13:23 <logmsgbot> @deploy1002 helmfile [codfw] START helmfile.d/services/cirrus-streaming-updater: apply [production]
13:22 <jiji@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on mc2044.codfw.wmnet with reason: host reimage [production]
13:20 <jiji@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mc1044.eqiad.wmnet with reason: host reimage [production]
13:20 <arnaudb@cumin1002> END (PASS) - Cookbook sre.mysql.upgrade (exit_code=0) for db1173.eqiad.wmnet [production]
13:19 <dcausse@deploy1002> Finished scap: Backport for [[gerrit:1037153|cirrus: Send weighted tags to known clusters]] (duration: 12m 43s) [production]
13:17 <logmsgbot> @deploy1002 helmfile [codfw] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
13:17 <logmsgbot> @deploy1002 helmfile [codfw] START helmfile.d/services/cirrus-streaming-updater: apply [production]
13:17 <jiji@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on mc1044.eqiad.wmnet with reason: host reimage [production]
13:15 <logmsgbot> @deploy1002 helmfile [codfw] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
13:15 <logmsgbot> @deploy1002 helmfile [codfw] START helmfile.d/services/cirrus-streaming-updater: apply [production]
13:15 <arnaudb@cumin1002> START - Cookbook sre.mysql.upgrade for db1173.eqiad.wmnet [production]
13:14 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1173.eqiad.wmnet with reason: upgrade [production]
13:14 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on db1173.eqiad.wmnet with reason: upgrade [production]
13:13 <arnaudb@cumin1002> dbctl commit (dc=all): 'Depool db1173 T356240', diff saved to https://phabricator.wikimedia.org/P63695 and previous config saved to /var/cache/conftool/dbconfig/20240530-131349-arnaudb.json [production]
13:12 <logmsgbot> @deploy1002 helmfile [codfw] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
13:12 <logmsgbot> @deploy1002 helmfile [codfw] START helmfile.d/services/cirrus-streaming-updater: apply [production]
13:10 <dcausse@deploy1002> dcausse and ebernhardson: Continuing with sync [production]
13:10 <logmsgbot> @deploy1002 helmfile [codfw] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
13:10 <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db2220 (T366123)', diff saved to https://phabricator.wikimedia.org/P63694 and previous config saved to /var/cache/conftool/dbconfig/20240530-131012-marostegui.json [production]
13:10 <logmsgbot> @deploy1002 helmfile [codfw] START helmfile.d/services/cirrus-streaming-updater: apply [production]
13:10 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db2220.codfw.wmnet with reason: Maintenance [production]
13:09 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 8:00:00 on db2220.codfw.wmnet with reason: Maintenance [production]
13:09 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2208 (T366123)', diff saved to https://phabricator.wikimedia.org/P63693 and previous config saved to /var/cache/conftool/dbconfig/20240530-130946-marostegui.json [production]
13:09 <dcausse@deploy1002> dcausse and ebernhardson: Backport for [[gerrit:1037153|cirrus: Send weighted tags to known clusters]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
13:08 <logmsgbot> @deploy1002 helmfile [codfw] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
13:08 <logmsgbot> @deploy1002 helmfile [codfw] START helmfile.d/services/cirrus-streaming-updater: apply [production]
13:06 <dcausse@deploy1002> Started scap: Backport for [[gerrit:1037153|cirrus: Send weighted tags to known clusters]] [production]
13:06 <logmsgbot> @deploy1002 helmfile [codfw] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
13:06 <logmsgbot> @deploy1002 helmfile [codfw] START helmfile.d/services/cirrus-streaming-updater: apply [production]
13:04 <jiji@cumin1002> START - Cookbook sre.hosts.reimage for host mc1044.eqiad.wmnet with OS bookworm [production]
13:04 <jiji@cumin2002> START - Cookbook sre.hosts.reimage for host mc2044.codfw.wmnet with OS bookworm [production]