2551-2600 of 10000 results (78ms)
2024-04-10 ยง
11:38 <cgoubert@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw1492.eqiad.wmnet with reason: host reimage [production]
11:34 <cgoubert@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw1491.eqiad.wmnet with reason: host reimage [production]
11:32 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1186 (T360332)', diff saved to https://phabricator.wikimedia.org/P60222 and previous config saved to /var/cache/conftool/dbconfig/20240410-113220-arnaudb.json [production]
11:31 <cgoubert@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw1422.eqiad.wmnet with reason: host reimage [production]
11:29 <arnaudb@cumin1002> dbctl commit (dc=all): 'Depooling db1186 (T360332)', diff saved to https://phabricator.wikimedia.org/P60221 and previous config saved to /var/cache/conftool/dbconfig/20240410-112929-arnaudb.json [production]
11:29 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1186.eqiad.wmnet with reason: Maintenance [production]
11:29 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 12:00:00 on db1186.eqiad.wmnet with reason: Maintenance [production]
11:29 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1169 (T360332)', diff saved to https://phabricator.wikimedia.org/P60220 and previous config saved to /var/cache/conftool/dbconfig/20240410-112907-arnaudb.json [production]
11:28 <jiji@deploy1002> Finished scap: Deploy chart changes in gerrit:1015342 (duration: 08m 18s) [production]
11:27 <cgoubert@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw1493.eqiad.wmnet with reason: host reimage [production]
11:24 <marostegui@cumin1002> dbctl commit (dc=all): 'db1175 (re)pooling @ 10%: Repooling', diff saved to https://phabricator.wikimedia.org/P60219 and previous config saved to /var/cache/conftool/dbconfig/20240410-112455-root.json [production]
11:24 <cgoubert@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw1421.eqiad.wmnet with reason: host reimage [production]
11:23 <cgoubert@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on mw1493.eqiad.wmnet with reason: host reimage [production]
11:22 <cgoubert@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on mw1492.eqiad.wmnet with reason: host reimage [production]
11:22 <cgoubert@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on mw1491.eqiad.wmnet with reason: host reimage [production]
11:21 <cgoubert@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on mw1422.eqiad.wmnet with reason: host reimage [production]
11:21 <cgoubert@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on mw1421.eqiad.wmnet with reason: host reimage [production]
11:19 <jiji@deploy1002> Started scap: Deploy chart changes in gerrit:1015342 [production]
11:16 <mvolz@deploy1002> helmfile [eqiad] DONE helmfile.d/services/citoid: apply [production]
11:15 <mvolz@deploy1002> helmfile [eqiad] START helmfile.d/services/citoid: apply [production]
11:14 <mvolz@deploy1002> helmfile [codfw] DONE helmfile.d/services/citoid: apply [production]
11:14 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1169', diff saved to https://phabricator.wikimedia.org/P60218 and previous config saved to /var/cache/conftool/dbconfig/20240410-111400-arnaudb.json [production]
11:13 <mvolz@deploy1002> helmfile [codfw] START helmfile.d/services/citoid: apply [production]
11:12 <mvolz@deploy1002> helmfile [staging] DONE helmfile.d/services/citoid: apply [production]
11:12 <mvolz@deploy1002> helmfile [staging] START helmfile.d/services/citoid: apply [production]
11:10 <cgoubert@cumin1002> START - Cookbook sre.hosts.reimage for host mw1493.eqiad.wmnet with OS bullseye [production]
11:09 <marostegui@cumin1002> dbctl commit (dc=all): 'db1175 (re)pooling @ 5%: Repooling', diff saved to https://phabricator.wikimedia.org/P60217 and previous config saved to /var/cache/conftool/dbconfig/20240410-110949-root.json [production]
11:09 <cgoubert@cumin1002> START - Cookbook sre.hosts.reimage for host mw1492.eqiad.wmnet with OS bullseye [production]
11:09 <cgoubert@cumin1002> START - Cookbook sre.hosts.reimage for host mw1491.eqiad.wmnet with OS bullseye [production]
11:08 <jiji@deploy1002> helmfile [codfw] DONE helmfile.d/services/mw-debug: apply [production]
11:08 <cgoubert@cumin1002> START - Cookbook sre.hosts.reimage for host mw1422.eqiad.wmnet with OS bullseye [production]
11:08 <cgoubert@cumin1002> START - Cookbook sre.hosts.reimage for host mw1421.eqiad.wmnet with OS bullseye [production]
11:07 <jiji@deploy1002> helmfile [codfw] START helmfile.d/services/mw-debug: apply [production]
11:07 <jiji@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply [production]
11:07 <jiji@deploy1002> helmfile [eqiad] START helmfile.d/services/mw-debug: apply [production]
11:03 <jiji@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply [production]
11:02 <jiji@deploy1002> helmfile [eqiad] START helmfile.d/services/mw-debug: apply [production]
11:02 <jiji@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply [production]
11:02 <jiji@deploy1002> helmfile [eqiad] START helmfile.d/services/mw-debug: apply [production]
10:59 <claime> Depooling mw1421.eqiad.wmnet,mw1422.eqiad.wmnet,mw1491.eqiad.wmnet,mw1492.eqiad.wmnet,mw1493.eqiad.wmnet - T351074 [production]
10:58 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1169', diff saved to https://phabricator.wikimedia.org/P60216 and previous config saved to /var/cache/conftool/dbconfig/20240410-105852-arnaudb.json [production]
10:56 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1175.eqiad.wmnet with OS bookworm [production]
10:54 <marostegui@cumin1002> dbctl commit (dc=all): 'db1175 (re)pooling @ 1%: Repooling', diff saved to https://phabricator.wikimedia.org/P60215 and previous config saved to /var/cache/conftool/dbconfig/20240410-105444-root.json [production]
10:53 <jiji@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mw-debug: apply [production]
10:53 <jiji@deploy1002> helmfile [eqiad] START helmfile.d/services/mw-debug: apply [production]
10:43 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1169 (T360332)', diff saved to https://phabricator.wikimedia.org/P60214 and previous config saved to /var/cache/conftool/dbconfig/20240410-104345-arnaudb.json [production]
10:40 <arnaudb@cumin1002> dbctl commit (dc=all): 'Depooling db1169 (T360332)', diff saved to https://phabricator.wikimedia.org/P60213 and previous config saved to /var/cache/conftool/dbconfig/20240410-104053-arnaudb.json [production]
10:40 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1169.eqiad.wmnet with reason: Maintenance [production]
10:40 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 12:00:00 on db1169.eqiad.wmnet with reason: Maintenance [production]
10:40 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1163 (T360332)', diff saved to https://phabricator.wikimedia.org/P60212 and previous config saved to /var/cache/conftool/dbconfig/20240410-104030-arnaudb.json [production]