6001-6050 of 10000 results (117ms)
2024-02-20 ยง
07:39 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db2168', diff saved to https://phabricator.wikimedia.org/P57221 and previous config saved to /var/cache/conftool/dbconfig/20240220-073912-root.json [production]
07:38 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2028.codfw.wmnet [production]
07:36 <marostegui@cumin1002> dbctl commit (dc=all): 'db2171 (re)pooling @ 5%: After rearraging sections T354826', diff saved to https://phabricator.wikimedia.org/P57220 and previous config saved to /var/cache/conftool/dbconfig/20240220-073658-root.json [production]
07:36 <marostegui@cumin1002> dbctl commit (dc=all): 'es2021 (re)pooling @ 75%: After migration', diff saved to https://phabricator.wikimedia.org/P57219 and previous config saved to /var/cache/conftool/dbconfig/20240220-073623-root.json [production]
07:34 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2028.codfw.wmnet [production]
07:33 <marostegui@cumin1002> dbctl commit (dc=all): 'db1170 (re)pooling @ 100%: After migration to 10.6', diff saved to https://phabricator.wikimedia.org/P57218 and previous config saved to /var/cache/conftool/dbconfig/20240220-073313-root.json [production]
07:32 <marostegui@cumin1002> START - Cookbook sre.hosts.reimage for host db2170.codfw.wmnet with OS bookworm [production]
07:31 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db2170', diff saved to https://phabricator.wikimedia.org/P57217 and previous config saved to /var/cache/conftool/dbconfig/20240220-073139-root.json [production]
07:28 <marostegui@cumin1002> dbctl commit (dc=all): 'db1244 (re)pooling @ 25%: After rearraging sections T354826', diff saved to https://phabricator.wikimedia.org/P57216 and previous config saved to /var/cache/conftool/dbconfig/20240220-072838-root.json [production]
07:28 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2171.codfw.wmnet with OS bookworm [production]
07:27 <ayounsi@cumin1002> END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'email' for AS: 56286 [production]
07:27 <ayounsi@cumin1002> START - Cookbook sre.network.peering with action 'email' for AS: 56286 [production]
07:27 <ayounsi@cumin1002> END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'email' for AS: 60501 [production]
07:26 <ayounsi@cumin1002> START - Cookbook sre.network.peering with action 'email' for AS: 60501 [production]
07:26 <ayounsi@cumin1002> END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'email' for AS: 18779 [production]
07:26 <ayounsi@cumin1002> START - Cookbook sre.network.peering with action 'email' for AS: 18779 [production]
07:26 <ayounsi@cumin1002> END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'email' for AS: 26554 [production]
07:25 <ayounsi@cumin1002> START - Cookbook sre.network.peering with action 'email' for AS: 26554 [production]
07:24 <marostegui@cumin1002> dbctl commit (dc=all): 'db1246 (re)pooling @ 10%: After rearraging sections T354826', diff saved to https://phabricator.wikimedia.org/P57215 and previous config saved to /var/cache/conftool/dbconfig/20240220-072455-root.json [production]
07:21 <marostegui@cumin1002> dbctl commit (dc=all): 'es2021 (re)pooling @ 50%: After migration', diff saved to https://phabricator.wikimedia.org/P57214 and previous config saved to /var/cache/conftool/dbconfig/20240220-072118-root.json [production]
07:18 <marostegui@cumin1002> dbctl commit (dc=all): 'db1170 (re)pooling @ 75%: After migration to 10.6', diff saved to https://phabricator.wikimedia.org/P57213 and previous config saved to /var/cache/conftool/dbconfig/20240220-071808-root.json [production]
07:13 <marostegui@cumin1002> dbctl commit (dc=all): 'db1244 (re)pooling @ 10%: After rearraging sections T354826', diff saved to https://phabricator.wikimedia.org/P57212 and previous config saved to /var/cache/conftool/dbconfig/20240220-071333-root.json [production]
07:09 <marostegui@cumin1002> dbctl commit (dc=all): 'db1246 (re)pooling @ 5%: After rearraging sections T354826', diff saved to https://phabricator.wikimedia.org/P57211 and previous config saved to /var/cache/conftool/dbconfig/20240220-070948-root.json [production]
07:06 <marostegui@cumin1002> dbctl commit (dc=all): 'es2021 (re)pooling @ 25%: After migration', diff saved to https://phabricator.wikimedia.org/P57210 and previous config saved to /var/cache/conftool/dbconfig/20240220-070613-root.json [production]
07:04 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1246.eqiad.wmnet with OS bookworm [production]
07:03 <marostegui@cumin1002> dbctl commit (dc=all): 'db1170 (re)pooling @ 50%: After migration to 10.6', diff saved to https://phabricator.wikimedia.org/P57209 and previous config saved to /var/cache/conftool/dbconfig/20240220-070303-root.json [production]
07:02 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2171.codfw.wmnet with reason: host reimage [production]
06:59 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1244.eqiad.wmnet with OS bookworm [production]
06:58 <marostegui@cumin1002> dbctl commit (dc=all): 'db1244 (re)pooling @ 5%: After rearraging sections T354826', diff saved to https://phabricator.wikimedia.org/P57208 and previous config saved to /var/cache/conftool/dbconfig/20240220-065828-root.json [production]
06:58 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on db2171.codfw.wmnet with reason: host reimage [production]
06:51 <marostegui@cumin1002> dbctl commit (dc=all): 'es2021 (re)pooling @ 10%: After migration', diff saved to https://phabricator.wikimedia.org/P57207 and previous config saved to /var/cache/conftool/dbconfig/20240220-065108-root.json [production]
06:47 <marostegui@cumin1002> dbctl commit (dc=all): 'db1170 (re)pooling @ 25%: After migration to 10.6', diff saved to https://phabricator.wikimedia.org/P57206 and previous config saved to /var/cache/conftool/dbconfig/20240220-064758-root.json [production]
06:44 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1246.eqiad.wmnet with reason: host reimage [production]
06:42 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on db1246.eqiad.wmnet with reason: host reimage [production]
06:41 <marostegui@cumin1002> dbctl commit (dc=all): 'Place db2171 in s5 depooled T354826', diff saved to https://phabricator.wikimedia.org/P57205 and previous config saved to /var/cache/conftool/dbconfig/20240220-064152-marostegui.json [production]
06:40 <marostegui@cumin1002> dbctl commit (dc=all): 'Remove db2171 multi-instance', diff saved to https://phabricator.wikimedia.org/P57204 and previous config saved to /var/cache/conftool/dbconfig/20240220-064014-marostegui.json [production]
06:39 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1244.eqiad.wmnet with reason: host reimage [production]
06:39 <marostegui@cumin1002> START - Cookbook sre.hosts.reimage for host db2171.codfw.wmnet with OS bookworm [production]
06:37 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on db1244.eqiad.wmnet with reason: host reimage [production]
06:36 <marostegui@cumin1002> dbctl commit (dc=all): 'es2021 (re)pooling @ 5%: After migration', diff saved to https://phabricator.wikimedia.org/P57203 and previous config saved to /var/cache/conftool/dbconfig/20240220-063603-root.json [production]
06:35 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db2171 T354826', diff saved to https://phabricator.wikimedia.org/P57202 and previous config saved to /var/cache/conftool/dbconfig/20240220-063521-marostegui.json [production]
06:32 <marostegui@cumin1002> dbctl commit (dc=all): 'db1170 (re)pooling @ 10%: After migration to 10.6', diff saved to https://phabricator.wikimedia.org/P57201 and previous config saved to /var/cache/conftool/dbconfig/20240220-063254-root.json [production]
06:29 <marostegui@cumin1002> START - Cookbook sre.hosts.reimage for host db1246.eqiad.wmnet with OS bookworm [production]
06:28 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db1246', diff saved to https://phabricator.wikimedia.org/P57200 and previous config saved to /var/cache/conftool/dbconfig/20240220-062759-root.json [production]
06:24 <marostegui@cumin1002> START - Cookbook sre.hosts.reimage for host db1244.eqiad.wmnet with OS bookworm [production]
06:22 <marostegui@deploy2002> Finished scap: Backport for [[gerrit:1004715|Revert "db-production.php: Disable writes on es4"]] (duration: 09m 32s) [production]
06:20 <marostegui@cumin1002> dbctl commit (dc=all): 'es2021 (re)pooling @ 1%: After migration', diff saved to https://phabricator.wikimedia.org/P57199 and previous config saved to /var/cache/conftool/dbconfig/20240220-062058-root.json [production]
06:19 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool db1244', diff saved to https://phabricator.wikimedia.org/P57198 and previous config saved to /var/cache/conftool/dbconfig/20240220-061932-root.json [production]
06:17 <marostegui@cumin1002> dbctl commit (dc=all): 'db1170 (re)pooling @ 5%: After migration to 10.6', diff saved to https://phabricator.wikimedia.org/P57197 and previous config saved to /var/cache/conftool/dbconfig/20240220-061749-root.json [production]
06:14 <marostegui@deploy2002> marostegui: Continuing with sync [production]