3951-4000 of 10000 results (64ms)
2022-05-31 ยง
16:16 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1170.eqiad.wmnet with reason: Maintenance [production]
16:16 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1170.eqiad.wmnet with reason: Maintenance [production]
16:16 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1136 (T309311)', diff saved to https://phabricator.wikimedia.org/P29231 and previous config saved to /var/cache/conftool/dbconfig/20220531-161609-ladsgroup.json [production]
16:14 <robh@cumin1001> START - Cookbook sre.hosts.reimage for host cloudelastic1006.wikimedia.org with OS bullseye [production]
16:13 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1099:3318 (T60674)', diff saved to https://phabricator.wikimedia.org/P29230 and previous config saved to /var/cache/conftool/dbconfig/20220531-161329-ladsgroup.json [production]
16:13 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1099.eqiad.wmnet with reason: Maintenance [production]
16:13 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1099.eqiad.wmnet with reason: Maintenance [production]
16:11 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166', diff saved to https://phabricator.wikimedia.org/P29229 and previous config saved to /var/cache/conftool/dbconfig/20220531-161139-ladsgroup.json [production]
16:09 <ladsgroup@cumin1001> dbctl commit (dc=all): 'db1100 (re)pooling @ 50%: Maint done', diff saved to https://phabricator.wikimedia.org/P29228 and previous config saved to /var/cache/conftool/dbconfig/20220531-160907-ladsgroup.json [production]
16:07 <bking@cumin1001> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host cloudelastic1006.wikimedia.org with OS bullseye [production]
16:07 <robh@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudelastic1006.wikimedia.org with OS bullseye [production]
16:07 <robh@cumin1001> START - Cookbook sre.hosts.reimage for host cloudelastic1006.wikimedia.org with OS bullseye [production]
16:01 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1136', diff saved to https://phabricator.wikimedia.org/P29227 and previous config saved to /var/cache/conftool/dbconfig/20220531-160103-ladsgroup.json [production]
15:57 <bking@cumin1001> START - Cookbook sre.hosts.reimage for host cloudelastic1006.wikimedia.org with OS bullseye [production]
15:56 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166 (T307525)', diff saved to https://phabricator.wikimedia.org/P29226 and previous config saved to /var/cache/conftool/dbconfig/20220531-155634-ladsgroup.json [production]
15:54 <ladsgroup@cumin1001> dbctl commit (dc=all): 'db1100 (re)pooling @ 10%: Maint done', diff saved to https://phabricator.wikimedia.org/P29225 and previous config saved to /var/cache/conftool/dbconfig/20220531-155403-ladsgroup.json [production]
15:49 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on 12 hosts with reason: Maintenance [production]
15:49 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 12:00:00 on 12 hosts with reason: Maintenance [production]
15:49 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2079.codfw.wmnet with reason: Maintenance [production]
15:49 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db2079.codfw.wmnet with reason: Maintenance [production]
15:49 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1101:3318 (T60674)', diff saved to https://phabricator.wikimedia.org/P29224 and previous config saved to /var/cache/conftool/dbconfig/20220531-154928-ladsgroup.json [production]
15:45 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1136', diff saved to https://phabricator.wikimedia.org/P29223 and previous config saved to /var/cache/conftool/dbconfig/20220531-154558-ladsgroup.json [production]
15:40 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1166 (T307525)', diff saved to https://phabricator.wikimedia.org/P29222 and previous config saved to /var/cache/conftool/dbconfig/20220531-154025-ladsgroup.json [production]
15:40 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1166.eqiad.wmnet with reason: Maintenance [production]
15:40 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1166.eqiad.wmnet with reason: Maintenance [production]
15:39 <ladsgroup@cumin1001> dbctl commit (dc=all): 'db1100 (re)pooling @ 5%: Maint done', diff saved to https://phabricator.wikimedia.org/P29221 and previous config saved to /var/cache/conftool/dbconfig/20220531-153859-ladsgroup.json [production]
15:38 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166 (T307525)', diff saved to https://phabricator.wikimedia.org/P29220 and previous config saved to /var/cache/conftool/dbconfig/20220531-153846-ladsgroup.json [production]
15:34 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1101:3318', diff saved to https://phabricator.wikimedia.org/P29219 and previous config saved to /var/cache/conftool/dbconfig/20220531-153422-ladsgroup.json [production]
15:30 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1136 (T309311)', diff saved to https://phabricator.wikimedia.org/P29218 and previous config saved to /var/cache/conftool/dbconfig/20220531-153053-ladsgroup.json [production]
15:23 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166', diff saved to https://phabricator.wikimedia.org/P29217 and previous config saved to /var/cache/conftool/dbconfig/20220531-152341-ladsgroup.json [production]
15:19 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1101:3318', diff saved to https://phabricator.wikimedia.org/P29216 and previous config saved to /var/cache/conftool/dbconfig/20220531-151916-ladsgroup.json [production]
15:15 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1136 (T309311)', diff saved to https://phabricator.wikimedia.org/P29215 and previous config saved to /var/cache/conftool/dbconfig/20220531-151515-ladsgroup.json [production]
15:15 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1136.eqiad.wmnet with reason: Maintenance [production]
15:15 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1136.eqiad.wmnet with reason: Maintenance [production]
15:12 <jelto> migrate gitlab-replica to gitlab1003 [production]
15:08 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166', diff saved to https://phabricator.wikimedia.org/P29214 and previous config saved to /var/cache/conftool/dbconfig/20220531-150836-ladsgroup.json [production]
15:04 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1101:3318 (T60674)', diff saved to https://phabricator.wikimedia.org/P29213 and previous config saved to /var/cache/conftool/dbconfig/20220531-150411-ladsgroup.json [production]
14:37 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1101:3318 (T60674)', diff saved to https://phabricator.wikimedia.org/P29211 and previous config saved to /var/cache/conftool/dbconfig/20220531-143720-ladsgroup.json [production]
14:37 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1166 (T307525)', diff saved to https://phabricator.wikimedia.org/P29210 and previous config saved to /var/cache/conftool/dbconfig/20220531-143716-ladsgroup.json [production]
14:37 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1101.eqiad.wmnet with reason: Maintenance [production]
14:37 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1101.eqiad.wmnet with reason: Maintenance [production]
14:37 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1109 (T60674)', diff saved to https://phabricator.wikimedia.org/P29209 and previous config saved to /var/cache/conftool/dbconfig/20220531-143712-ladsgroup.json [production]
14:37 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1166.eqiad.wmnet with reason: Maintenance [production]
14:37 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1166.eqiad.wmnet with reason: Maintenance [production]
14:36 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1171.eqiad.wmnet with reason: Maintenance [production]
14:36 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1171.eqiad.wmnet with reason: Maintenance [production]
14:36 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1127 (T309311)', diff saved to https://phabricator.wikimedia.org/P29208 and previous config saved to /var/cache/conftool/dbconfig/20220531-143621-ladsgroup.json [production]
14:35 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166 (T307525)', diff saved to https://phabricator.wikimedia.org/P29207 and previous config saved to /var/cache/conftool/dbconfig/20220531-143538-ladsgroup.json [production]
14:25 <jbond@deploy1002> Finished deploy [netbox/deploy@7bbf659]: deploying v2.10.4-wmf6 to new hosts (duration: 00m 52s) [production]
14:24 <jbond@deploy1002> Started deploy [netbox/deploy@7bbf659]: deploying v2.10.4-wmf6 to new hosts [production]