1601-1650 of 10000 results (78ms)
2022-11-28 ยง
17:54 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2177 (T323827)', diff saved to https://phabricator.wikimedia.org/P41414 and previous config saved to /var/cache/conftool/dbconfig/20221128-175458-ladsgroup.json [production]
17:54 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2110 (T321126)', diff saved to https://phabricator.wikimedia.org/P41413 and previous config saved to /var/cache/conftool/dbconfig/20221128-175445-marostegui.json [production]
17:52 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db2110 (T321126)', diff saved to https://phabricator.wikimedia.org/P41412 and previous config saved to /var/cache/conftool/dbconfig/20221128-175232-marostegui.json [production]
17:52 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2110.codfw.wmnet with reason: Maintenance [production]
17:52 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 5:00:00 on db2110.codfw.wmnet with reason: Maintenance [production]
17:52 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2106 (T321126)', diff saved to https://phabricator.wikimedia.org/P41411 and previous config saved to /var/cache/conftool/dbconfig/20221128-175210-marostegui.json [production]
17:49 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2170:3312 (T323827)', diff saved to https://phabricator.wikimedia.org/P41410 and previous config saved to /var/cache/conftool/dbconfig/20221128-174951-ladsgroup.json [production]
17:49 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1175', diff saved to https://phabricator.wikimedia.org/P41409 and previous config saved to /var/cache/conftool/dbconfig/20221128-174925-ladsgroup.json [production]
17:43 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1156 (T323827)', diff saved to https://phabricator.wikimedia.org/P41408 and previous config saved to /var/cache/conftool/dbconfig/20221128-174324-ladsgroup.json [production]
17:43 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 20:00:00 on clouddb[1014,1018,1021].eqiad.wmnet,db1155.eqiad.wmnet with reason: Maintenance [production]
17:43 <jbond@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ms-be2050.codfw.wmnet with reason: host reimage [production]
17:43 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 20:00:00 on clouddb[1014,1018,1021].eqiad.wmnet,db1155.eqiad.wmnet with reason: Maintenance [production]
17:43 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on db1156.eqiad.wmnet with reason: Maintenance [production]
17:42 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 10:00:00 on db1156.eqiad.wmnet with reason: Maintenance [production]
17:42 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1146:3312 (T323827)', diff saved to https://phabricator.wikimedia.org/P41407 and previous config saved to /var/cache/conftool/dbconfig/20221128-174213-ladsgroup.json [production]
17:39 <jbond@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on ms-be2050.codfw.wmnet with reason: host reimage [production]
17:37 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2106', diff saved to https://phabricator.wikimedia.org/P41406 and previous config saved to /var/cache/conftool/dbconfig/20221128-173704-marostegui.json [production]
17:35 <jnuche@deploy1002> Installation of scap version "4.29.2" completed for 558 hosts [production]
17:35 <jnuche@deploy1002> Installing scap version "4.29.2" for 558 hosts [production]
17:34 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1175', diff saved to https://phabricator.wikimedia.org/P41405 and previous config saved to /var/cache/conftool/dbconfig/20221128-173418-ladsgroup.json [production]
17:32 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2170:3312 (T323827)', diff saved to https://phabricator.wikimedia.org/P41404 and previous config saved to /var/cache/conftool/dbconfig/20221128-173227-ladsgroup.json [production]
17:32 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on db2170.codfw.wmnet with reason: Maintenance [production]
17:32 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 10:00:00 on db2170.codfw.wmnet with reason: Maintenance [production]
17:32 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2148 (T323827)', diff saved to https://phabricator.wikimedia.org/P41403 and previous config saved to /var/cache/conftool/dbconfig/20221128-173206-ladsgroup.json [production]
17:27 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1146:3312', diff saved to https://phabricator.wikimedia.org/P41402 and previous config saved to /var/cache/conftool/dbconfig/20221128-172707-ladsgroup.json [production]
17:24 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2177 (T323827)', diff saved to https://phabricator.wikimedia.org/P41401 and previous config saved to /var/cache/conftool/dbconfig/20221128-172442-ladsgroup.json [production]
17:24 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on db2177.codfw.wmnet with reason: Maintenance [production]
17:24 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 10:00:00 on db2177.codfw.wmnet with reason: Maintenance [production]
17:24 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2156 (T323827)', diff saved to https://phabricator.wikimedia.org/P41400 and previous config saved to /var/cache/conftool/dbconfig/20221128-172419-ladsgroup.json [production]
17:21 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2106', diff saved to https://phabricator.wikimedia.org/P41399 and previous config saved to /var/cache/conftool/dbconfig/20221128-172157-marostegui.json [production]
17:21 <jbond@cumin2002> START - Cookbook sre.hosts.reimage for host ms-be2050.codfw.wmnet with OS bullseye [production]
17:20 <jbond@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host ms-be2050.codfw.wmnet with OS bullseye [production]
17:19 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1175 (T323907)', diff saved to https://phabricator.wikimedia.org/P41398 and previous config saved to /var/cache/conftool/dbconfig/20221128-171911-ladsgroup.json [production]
17:17 <jbond@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ms-be2050.codfw.wmnet with reason: host reimage [production]
17:17 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2148', diff saved to https://phabricator.wikimedia.org/P41397 and previous config saved to /var/cache/conftool/dbconfig/20221128-171659-ladsgroup.json [production]
17:14 <akosiaris@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:15:00 on mc-wf2002.codfw.wmnet with reason: Kernel upgrade [production]
17:14 <akosiaris@cumin1001> START - Cookbook sre.hosts.downtime for 0:15:00 on mc-wf2002.codfw.wmnet with reason: Kernel upgrade [production]
17:14 <akosiaris@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:15:00 on mc-wf2001.codfw.wmnet with reason: Kernel upgrade [production]
17:13 <akosiaris@cumin1001> START - Cookbook sre.hosts.downtime for 0:15:00 on mc-wf2001.codfw.wmnet with reason: Kernel upgrade [production]
17:13 <jbond@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on ms-be2050.codfw.wmnet with reason: host reimage [production]
17:12 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1146:3312', diff saved to https://phabricator.wikimedia.org/P41396 and previous config saved to /var/cache/conftool/dbconfig/20221128-171200-ladsgroup.json [production]
17:09 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2156', diff saved to https://phabricator.wikimedia.org/P41395 and previous config saved to /var/cache/conftool/dbconfig/20221128-170912-ladsgroup.json [production]
17:06 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2106 (T321126)', diff saved to https://phabricator.wikimedia.org/P41394 and previous config saved to /var/cache/conftool/dbconfig/20221128-170651-marostegui.json [production]
17:04 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db2106 (T321126)', diff saved to https://phabricator.wikimedia.org/P41393 and previous config saved to /var/cache/conftool/dbconfig/20221128-170438-marostegui.json [production]
17:04 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2106.codfw.wmnet with reason: Maintenance [production]
17:04 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 5:00:00 on db2106.codfw.wmnet with reason: Maintenance [production]
17:04 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2099.codfw.wmnet with reason: Maintenance [production]
17:04 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 5:00:00 on db2099.codfw.wmnet with reason: Maintenance [production]
17:03 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance [production]
17:03 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 5:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance [production]