751-800 of 10000 results (37ms)
2022-04-20 ยง
06:24 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on 6 hosts with reason: Maintenance [production]
06:24 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 12:00:00 on 6 hosts with reason: Maintenance [production]
06:24 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2105.codfw.wmnet with reason: Maintenance [production]
06:24 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db2105.codfw.wmnet with reason: Maintenance [production]
06:22 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1105:3311 (T306269)', diff saved to https://phabricator.wikimedia.org/P25553 and previous config saved to /var/cache/conftool/dbconfig/20220420-062206-marostegui.json [production]
06:21 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1096:3315 (T298565)', diff saved to https://phabricator.wikimedia.org/P25552 and previous config saved to /var/cache/conftool/dbconfig/20220420-062133-ladsgroup.json [production]
06:18 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1105:3311 (T306269)', diff saved to https://phabricator.wikimedia.org/P25551 and previous config saved to /var/cache/conftool/dbconfig/20220420-061848-marostegui.json [production]
06:18 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1105.eqiad.wmnet with reason: Maintenance [production]
06:18 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1105.eqiad.wmnet with reason: Maintenance [production]
06:18 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1140.eqiad.wmnet with reason: Maintenance [production]
06:18 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1140.eqiad.wmnet with reason: Maintenance [production]
06:18 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on dbstore1003.eqiad.wmnet with reason: Maintenance [production]
06:18 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on dbstore1003.eqiad.wmnet with reason: Maintenance [production]
06:18 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1163 (T306269)', diff saved to https://phabricator.wikimedia.org/P25550 and previous config saved to /var/cache/conftool/dbconfig/20220420-061834-marostegui.json [production]
06:14 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1096:3315 (T298565)', diff saved to https://phabricator.wikimedia.org/P25549 and previous config saved to /var/cache/conftool/dbconfig/20220420-061433-ladsgroup.json [production]
06:14 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1096.eqiad.wmnet with reason: Maintenance [production]
06:14 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1096.eqiad.wmnet with reason: Maintenance [production]
06:14 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1144:3315 (T298565)', diff saved to https://phabricator.wikimedia.org/P25548 and previous config saved to /var/cache/conftool/dbconfig/20220420-061425-ladsgroup.json [production]
06:07 <marostegui@cumin1001> dbctl commit (dc=all): 'db1136 (re)pooling @ 100%: After reimage', diff saved to https://phabricator.wikimedia.org/P25547 and previous config saved to /var/cache/conftool/dbconfig/20220420-060732-root.json [production]
06:03 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1163', diff saved to https://phabricator.wikimedia.org/P25546 and previous config saved to /var/cache/conftool/dbconfig/20220420-060329-marostegui.json [production]
05:59 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1144:3315', diff saved to https://phabricator.wikimedia.org/P25545 and previous config saved to /var/cache/conftool/dbconfig/20220420-055920-ladsgroup.json [production]
05:52 <marostegui@cumin1001> dbctl commit (dc=all): 'db1136 (re)pooling @ 75%: After reimage', diff saved to https://phabricator.wikimedia.org/P25544 and previous config saved to /var/cache/conftool/dbconfig/20220420-055228-root.json [production]
05:48 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1163', diff saved to https://phabricator.wikimedia.org/P25543 and previous config saved to /var/cache/conftool/dbconfig/20220420-054824-marostegui.json [production]
05:44 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1144:3315', diff saved to https://phabricator.wikimedia.org/P25542 and previous config saved to /var/cache/conftool/dbconfig/20220420-054415-ladsgroup.json [production]
05:39 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance [production]
05:39 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance [production]
05:39 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166 (T298565)', diff saved to https://phabricator.wikimedia.org/P25541 and previous config saved to /var/cache/conftool/dbconfig/20220420-053932-ladsgroup.json [production]
05:39 <ayounsi@cumin2002> END (PASS) - Cookbook sre.network.cf (exit_code=0) [production]
05:39 <ayounsi@cumin2002> START - Cookbook sre.network.cf [production]
05:38 <XioNoX> start CF in monitoring mode for drmrs [production]
05:37 <marostegui@cumin1001> dbctl commit (dc=all): 'db1136 (re)pooling @ 50%: After reimage', diff saved to https://phabricator.wikimedia.org/P25540 and previous config saved to /var/cache/conftool/dbconfig/20220420-053724-root.json [production]
05:33 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1163 (T306269)', diff saved to https://phabricator.wikimedia.org/P25539 and previous config saved to /var/cache/conftool/dbconfig/20220420-053319-marostegui.json [production]
05:30 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1163 (T306269)', diff saved to https://phabricator.wikimedia.org/P25538 and previous config saved to /var/cache/conftool/dbconfig/20220420-053006-marostegui.json [production]
05:30 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1163.eqiad.wmnet with reason: Maintenance [production]
05:30 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1163.eqiad.wmnet with reason: Maintenance [production]
05:29 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1169 (T306269)', diff saved to https://phabricator.wikimedia.org/P25537 and previous config saved to /var/cache/conftool/dbconfig/20220420-052958-marostegui.json [production]
05:29 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1144:3315 (T298565)', diff saved to https://phabricator.wikimedia.org/P25536 and previous config saved to /var/cache/conftool/dbconfig/20220420-052910-ladsgroup.json [production]
05:24 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166', diff saved to https://phabricator.wikimedia.org/P25535 and previous config saved to /var/cache/conftool/dbconfig/20220420-052427-ladsgroup.json [production]
05:22 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1144:3315 (T298565)', diff saved to https://phabricator.wikimedia.org/P25534 and previous config saved to /var/cache/conftool/dbconfig/20220420-052223-ladsgroup.json [production]
05:22 <marostegui@cumin1001> dbctl commit (dc=all): 'db1136 (re)pooling @ 25%: After reimage', diff saved to https://phabricator.wikimedia.org/P25533 and previous config saved to /var/cache/conftool/dbconfig/20220420-052220-root.json [production]
05:22 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1144.eqiad.wmnet with reason: Maintenance [production]
05:22 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1144.eqiad.wmnet with reason: Maintenance [production]
05:22 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1113:3315 (T298565)', diff saved to https://phabricator.wikimedia.org/P25532 and previous config saved to /var/cache/conftool/dbconfig/20220420-052215-ladsgroup.json [production]
05:14 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1169', diff saved to https://phabricator.wikimedia.org/P25531 and previous config saved to /var/cache/conftool/dbconfig/20220420-051453-marostegui.json [production]
05:09 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166', diff saved to https://phabricator.wikimedia.org/P25530 and previous config saved to /var/cache/conftool/dbconfig/20220420-050921-ladsgroup.json [production]
05:07 <marostegui@cumin1001> dbctl commit (dc=all): 'db1136 (re)pooling @ 10%: After reimage', diff saved to https://phabricator.wikimedia.org/P25529 and previous config saved to /var/cache/conftool/dbconfig/20220420-050716-root.json [production]
05:07 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1113:3315', diff saved to https://phabricator.wikimedia.org/P25528 and previous config saved to /var/cache/conftool/dbconfig/20220420-050710-ladsgroup.json [production]
04:59 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1169', diff saved to https://phabricator.wikimedia.org/P25527 and previous config saved to /var/cache/conftool/dbconfig/20220420-045948-marostegui.json [production]
04:54 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166 (T298565)', diff saved to https://phabricator.wikimedia.org/P25526 and previous config saved to /var/cache/conftool/dbconfig/20220420-045416-ladsgroup.json [production]
04:52 <marostegui@cumin1001> dbctl commit (dc=all): 'db1136 (re)pooling @ 5%: After reimage', diff saved to https://phabricator.wikimedia.org/P25525 and previous config saved to /var/cache/conftool/dbconfig/20220420-045212-root.json [production]