501-550 of 10000 results (67ms)
2022-11-02 ยง
10:23 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db2129.codfw.wmnet with reason: Maintenance [production]
10:23 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2124 (T318950)', diff saved to https://phabricator.wikimedia.org/P37630 and previous config saved to /var/cache/conftool/dbconfig/20221102-102256-ladsgroup.json [production]
10:23 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 8:00:00 on db2126.codfw.wmnet with reason: Maintenance [production]
10:23 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2125 (T318955)', diff saved to https://phabricator.wikimedia.org/P37629 and previous config saved to /var/cache/conftool/dbconfig/20221102-102243-ladsgroup.json [production]
10:22 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1113:3316 (T318950)', diff saved to https://phabricator.wikimedia.org/P37628 and previous config saved to /var/cache/conftool/dbconfig/20221102-102233-ladsgroup.json [production]
10:22 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1113.eqiad.wmnet with reason: Maintenance [production]
10:22 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1113.eqiad.wmnet with reason: Maintenance [production]
10:22 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1098:3316 (T318950)', diff saved to https://phabricator.wikimedia.org/P37627 and previous config saved to /var/cache/conftool/dbconfig/20221102-102221-ladsgroup.json [production]
10:19 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1144:3315', diff saved to https://phabricator.wikimedia.org/P37626 and previous config saved to /var/cache/conftool/dbconfig/20221102-101946-marostegui.json [production]
10:19 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1129', diff saved to https://phabricator.wikimedia.org/P37625 and previous config saved to /var/cache/conftool/dbconfig/20221102-101916-ladsgroup.json [production]
10:07 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2124', diff saved to https://phabricator.wikimedia.org/P37624 and previous config saved to /var/cache/conftool/dbconfig/20221102-100746-ladsgroup.json [production]
10:07 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2125', diff saved to https://phabricator.wikimedia.org/P37623 and previous config saved to /var/cache/conftool/dbconfig/20221102-100736-ladsgroup.json [production]
10:07 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1098:3316', diff saved to https://phabricator.wikimedia.org/P37622 and previous config saved to /var/cache/conftool/dbconfig/20221102-100713-ladsgroup.json [production]
10:04 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1144:3315', diff saved to https://phabricator.wikimedia.org/P37621 and previous config saved to /var/cache/conftool/dbconfig/20221102-100438-marostegui.json [production]
10:04 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1129 (T318955)', diff saved to https://phabricator.wikimedia.org/P37620 and previous config saved to /var/cache/conftool/dbconfig/20221102-100408-ladsgroup.json [production]
10:02 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1129 (T318955)', diff saved to https://phabricator.wikimedia.org/P37619 and previous config saved to /var/cache/conftool/dbconfig/20221102-100156-ladsgroup.json [production]
10:02 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1129.eqiad.wmnet with reason: Maintenance [production]
10:01 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 8:00:00 on db1129.eqiad.wmnet with reason: Maintenance [production]
10:01 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1105:3312 (T318955)', diff saved to https://phabricator.wikimedia.org/P37618 and previous config saved to /var/cache/conftool/dbconfig/20221102-100133-ladsgroup.json [production]
09:54 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2124', diff saved to https://phabricator.wikimedia.org/P37617 and previous config saved to /var/cache/conftool/dbconfig/20221102-095237-ladsgroup.json [production]
09:52 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2125', diff saved to https://phabricator.wikimedia.org/P37616 and previous config saved to /var/cache/conftool/dbconfig/20221102-095225-ladsgroup.json [production]
09:52 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1098:3316', diff saved to https://phabricator.wikimedia.org/P37615 and previous config saved to /var/cache/conftool/dbconfig/20221102-095205-ladsgroup.json [production]
09:51 <moritzm> installing exim4 security updates [production]
09:49 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1144:3315 (T321123)', diff saved to https://phabricator.wikimedia.org/P37614 and previous config saved to /var/cache/conftool/dbconfig/20221102-094928-marostegui.json [production]
09:47 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1144:3315 (T321123)', diff saved to https://phabricator.wikimedia.org/P37613 and previous config saved to /var/cache/conftool/dbconfig/20221102-094709-marostegui.json [production]
09:46 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1144.eqiad.wmnet with reason: Maintenance [production]
09:46 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 8:00:00 on db1144.eqiad.wmnet with reason: Maintenance [production]
09:46 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1113:3315 (T321123)', diff saved to https://phabricator.wikimedia.org/P37612 and previous config saved to /var/cache/conftool/dbconfig/20221102-094644-marostegui.json [production]
09:46 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1105:3312', diff saved to https://phabricator.wikimedia.org/P37611 and previous config saved to /var/cache/conftool/dbconfig/20221102-094622-ladsgroup.json [production]
09:41 <jmm@cumin2002> END (FAIL) - Cookbook sre.ganeti.addnode (exit_code=99) for new host ganeti1016.eqiad.wmnet to cluster eqiad and group B [production]
09:40 <jmm@cumin2002> START - Cookbook sre.ganeti.addnode for new host ganeti1016.eqiad.wmnet to cluster eqiad and group B [production]
09:38 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2124 (T318950)', diff saved to https://phabricator.wikimedia.org/P37610 and previous config saved to /var/cache/conftool/dbconfig/20221102-093730-ladsgroup.json [production]
09:37 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2127.codfw.wmnet with reason: Maintenance [production]
09:37 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2125 (T318955)', diff saved to https://phabricator.wikimedia.org/P37609 and previous config saved to /var/cache/conftool/dbconfig/20221102-093717-ladsgroup.json [production]
09:37 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db2127.codfw.wmnet with reason: Maintenance [production]
09:37 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1098:3316 (T318950)', diff saved to https://phabricator.wikimedia.org/P37608 and previous config saved to /var/cache/conftool/dbconfig/20221102-093657-ladsgroup.json [production]
09:36 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti1016.eqiad.wmnet [production]
09:36 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2124 (T318950)', diff saved to https://phabricator.wikimedia.org/P37607 and previous config saved to /var/cache/conftool/dbconfig/20221102-093517-ladsgroup.json [production]
09:36 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1123.eqiad.wmnet with reason: Maintenance [production]
09:35 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2124.codfw.wmnet with reason: Maintenance [production]
09:35 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1123.eqiad.wmnet with reason: Maintenance [production]
09:35 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2125 (T318955)', diff saved to https://phabricator.wikimedia.org/P37606 and previous config saved to /var/cache/conftool/dbconfig/20221102-093459-ladsgroup.json [production]
09:35 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db2124.codfw.wmnet with reason: Maintenance [production]
09:35 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2117 (T318950)', diff saved to https://phabricator.wikimedia.org/P37605 and previous config saved to /var/cache/conftool/dbconfig/20221102-093453-ladsgroup.json [production]
09:35 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db2125.codfw.wmnet with reason: Maintenance [production]
09:35 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1098:3316 (T318950)', diff saved to https://phabricator.wikimedia.org/P37604 and previous config saved to /var/cache/conftool/dbconfig/20221102-093443-ladsgroup.json [production]
09:35 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 8:00:00 on db2125.codfw.wmnet with reason: Maintenance [production]
09:35 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2104 (T318955)', diff saved to https://phabricator.wikimedia.org/P37603 and previous config saved to /var/cache/conftool/dbconfig/20221102-093436-ladsgroup.json [production]
09:35 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1098.eqiad.wmnet with reason: Maintenance [production]
09:34 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1098.eqiad.wmnet with reason: Maintenance [production]