4201-4250 of 10000 results (57ms)
2022-04-20 ยง
12:21 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host puppetmaster1005.eqiad.wmnet [production]
12:20 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1134 (T306269)', diff saved to https://phabricator.wikimedia.org/P25659 and previous config saved to /var/cache/conftool/dbconfig/20220420-122000-marostegui.json [production]
12:17 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1134 (T306269)', diff saved to https://phabricator.wikimedia.org/P25658 and previous config saved to /var/cache/conftool/dbconfig/20220420-121745-marostegui.json [production]
12:17 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1134.eqiad.wmnet with reason: Maintenance [production]
12:17 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1134.eqiad.wmnet with reason: Maintenance [production]
12:17 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1119 (T306269)', diff saved to https://phabricator.wikimedia.org/P25657 and previous config saved to /var/cache/conftool/dbconfig/20220420-121737-marostegui.json [production]
12:17 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host puppetmaster1005.eqiad.wmnet [production]
12:13 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host moss-fe1002.eqiad.wmnet [production]
12:07 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host moss-fe1002.eqiad.wmnet [production]
12:04 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host moss-fe1001.eqiad.wmnet [production]
12:02 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1119', diff saved to https://phabricator.wikimedia.org/P25656 and previous config saved to /var/cache/conftool/dbconfig/20220420-120232-marostegui.json [production]
11:59 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host moss-fe1001.eqiad.wmnet [production]
11:57 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host moss-fe2002.codfw.wmnet [production]
11:51 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host moss-fe2002.codfw.wmnet [production]
11:47 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1119', diff saved to https://phabricator.wikimedia.org/P25655 and previous config saved to /var/cache/conftool/dbconfig/20220420-114727-marostegui.json [production]
11:43 <kormat@cumin1001> dbctl commit (dc=all): 'es1031 (re)pooling @ 100%: repooling T303174', diff saved to https://phabricator.wikimedia.org/P25654 and previous config saved to /var/cache/conftool/dbconfig/20220420-114326-kormat.json [production]
11:41 <kormat@cumin1001> dbctl commit (dc=all): 'db1143 (re)pooling @ 100%: Reboot T303174', diff saved to https://phabricator.wikimedia.org/P25653 and previous config saved to /var/cache/conftool/dbconfig/20220420-114159-kormat.json [production]
11:35 <kormat@cumin1001> dbctl commit (dc=all): 'es1032 (re)pooling @ 100%: repooling T303174', diff saved to https://phabricator.wikimedia.org/P25652 and previous config saved to /var/cache/conftool/dbconfig/20220420-113547-kormat.json [production]
11:35 <kormat@cumin1001> dbctl commit (dc=all): 'db1127 (re)pooling @ 100%: repooling T303174', diff saved to https://phabricator.wikimedia.org/P25651 and previous config saved to /var/cache/conftool/dbconfig/20220420-113503-kormat.json [production]
11:34 <kormat@cumin1001> dbctl commit (dc=all): 'db1165 (re)pooling @ 100%: Reboot T303174', diff saved to https://phabricator.wikimedia.org/P25650 and previous config saved to /var/cache/conftool/dbconfig/20220420-113432-kormat.json [production]
11:34 <hnowlan@puppetmaster1001> conftool action : set/pooled=true; selector: dnsdisc=kartotherian,name=eqiad [production]
11:33 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host moss-fe2001.codfw.wmnet [production]
11:32 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1119 (T306269)', diff saved to https://phabricator.wikimedia.org/P25649 and previous config saved to /var/cache/conftool/dbconfig/20220420-113219-marostegui.json [production]
11:30 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1119 (T306269)', diff saved to https://phabricator.wikimedia.org/P25648 and previous config saved to /var/cache/conftool/dbconfig/20220420-113000-marostegui.json [production]
11:30 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1119.eqiad.wmnet with reason: Maintenance [production]
11:29 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1119.eqiad.wmnet with reason: Maintenance [production]
11:29 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1099:3311 (T306269)', diff saved to https://phabricator.wikimedia.org/P25647 and previous config saved to /var/cache/conftool/dbconfig/20220420-112952-marostegui.json [production]
11:29 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host moss-fe2001.codfw.wmnet [production]
11:28 <kormat@cumin1001> dbctl commit (dc=all): 'es1031 (re)pooling @ 75%: repooling T303174', diff saved to https://phabricator.wikimedia.org/P25646 and previous config saved to /var/cache/conftool/dbconfig/20220420-112823-kormat.json [production]
11:26 <kormat@cumin1001> dbctl commit (dc=all): 'db1143 (re)pooling @ 75%: Reboot T303174', diff saved to https://phabricator.wikimedia.org/P25645 and previous config saved to /var/cache/conftool/dbconfig/20220420-112655-kormat.json [production]
11:26 <hnowlan@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:15:00 on 6 hosts with reason: postgres config change [production]
11:26 <hnowlan@cumin1001> START - Cookbook sre.hosts.downtime for 0:15:00 on 6 hosts with reason: postgres config change [production]
11:25 <hnowlan@puppetmaster1001> conftool action : set/pooled=false; selector: dnsdisc=kartotherian,name=eqiad [production]
11:25 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host moss-be2002.codfw.wmnet [production]
11:20 <kormat@cumin1001> dbctl commit (dc=all): 'es1032 (re)pooling @ 75%: repooling T303174', diff saved to https://phabricator.wikimedia.org/P25644 and previous config saved to /var/cache/conftool/dbconfig/20220420-112043-kormat.json [production]
11:19 <kormat@cumin1001> dbctl commit (dc=all): 'db1127 (re)pooling @ 75%: repooling T303174', diff saved to https://phabricator.wikimedia.org/P25643 and previous config saved to /var/cache/conftool/dbconfig/20220420-111959-kormat.json [production]
11:19 <kormat@cumin1001> dbctl commit (dc=all): 'db1165 (re)pooling @ 75%: Reboot T303174', diff saved to https://phabricator.wikimedia.org/P25642 and previous config saved to /var/cache/conftool/dbconfig/20220420-111928-kormat.json [production]
11:19 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host moss-be2002.codfw.wmnet [production]
11:19 <kormat@cumin1001> dbctl commit (dc=all): 'db1167 (re)pooling @ 100%: Reboot T303174', diff saved to https://phabricator.wikimedia.org/P25641 and previous config saved to /var/cache/conftool/dbconfig/20220420-111911-kormat.json [production]
11:18 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host moss-be2001.codfw.wmnet [production]
11:16 <kormat@cumin1001> dbctl commit (dc=all): 'db1156 (re)pooling @ 100%: Reboot T303174', diff saved to https://phabricator.wikimedia.org/P25640 and previous config saved to /var/cache/conftool/dbconfig/20220420-111626-kormat.json [production]
11:14 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1099:3311', diff saved to https://phabricator.wikimedia.org/P25639 and previous config saved to /var/cache/conftool/dbconfig/20220420-111447-marostegui.json [production]
11:13 <kormat@cumin1001> dbctl commit (dc=all): 'es1031 (re)pooling @ 50%: repooling T303174', diff saved to https://phabricator.wikimedia.org/P25638 and previous config saved to /var/cache/conftool/dbconfig/20220420-111319-kormat.json [production]
11:12 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host moss-be2001.codfw.wmnet [production]
11:11 <kormat@cumin1001> dbctl commit (dc=all): 'db1143 (re)pooling @ 50%: Reboot T303174', diff saved to https://phabricator.wikimedia.org/P25637 and previous config saved to /var/cache/conftool/dbconfig/20220420-111150-kormat.json [production]
11:05 <kormat@cumin1001> dbctl commit (dc=all): 'es1032 (re)pooling @ 50%: repooling T303174', diff saved to https://phabricator.wikimedia.org/P25636 and previous config saved to /var/cache/conftool/dbconfig/20220420-110539-kormat.json [production]
11:04 <kormat@cumin1001> dbctl commit (dc=all): 'db1127 (re)pooling @ 50%: repooling T303174', diff saved to https://phabricator.wikimedia.org/P25635 and previous config saved to /var/cache/conftool/dbconfig/20220420-110455-kormat.json [production]
11:04 <kormat@cumin1001> dbctl commit (dc=all): 'db1165 (re)pooling @ 50%: Reboot T303174', diff saved to https://phabricator.wikimedia.org/P25634 and previous config saved to /var/cache/conftool/dbconfig/20220420-110424-kormat.json [production]
11:04 <kormat@cumin1001> dbctl commit (dc=all): 'db1167 (re)pooling @ 75%: Reboot T303174', diff saved to https://phabricator.wikimedia.org/P25633 and previous config saved to /var/cache/conftool/dbconfig/20220420-110408-kormat.json [production]
11:01 <kormat@cumin1001> dbctl commit (dc=all): 'db1156 (re)pooling @ 75%: Reboot T303174', diff saved to https://phabricator.wikimedia.org/P25632 and previous config saved to /var/cache/conftool/dbconfig/20220420-110122-kormat.json [production]