801-850 of 10000 results (32ms)
2021-12-14 ยง
13:08 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 4:00:00 on db1099.eqiad.wmnet with reason: Maintenance [production]
13:08 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1105:3311 (T277354)', diff saved to https://phabricator.wikimedia.org/P18224 and previous config saved to /var/cache/conftool/dbconfig/20211214-130845-marostegui.json [production]
12:57 <ayounsi@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts netflow5001.eqsin.wmnet [production]
12:53 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1105:3311', diff saved to https://phabricator.wikimedia.org/P18223 and previous config saved to /var/cache/conftool/dbconfig/20211214-125340-marostegui.json [production]
12:45 <ayounsi@cumin1001> START - Cookbook sre.hosts.decommission for hosts netflow5001.eqsin.wmnet [production]
12:44 <ayounsi@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts netflow3001.esams.wmnet [production]
12:38 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1105:3311', diff saved to https://phabricator.wikimedia.org/P18222 and previous config saved to /var/cache/conftool/dbconfig/20211214-123836-marostegui.json [production]
12:27 <ayounsi@cumin1001> START - Cookbook sre.hosts.decommission for hosts netflow3001.esams.wmnet [production]
12:24 <ayounsi@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts netflow1001.eqiad.wmnet [production]
12:23 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1105:3311 (T277354)', diff saved to https://phabricator.wikimedia.org/P18221 and previous config saved to /var/cache/conftool/dbconfig/20211214-122331-marostegui.json [production]
12:21 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1105:3311 (T277354)', diff saved to https://phabricator.wikimedia.org/P18220 and previous config saved to /var/cache/conftool/dbconfig/20211214-122137-marostegui.json [production]
12:21 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1105.eqiad.wmnet with reason: Maintenance [production]
12:21 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 4:00:00 on db1105.eqiad.wmnet with reason: Maintenance [production]
12:21 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1106 (T277354)', diff saved to https://phabricator.wikimedia.org/P18219 and previous config saved to /var/cache/conftool/dbconfig/20211214-122129-marostegui.json [production]
12:19 <moritzm> drain primary/secondary instances off ganeti2018 T296622 [production]
12:14 <ayounsi@cumin1001> START - Cookbook sre.hosts.decommission for hosts netflow1001.eqiad.wmnet [production]
12:12 <_joe_> regenerating pybal config-master files [production]
12:11 <Lucas_WMDE> UTC morning backport+config window done [production]
12:07 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
12:07 <lucaswerkmeister-wmde@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:746930|Set wgLexemeEnableDataTransclusion to false everywhere (T294159)]] (no-op) (duration: 00m 59s) [production]
12:06 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
12:06 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1106', diff saved to https://phabricator.wikimedia.org/P18218 and previous config saved to /var/cache/conftool/dbconfig/20211214-120624-marostegui.json [production]
12:04 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2017.codfw.wmnet [production]
12:01 <ayounsi@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts netflow2001.codfw.wmnet [production]
11:58 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host ganeti2017.codfw.wmnet [production]
11:51 <ayounsi@cumin1001> START - Cookbook sre.hosts.decommission for hosts netflow2001.codfw.wmnet [production]
11:51 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1106', diff saved to https://phabricator.wikimedia.org/P18217 and previous config saved to /var/cache/conftool/dbconfig/20211214-115119-marostegui.json [production]
11:41 <hnowlan@puppetmaster1001> conftool action : set/pooled=yes; selector: name=restbase2025.codfw.wmnet [production]
11:40 <hnowlan@puppetmaster1001> conftool action : set/weight=10; selector: name=restbase2025.codfw.wmnet [production]
11:36 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1106 (T277354)', diff saved to https://phabricator.wikimedia.org/P18216 and previous config saved to /var/cache/conftool/dbconfig/20211214-113615-marostegui.json [production]
11:35 <hnowlan> joining restbase2026-b to cassandra [production]
11:34 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2008.codfw.wmnet [production]
11:34 <jbond> move idp service to eqiad [production]
11:34 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1106 (T277354)', diff saved to https://phabricator.wikimedia.org/P18215 and previous config saved to /var/cache/conftool/dbconfig/20211214-113428-marostegui.json [production]
11:34 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on clouddb[1013,1017,1021].eqiad.wmnet,db[1106,1154].eqiad.wmnet with reason: Maintenance [production]
11:34 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 4:00:00 on clouddb[1013,1017,1021].eqiad.wmnet,db[1106,1154].eqiad.wmnet with reason: Maintenance [production]
11:34 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1119 (T277354)', diff saved to https://phabricator.wikimedia.org/P18214 and previous config saved to /var/cache/conftool/dbconfig/20211214-113417-marostegui.json [production]
11:30 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host ganeti2008.codfw.wmnet [production]
11:19 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1119', diff saved to https://phabricator.wikimedia.org/P18213 and previous config saved to /var/cache/conftool/dbconfig/20211214-111912-marostegui.json [production]
11:14 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2021.codfw.wmnet [production]
11:09 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host ganeti2021.codfw.wmnet [production]
11:04 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1119', diff saved to https://phabricator.wikimedia.org/P18212 and previous config saved to /var/cache/conftool/dbconfig/20211214-110407-marostegui.json [production]
10:49 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1119 (T277354)', diff saved to https://phabricator.wikimedia.org/P18211 and previous config saved to /var/cache/conftool/dbconfig/20211214-104903-marostegui.json [production]
10:47 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1119 (T277354)', diff saved to https://phabricator.wikimedia.org/P18210 and previous config saved to /var/cache/conftool/dbconfig/20211214-104717-marostegui.json [production]
10:47 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1119.eqiad.wmnet with reason: Maintenance [production]
10:47 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 4:00:00 on db1119.eqiad.wmnet with reason: Maintenance [production]
10:46 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1128.eqiad.wmnet with reason: Maintenance [production]
10:46 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 4:00:00 on db1128.eqiad.wmnet with reason: Maintenance [production]
10:45 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1133.eqiad.wmnet with reason: Maintenance [production]
10:45 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 4:00:00 on db1133.eqiad.wmnet with reason: Maintenance [production]