651-700 of 10000 results (81ms)
2024-04-05 ยง
14:29 <taavi@cumin1002> conftool action : set/pooled=no; selector: name=clouddb1013.eqiad.wmnet [production]
14:26 <taavi@cumin1002> conftool action : set/pooled=yes; selector: name=clouddb1020.eqiad.wmnet [production]
14:25 <taavi@cumin1002> conftool action : set/pooled=no; selector: name=clouddb1020.eqiad.wmnet [production]
14:25 <taavi@cumin1002> conftool action : set/pooled=yes; selector: name=clouddb1019.eqiad.wmnet [production]
14:22 <taavi@cumin1002> conftool action : set/pooled=no; selector: name=clouddb1019.eqiad.wmnet [production]
14:22 <taavi@cumin1002> conftool action : set/pooled=yes; selector: name=clouddb1018.eqiad.wmnet [production]
14:20 <taavi@cumin1002> conftool action : set/pooled=no; selector: name=clouddb1018.eqiad.wmnet [production]
14:19 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P59678 and previous config saved to /var/cache/conftool/dbconfig/20240405-141919-arnaudb.json [production]
14:18 <taavi@cumin1002> conftool action : set/pooled=yes; selector: name=clouddb1017.eqiad.wmnet [production]
14:15 <taavi@cumin1002> conftool action : set/pooled=no; selector: name=clouddb1017.eqiad.wmnet [production]
14:04 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1178 (T360332)', diff saved to https://phabricator.wikimedia.org/P59677 and previous config saved to /var/cache/conftool/dbconfig/20240405-140412-arnaudb.json [production]
13:03 <arnaudb@cumin1002> dbctl commit (dc=all): 'Depooling db1178 (T360332)', diff saved to https://phabricator.wikimedia.org/P59676 and previous config saved to /var/cache/conftool/dbconfig/20240405-130347-arnaudb.json [production]
13:03 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1178.eqiad.wmnet with reason: Maintenance [production]
13:03 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 12:00:00 on db1178.eqiad.wmnet with reason: Maintenance [production]
13:03 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1177 (T360332)', diff saved to https://phabricator.wikimedia.org/P59675 and previous config saved to /var/cache/conftool/dbconfig/20240405-130324-arnaudb.json [production]
12:48 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P59674 and previous config saved to /var/cache/conftool/dbconfig/20240405-124816-arnaudb.json [production]
12:33 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P59673 and previous config saved to /var/cache/conftool/dbconfig/20240405-123309-arnaudb.json [production]
12:32 <fabfur@cumin1002> conftool action : set/pooled=yes; selector: name=cp4037.ulsfo.wmnet [production]
12:32 <fabfur> repool cp4037 (T361845) [production]
12:18 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1177 (T360332)', diff saved to https://phabricator.wikimedia.org/P59672 and previous config saved to /var/cache/conftool/dbconfig/20240405-121801-arnaudb.json [production]
12:10 <ayounsi@cumin1002> END (FAIL) - Cookbook sre.hosts.decommission (exit_code=1) for hosts testvm2006.codfw.wmnet [production]
12:10 <ayounsi@cumin1002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
12:10 <ayounsi@cumin1002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: testvm2006.codfw.wmnet decommissioned, removing all IPs except the asset tag one - ayounsi@cumin1002" [production]
11:17 <arnaudb@cumin1002> dbctl commit (dc=all): 'Depooling db1177 (T360332)', diff saved to https://phabricator.wikimedia.org/P59671 and previous config saved to /var/cache/conftool/dbconfig/20240405-111736-arnaudb.json [production]
11:17 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1177.eqiad.wmnet with reason: Maintenance [production]
11:17 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 12:00:00 on db1177.eqiad.wmnet with reason: Maintenance [production]
11:17 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1172 (T360332)', diff saved to https://phabricator.wikimedia.org/P59670 and previous config saved to /var/cache/conftool/dbconfig/20240405-111713-arnaudb.json [production]
11:02 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P59669 and previous config saved to /var/cache/conftool/dbconfig/20240405-110204-arnaudb.json [production]
10:46 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P59668 and previous config saved to /var/cache/conftool/dbconfig/20240405-104657-arnaudb.json [production]
10:31 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1172 (T360332)', diff saved to https://phabricator.wikimedia.org/P59667 and previous config saved to /var/cache/conftool/dbconfig/20240405-103149-arnaudb.json [production]
09:51 <ayounsi@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: testvm2006.codfw.wmnet decommissioned, removing all IPs except the asset tag one - ayounsi@cumin1002" [production]
09:38 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on dbstore1008.eqiad.wmnet with reason: Maintenance [production]
09:38 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 8:00:00 on dbstore1008.eqiad.wmnet with reason: Maintenance [production]
09:38 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1240.eqiad.wmnet with reason: Maintenance [production]
09:38 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 8:00:00 on db1240.eqiad.wmnet with reason: Maintenance [production]
09:38 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1239.eqiad.wmnet with reason: Maintenance [production]
09:37 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 8:00:00 on db1239.eqiad.wmnet with reason: Maintenance [production]
09:37 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1235 (T356166)', diff saved to https://phabricator.wikimedia.org/P59666 and previous config saved to /var/cache/conftool/dbconfig/20240405-093745-marostegui.json [production]
09:31 <arnaudb@cumin1002> dbctl commit (dc=all): 'Depooling db1172 (T360332)', diff saved to https://phabricator.wikimedia.org/P59665 and previous config saved to /var/cache/conftool/dbconfig/20240405-093124-arnaudb.json [production]
09:31 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1172.eqiad.wmnet with reason: Maintenance [production]
09:31 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 12:00:00 on db1172.eqiad.wmnet with reason: Maintenance [production]
09:30 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1171.eqiad.wmnet with reason: Maintenance [production]
09:30 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 12:00:00 on db1171.eqiad.wmnet with reason: Maintenance [production]
09:30 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1167 (T360332)', diff saved to https://phabricator.wikimedia.org/P59664 and previous config saved to /var/cache/conftool/dbconfig/20240405-093038-arnaudb.json [production]
09:24 <ayounsi@cumin1002> START - Cookbook sre.dns.netbox [production]
09:22 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1235', diff saved to https://phabricator.wikimedia.org/P59663 and previous config saved to /var/cache/conftool/dbconfig/20240405-092237-marostegui.json [production]
09:19 <ayounsi@cumin1002> START - Cookbook sre.hosts.decommission for hosts testvm2006.codfw.wmnet [production]
09:15 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P59662 and previous config saved to /var/cache/conftool/dbconfig/20240405-091531-arnaudb.json [production]
09:07 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1235', diff saved to https://phabricator.wikimedia.org/P59661 and previous config saved to /var/cache/conftool/dbconfig/20240405-090730-marostegui.json [production]
09:00 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P59660 and previous config saved to /var/cache/conftool/dbconfig/20240405-090023-arnaudb.json [production]