2601-2650 of 10000 results (58ms)
2022-06-28 §
07:25 <marostegui@cumin1001> dbctl commit (dc=all): 'db1149 (re)pooling @ 25%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30514 and previous config saved to /var/cache/conftool/dbconfig/20220628-072505-root.json [production]
07:20 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1165 (T298557)', diff saved to https://phabricator.wikimedia.org/P30513 and previous config saved to /var/cache/conftool/dbconfig/20220628-072024-marostegui.json [production]
07:14 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1142 (T309311)', diff saved to https://phabricator.wikimedia.org/P30512 and previous config saved to /var/cache/conftool/dbconfig/20220628-071433-ladsgroup.json [production]
07:14 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1142.eqiad.wmnet with reason: Maintenance [production]
07:14 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1142.eqiad.wmnet with reason: Maintenance [production]
07:12 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1165 (T298557)', diff saved to https://phabricator.wikimedia.org/P30511 and previous config saved to /var/cache/conftool/dbconfig/20220628-071210-marostegui.json [production]
07:12 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on clouddb[1015,1019,1021].eqiad.wmnet,db1155.eqiad.wmnet with reason: Maintenance [production]
07:12 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 12:00:00 on clouddb[1015,1019,1021].eqiad.wmnet,db1155.eqiad.wmnet with reason: Maintenance [production]
07:12 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1165.eqiad.wmnet with reason: Maintenance [production]
07:12 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1165.eqiad.wmnet with reason: Maintenance [production]
07:11 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1173 (T298557)', diff saved to https://phabricator.wikimedia.org/P30510 and previous config saved to /var/cache/conftool/dbconfig/20220628-071157-marostegui.json [production]
07:10 <marostegui@cumin1001> dbctl commit (dc=all): 'db1149 (re)pooling @ 10%: After kernel upgrade', diff saved to https://phabricator.wikimedia.org/P30509 and previous config saved to /var/cache/conftool/dbconfig/20220628-071001-root.json [production]
07:05 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1150.eqiad.wmnet with reason: Maintenance [production]
07:05 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1150.eqiad.wmnet with reason: Maintenance [production]
07:05 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1121 (T309311)', diff saved to https://phabricator.wikimedia.org/P30508 and previous config saved to /var/cache/conftool/dbconfig/20220628-070525-ladsgroup.json [production]
07:02 <marostegui> dbmaint s7@eqiad T302659 [production]
07:02 <marostegui> dbmaint s7@eqiad T310011 [production]
06:00 <marostegui> Starting s7 eqiad failover from db1136 to db1181 - T311033 [production]
05:29 <marostegui> dbmaint s6@codfw T298557 [production]
2022-06-27 §
23:51 <cmjohnson@cumin1001> START - Cookbook sre.hosts.provision for host mw1474.mgmt.eqiad.wmnet with reboot policy FORCED [production]
23:51 <cmjohnson@cumin1001> START - Cookbook sre.hosts.provision for host mw1470.mgmt.eqiad.wmnet with reboot policy FORCED [production]
23:51 <cmjohnson@cumin1001> START - Cookbook sre.hosts.provision for host mw1468.mgmt.eqiad.wmnet with reboot policy FORCED [production]
23:51 <cmjohnson@cumin1001> START - Cookbook sre.hosts.provision for host mw1476.mgmt.eqiad.wmnet with reboot policy FORCED [production]
23:51 <cmjohnson@cumin1001> START - Cookbook sre.hosts.provision for host mw1472.mgmt.eqiad.wmnet with reboot policy FORCED [production]
23:51 <cmjohnson@cumin1001> START - Cookbook sre.hosts.provision for host mw1471.mgmt.eqiad.wmnet with reboot policy FORCED [production]
23:51 <cmjohnson@cumin1001> START - Cookbook sre.hosts.provision for host mw1469.mgmt.eqiad.wmnet with reboot policy FORCED [production]
23:51 <cmjohnson@cumin1001> START - Cookbook sre.hosts.provision for host mw1475.mgmt.eqiad.wmnet with reboot policy FORCED [production]
23:51 <cmjohnson@cumin1001> START - Cookbook sre.hosts.provision for host mw1467.mgmt.eqiad.wmnet with reboot policy FORCED [production]
23:51 <cmjohnson@cumin1001> START - Cookbook sre.hosts.provision for host mw1473.mgmt.eqiad.wmnet with reboot policy FORCED [production]
23:49 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host mw1457.mgmt.eqiad.wmnet with reboot policy FORCED [production]
23:49 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host mw1461.mgmt.eqiad.wmnet with reboot policy FORCED [production]
23:49 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host mw1464.mgmt.eqiad.wmnet with reboot policy FORCED [production]
23:49 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host mw1460.mgmt.eqiad.wmnet with reboot policy FORCED [production]
23:49 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host mw1465.mgmt.eqiad.wmnet with reboot policy FORCED [production]
23:49 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host mw1458.mgmt.eqiad.wmnet with reboot policy FORCED [production]
23:49 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host mw1459.mgmt.eqiad.wmnet with reboot policy FORCED [production]
23:49 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host mw1462.mgmt.eqiad.wmnet with reboot policy FORCED [production]
23:49 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host mw1466.mgmt.eqiad.wmnet with reboot policy FORCED [production]
23:49 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host mw1463.mgmt.eqiad.wmnet with reboot policy FORCED [production]
23:36 <cmjohnson@cumin1001> START - Cookbook sre.hosts.provision for host mw1465.mgmt.eqiad.wmnet with reboot policy FORCED [production]
23:36 <cmjohnson@cumin1001> START - Cookbook sre.hosts.provision for host mw1464.mgmt.eqiad.wmnet with reboot policy FORCED [production]
23:36 <cmjohnson@cumin1001> START - Cookbook sre.hosts.provision for host mw1457.mgmt.eqiad.wmnet with reboot policy FORCED [production]
23:36 <cmjohnson@cumin1001> START - Cookbook sre.hosts.provision for host mw1462.mgmt.eqiad.wmnet with reboot policy FORCED [production]
23:36 <cmjohnson@cumin1001> START - Cookbook sre.hosts.provision for host mw1463.mgmt.eqiad.wmnet with reboot policy FORCED [production]
23:36 <cmjohnson@cumin1001> START - Cookbook sre.hosts.provision for host mw1459.mgmt.eqiad.wmnet with reboot policy FORCED [production]
23:36 <cmjohnson@cumin1001> START - Cookbook sre.hosts.provision for host mw1460.mgmt.eqiad.wmnet with reboot policy FORCED [production]
23:36 <cmjohnson@cumin1001> START - Cookbook sre.hosts.provision for host mw1466.mgmt.eqiad.wmnet with reboot policy FORCED [production]
23:36 <cmjohnson@cumin1001> START - Cookbook sre.hosts.provision for host mw1458.mgmt.eqiad.wmnet with reboot policy FORCED [production]
23:22 <cmjohnson@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
23:18 <cmjohnson@cumin1001> START - Cookbook sre.dns.netbox [production]