|
2023-06-06
ยง
|
| 20:30 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1161', diff saved to https://phabricator.wikimedia.org/P48917 and previous config saved to /var/cache/conftool/dbconfig/20230606-203000-ladsgroup.json |
[production] |
| 20:19 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1167', diff saved to https://phabricator.wikimedia.org/P48916 and previous config saved to /var/cache/conftool/dbconfig/20230606-201950-ladsgroup.json |
[production] |
| 20:16 |
<mutante> |
miscweb1003, miscweb2003 - rm -rf /srv/org/wikimedia/sitemaps after removing httpd virtual host T338064 |
[production] |
| 20:14 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1161', diff saved to https://phabricator.wikimedia.org/P48915 and previous config saved to /var/cache/conftool/dbconfig/20230606-201454-ladsgroup.json |
[production] |
| 20:09 |
<jclark@cumin1001> |
START - Cookbook sre.hosts.reimage for host dbproxy1027.eqiad.wmnet with OS bullseye |
[production] |
| 20:09 |
<jclark@cumin1001> |
START - Cookbook sre.hosts.reimage for host dbproxy1026.eqiad.wmnet with OS bullseye |
[production] |
| 20:04 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1167 (T336886)', diff saved to https://phabricator.wikimedia.org/P48914 and previous config saved to /var/cache/conftool/dbconfig/20230606-200444-ladsgroup.json |
[production] |
| 19:59 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1161 (T336886)', diff saved to https://phabricator.wikimedia.org/P48913 and previous config saved to /var/cache/conftool/dbconfig/20230606-195948-ladsgroup.json |
[production] |
| 19:55 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1161 (T336886)', diff saved to https://phabricator.wikimedia.org/P48912 and previous config saved to /var/cache/conftool/dbconfig/20230606-195557-ladsgroup.json |
[production] |
| 19:55 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on clouddb[1016,1020-1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance |
[production] |
| 19:55 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on clouddb[1016,1020-1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance |
[production] |
| 19:55 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1161.eqiad.wmnet with reason: Maintenance |
[production] |
| 19:55 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 12:00:00 on db1161.eqiad.wmnet with reason: Maintenance |
[production] |
| 19:53 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1145.eqiad.wmnet with reason: Maintenance |
[production] |
| 19:53 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 12:00:00 on db1145.eqiad.wmnet with reason: Maintenance |
[production] |
| 19:53 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1144:3315 (T336886)', diff saved to https://phabricator.wikimedia.org/P48911 and previous config saved to /var/cache/conftool/dbconfig/20230606-195320-ladsgroup.json |
[production] |
| 19:38 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1144:3315', diff saved to https://phabricator.wikimedia.org/P48910 and previous config saved to /var/cache/conftool/dbconfig/20230606-193814-ladsgroup.json |
[production] |
| 19:23 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1144:3315', diff saved to https://phabricator.wikimedia.org/P48909 and previous config saved to /var/cache/conftool/dbconfig/20230606-192308-ladsgroup.json |
[production] |
| 19:09 |
<andrewbogott> |
also increased RAM and secgroup-rule quota for Trove T337882 |
[admin] |
| 19:08 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1144:3315 (T336886)', diff saved to https://phabricator.wikimedia.org/P48908 and previous config saved to /var/cache/conftool/dbconfig/20230606-190802-ladsgroup.json |
[production] |
| 19:06 |
<andrewbogott> |
increased trove secgroups, instances, volumes quotas from 40 to 100. Trove is too popular! T337882 |
[admin] |
| 19:04 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1167 (T336886)', diff saved to https://phabricator.wikimedia.org/P48907 and previous config saved to /var/cache/conftool/dbconfig/20230606-190420-ladsgroup.json |
[production] |
| 19:04 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on clouddb[1016,1020-1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance |
[production] |
| 19:04 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on clouddb[1016,1020-1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance |
[production] |
| 19:04 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1167.eqiad.wmnet with reason: Maintenance |
[production] |
| 19:04 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 12:00:00 on db1167.eqiad.wmnet with reason: Maintenance |
[production] |
| 19:04 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1144:3315 (T336886)', diff saved to https://phabricator.wikimedia.org/P48906 and previous config saved to /var/cache/conftool/dbconfig/20230606-190402-ladsgroup.json |
[production] |
| 19:03 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1144.eqiad.wmnet with reason: Maintenance |
[production] |
| 19:03 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 12:00:00 on db1144.eqiad.wmnet with reason: Maintenance |
[production] |
| 18:42 |
<wm-bot> |
<jeanfred> Deploy 19b5d50 |
[tools.wikiloves] |
| 18:31 |
<wm-bot2> |
Restarting openstack services on cloudbackup2001: ['cinder-backup'] - cookbook ran by andrew@bullseye |
[admin] |
| 18:31 |
<wm-bot2> |
Restarting openstack services on cloudbackup2002: ['cinder-backup'] - cookbook ran by andrew@bullseye |
[admin] |
| 18:30 |
<wm-bot2> |
Restarting openstack services on cloudvirtlocal1003: ['nova-compute'] - cookbook ran by andrew@bullseye |
[admin] |
| 18:30 |
<wm-bot2> |
Restarting openstack services on cloudvirtlocal1002: ['nova-compute'] - cookbook ran by andrew@bullseye |
[admin] |
| 18:30 |
<wm-bot2> |
Restarting openstack services on cloudvirtlocal1001: ['nova-compute'] - cookbook ran by andrew@bullseye |
[admin] |
| 18:30 |
<wm-bot2> |
Restarting openstack services on cloudvirt1054: ['nova-compute'] - cookbook ran by andrew@bullseye |
[admin] |
| 18:30 |
<wm-bot2> |
Restarting openstack services on cloudvirt1055: ['nova-compute'] - cookbook ran by andrew@bullseye |
[admin] |
| 18:30 |
<wm-bot2> |
Restarting openstack services on cloudvirt1060: ['nova-compute'] - cookbook ran by andrew@bullseye |
[admin] |
| 18:30 |
<wm-bot2> |
Restarting openstack services on cloudvirt1058: ['nova-compute'] - cookbook ran by andrew@bullseye |
[admin] |
| 18:30 |
<wm-bot2> |
Restarting openstack services on cloudvirt1059: ['nova-compute'] - cookbook ran by andrew@bullseye |
[admin] |
| 18:30 |
<wm-bot2> |
Restarting openstack services on cloudvirt1061: ['nova-compute'] - cookbook ran by andrew@bullseye |
[admin] |
| 18:30 |
<wm-bot2> |
Restarting openstack services on cloudvirt1057: ['nova-compute'] - cookbook ran by andrew@bullseye |
[admin] |
| 18:30 |
<wm-bot2> |
Restarting openstack services on cloudvirt1056: ['nova-compute'] - cookbook ran by andrew@bullseye |
[admin] |
| 18:30 |
<wm-bot2> |
Restarting openstack services on cloudvirt1051: ['nova-compute'] - cookbook ran by andrew@bullseye |
[admin] |
| 18:30 |
<wm-bot2> |
Restarting openstack services on cloudvirt1050: ['nova-compute'] - cookbook ran by andrew@bullseye |
[admin] |
| 18:29 |
<wm-bot2> |
Restarting openstack services on cloudvirt1049: ['nova-compute'] - cookbook ran by andrew@bullseye |
[admin] |
| 18:29 |
<wm-bot2> |
Restarting openstack services on cloudvirt1053: ['nova-compute'] - cookbook ran by andrew@bullseye |
[admin] |
| 18:29 |
<wm-bot2> |
Restarting openstack services on cloudvirt1052: ['nova-compute'] - cookbook ran by andrew@bullseye |
[admin] |
| 18:29 |
<wm-bot2> |
Restarting openstack services on cloudvirt1048: ['nova-compute'] - cookbook ran by andrew@bullseye |
[admin] |
| 18:29 |
<wm-bot2> |
Restarting openstack services on cloudcontrol1007: ['nova-conductor', 'nova-scheduler', 'nova-api', 'nova-api-metadata', 'cinder-volume', 'cinder-scheduler'] - cookbook ran by andrew@bullseye |
[admin] |