3801-3850 of 10000 results (98ms)
2022-11-25 ยง
11:24 <elukey> restart turnilo on an-tool1007 to pick up new settings for webrequest_sampled_live [production]
11:21 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1189', diff saved to https://phabricator.wikimedia.org/P41100 and previous config saved to /var/cache/conftool/dbconfig/20221125-112148-marostegui.json [production]
11:06 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1189 (T321126)', diff saved to https://phabricator.wikimedia.org/P41099 and previous config saved to /var/cache/conftool/dbconfig/20221125-110642-marostegui.json [production]
10:50 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1189 (T321126)', diff saved to https://phabricator.wikimedia.org/P41098 and previous config saved to /var/cache/conftool/dbconfig/20221125-105036-marostegui.json [production]
10:50 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1189.eqiad.wmnet with reason: Maintenance [production]
10:50 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 5:00:00 on db1189.eqiad.wmnet with reason: Maintenance [production]
10:50 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1179 (T321126)', diff saved to https://phabricator.wikimedia.org/P41097 and previous config saved to /var/cache/conftool/dbconfig/20221125-105015-marostegui.json [production]
10:35 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1179', diff saved to https://phabricator.wikimedia.org/P41096 and previous config saved to /var/cache/conftool/dbconfig/20221125-103509-marostegui.json [production]
10:20 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1179', diff saved to https://phabricator.wikimedia.org/P41095 and previous config saved to /var/cache/conftool/dbconfig/20221125-102002-marostegui.json [production]
10:04 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1179 (T321126)', diff saved to https://phabricator.wikimedia.org/P41094 and previous config saved to /var/cache/conftool/dbconfig/20221125-100456-marostegui.json [production]
09:46 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1179 (T321126)', diff saved to https://phabricator.wikimedia.org/P41093 and previous config saved to /var/cache/conftool/dbconfig/20221125-094643-marostegui.json [production]
09:46 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1179.eqiad.wmnet with reason: Maintenance [production]
09:46 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 5:00:00 on db1179.eqiad.wmnet with reason: Maintenance [production]
09:46 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1175 (T321126)', diff saved to https://phabricator.wikimedia.org/P41092 and previous config saved to /var/cache/conftool/dbconfig/20221125-094622-marostegui.json [production]
09:31 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1175', diff saved to https://phabricator.wikimedia.org/P41091 and previous config saved to /var/cache/conftool/dbconfig/20221125-093115-marostegui.json [production]
09:16 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1175', diff saved to https://phabricator.wikimedia.org/P41090 and previous config saved to /var/cache/conftool/dbconfig/20221125-091609-marostegui.json [production]
09:01 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1175 (T321126)', diff saved to https://phabricator.wikimedia.org/P41089 and previous config saved to /var/cache/conftool/dbconfig/20221125-090102-marostegui.json [production]
08:51 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1175 (T321126)', diff saved to https://phabricator.wikimedia.org/P41088 and previous config saved to /var/cache/conftool/dbconfig/20221125-085101-marostegui.json [production]
08:50 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1175.eqiad.wmnet with reason: Maintenance [production]
08:50 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 5:00:00 on db1175.eqiad.wmnet with reason: Maintenance [production]
08:50 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166 (T321126)', diff saved to https://phabricator.wikimedia.org/P41087 and previous config saved to /var/cache/conftool/dbconfig/20221125-085040-marostegui.json [production]
08:35 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166', diff saved to https://phabricator.wikimedia.org/P41086 and previous config saved to /var/cache/conftool/dbconfig/20221125-083534-marostegui.json [production]
08:35 <moritzm> installing libarchive security updates [production]
08:20 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166', diff saved to https://phabricator.wikimedia.org/P41085 and previous config saved to /var/cache/conftool/dbconfig/20221125-082027-marostegui.json [production]
08:09 <moritzm> rebalance Ganeti group C/codfw following reboots [production]
08:05 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166 (T321126)', diff saved to https://phabricator.wikimedia.org/P41084 and previous config saved to /var/cache/conftool/dbconfig/20221125-080521-marostegui.json [production]
07:55 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1166 (T321126)', diff saved to https://phabricator.wikimedia.org/P41083 and previous config saved to /var/cache/conftool/dbconfig/20221125-075521-marostegui.json [production]
07:55 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1166.eqiad.wmnet with reason: Maintenance [production]
07:55 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 5:00:00 on db1166.eqiad.wmnet with reason: Maintenance [production]
07:55 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1157 (T321126)', diff saved to https://phabricator.wikimedia.org/P41082 and previous config saved to /var/cache/conftool/dbconfig/20221125-075500-marostegui.json [production]
07:39 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1157', diff saved to https://phabricator.wikimedia.org/P41081 and previous config saved to /var/cache/conftool/dbconfig/20221125-073953-marostegui.json [production]
07:24 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1157', diff saved to https://phabricator.wikimedia.org/P41080 and previous config saved to /var/cache/conftool/dbconfig/20221125-072447-marostegui.json [production]
07:09 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1157 (T321126)', diff saved to https://phabricator.wikimedia.org/P41079 and previous config saved to /var/cache/conftool/dbconfig/20221125-070940-marostegui.json [production]
06:59 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1157 (T321126)', diff saved to https://phabricator.wikimedia.org/P41078 and previous config saved to /var/cache/conftool/dbconfig/20221125-065930-marostegui.json [production]
06:59 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1157.eqiad.wmnet with reason: Maintenance [production]
06:59 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 5:00:00 on db1157.eqiad.wmnet with reason: Maintenance [production]
06:51 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1145.eqiad.wmnet with reason: Maintenance [production]
06:50 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 5:00:00 on db1145.eqiad.wmnet with reason: Maintenance [production]
06:50 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1112 (T321126)', diff saved to https://phabricator.wikimedia.org/P41077 and previous config saved to /var/cache/conftool/dbconfig/20221125-065049-marostegui.json [production]
06:35 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1112', diff saved to https://phabricator.wikimedia.org/P41076 and previous config saved to /var/cache/conftool/dbconfig/20221125-063543-marostegui.json [production]
06:20 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1112', diff saved to https://phabricator.wikimedia.org/P41075 and previous config saved to /var/cache/conftool/dbconfig/20221125-062036-marostegui.json [production]
06:05 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1112 (T321126)', diff saved to https://phabricator.wikimedia.org/P41074 and previous config saved to /var/cache/conftool/dbconfig/20221125-060530-marostegui.json [production]
05:55 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1112 (T321126)', diff saved to https://phabricator.wikimedia.org/P41073 and previous config saved to /var/cache/conftool/dbconfig/20221125-055517-marostegui.json [production]
05:55 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on clouddb[1013,1017,1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance [production]
05:54 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 10:00:00 on clouddb[1013,1017,1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance [production]
05:54 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1112.eqiad.wmnet with reason: Maintenance [production]
05:54 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 5:00:00 on db1112.eqiad.wmnet with reason: Maintenance [production]
05:46 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1102.eqiad.wmnet with reason: Maintenance [production]
05:45 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 5:00:00 on db1102.eqiad.wmnet with reason: Maintenance [production]
05:44 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1109.eqiad.wmnet with reason: Maintenance [production]