8851-8900 of 10000 results (55ms)
2022-04-11 §
04:42 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 8:00:00 on db1127.eqiad.wmnet with reason: Maintenance [production]
04:40 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1164', diff saved to https://phabricator.wikimedia.org/P24365 and previous config saved to /var/cache/conftool/dbconfig/20220411-044058-root.json [production]
04:06 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1134 (T298565)', diff saved to https://phabricator.wikimedia.org/P24364 and previous config saved to /var/cache/conftool/dbconfig/20220411-040656-ladsgroup.json [production]
04:06 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1134.eqiad.wmnet with reason: Maintenance [production]
04:06 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1134.eqiad.wmnet with reason: Maintenance [production]
04:06 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1184 (T298565)', diff saved to https://phabricator.wikimedia.org/P24363 and previous config saved to /var/cache/conftool/dbconfig/20220411-040648-ladsgroup.json [production]
03:51 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1184', diff saved to https://phabricator.wikimedia.org/P24362 and previous config saved to /var/cache/conftool/dbconfig/20220411-035143-ladsgroup.json [production]
03:36 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1184', diff saved to https://phabricator.wikimedia.org/P24361 and previous config saved to /var/cache/conftool/dbconfig/20220411-033638-ladsgroup.json [production]
03:21 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1184 (T298565)', diff saved to https://phabricator.wikimedia.org/P24360 and previous config saved to /var/cache/conftool/dbconfig/20220411-032132-ladsgroup.json [production]
02:28 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1184 (T298565)', diff saved to https://phabricator.wikimedia.org/P24359 and previous config saved to /var/cache/conftool/dbconfig/20220411-022840-ladsgroup.json [production]
02:28 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1184.eqiad.wmnet with reason: Maintenance [production]
02:28 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1184.eqiad.wmnet with reason: Maintenance [production]
02:28 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1164 (T298565)', diff saved to https://phabricator.wikimedia.org/P24358 and previous config saved to /var/cache/conftool/dbconfig/20220411-022832-ladsgroup.json [production]
02:13 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1164', diff saved to https://phabricator.wikimedia.org/P24357 and previous config saved to /var/cache/conftool/dbconfig/20220411-021327-ladsgroup.json [production]
01:58 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1164', diff saved to https://phabricator.wikimedia.org/P24356 and previous config saved to /var/cache/conftool/dbconfig/20220411-015822-ladsgroup.json [production]
01:43 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1164 (T298565)', diff saved to https://phabricator.wikimedia.org/P24355 and previous config saved to /var/cache/conftool/dbconfig/20220411-014316-ladsgroup.json [production]
00:48 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1164 (T298565)', diff saved to https://phabricator.wikimedia.org/P24354 and previous config saved to /var/cache/conftool/dbconfig/20220411-004826-ladsgroup.json [production]
00:48 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1164.eqiad.wmnet with reason: Maintenance [production]
00:48 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1164.eqiad.wmnet with reason: Maintenance [production]
00:48 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1169 (T298565)', diff saved to https://phabricator.wikimedia.org/P24353 and previous config saved to /var/cache/conftool/dbconfig/20220411-004817-ladsgroup.json [production]
00:33 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1169', diff saved to https://phabricator.wikimedia.org/P24352 and previous config saved to /var/cache/conftool/dbconfig/20220411-003312-ladsgroup.json [production]
00:18 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1169', diff saved to https://phabricator.wikimedia.org/P24351 and previous config saved to /var/cache/conftool/dbconfig/20220411-001807-ladsgroup.json [production]
00:03 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1169 (T298565)', diff saved to https://phabricator.wikimedia.org/P24350 and previous config saved to /var/cache/conftool/dbconfig/20220411-000302-ladsgroup.json [production]
2022-04-10 §
23:11 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1169 (T298565)', diff saved to https://phabricator.wikimedia.org/P24349 and previous config saved to /var/cache/conftool/dbconfig/20220410-231112-ladsgroup.json [production]
23:11 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1169.eqiad.wmnet with reason: Maintenance [production]
23:11 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1169.eqiad.wmnet with reason: Maintenance [production]
23:11 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1106 (T298565)', diff saved to https://phabricator.wikimedia.org/P24348 and previous config saved to /var/cache/conftool/dbconfig/20220410-231104-ladsgroup.json [production]
22:55 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1106', diff saved to https://phabricator.wikimedia.org/P24347 and previous config saved to /var/cache/conftool/dbconfig/20220410-225559-ladsgroup.json [production]
22:40 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1106', diff saved to https://phabricator.wikimedia.org/P24346 and previous config saved to /var/cache/conftool/dbconfig/20220410-224053-ladsgroup.json [production]
22:25 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1106 (T298565)', diff saved to https://phabricator.wikimedia.org/P24345 and previous config saved to /var/cache/conftool/dbconfig/20220410-222548-ladsgroup.json [production]
21:20 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1106 (T298565)', diff saved to https://phabricator.wikimedia.org/P24344 and previous config saved to /var/cache/conftool/dbconfig/20220410-212042-ladsgroup.json [production]
21:20 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on clouddb[1013,1017,1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance [production]
21:20 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 12:00:00 on clouddb[1013,1017,1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance [production]
21:20 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1106.eqiad.wmnet with reason: Maintenance [production]
21:20 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1106.eqiad.wmnet with reason: Maintenance [production]
21:20 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1106 (T298565)', diff saved to https://phabricator.wikimedia.org/P24343 and previous config saved to /var/cache/conftool/dbconfig/20220410-212024-ladsgroup.json [production]
21:05 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1106', diff saved to https://phabricator.wikimedia.org/P24342 and previous config saved to /var/cache/conftool/dbconfig/20220410-210519-ladsgroup.json [production]
20:50 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1106', diff saved to https://phabricator.wikimedia.org/P24341 and previous config saved to /var/cache/conftool/dbconfig/20220410-205014-ladsgroup.json [production]
20:35 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1106 (T298565)', diff saved to https://phabricator.wikimedia.org/P24340 and previous config saved to /var/cache/conftool/dbconfig/20220410-203508-ladsgroup.json [production]
19:39 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1106 (T298565)', diff saved to https://phabricator.wikimedia.org/P24339 and previous config saved to /var/cache/conftool/dbconfig/20220410-193900-ladsgroup.json [production]
19:38 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on clouddb[1013,1017,1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance [production]
19:38 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 12:00:00 on clouddb[1013,1017,1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance [production]
19:38 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1106.eqiad.wmnet with reason: Maintenance [production]
19:38 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1106.eqiad.wmnet with reason: Maintenance [production]
2022-04-09 §
12:39 <godog> bounce prometheus@ops on prometheus5001 [production]
12:27 <ariel@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host dumpsdata1002.eqiad.wmnet [production]
12:22 <ariel@cumin1001> START - Cookbook sre.hosts.reboot-single for host dumpsdata1002.eqiad.wmnet [production]
12:22 <ariel@cumin1001> END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=99) for host dumpsdata1002.eqiad.wmnet [production]
12:22 <ariel@cumin1001> START - Cookbook sre.hosts.reboot-single for host dumpsdata1002.eqiad.wmnet [production]
12:20 <ariel@cumin1001> END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=99) for host dumpsdata1002.eqiad.wmnet [production]