1151-1200 of 10000 results (98ms)
2024-08-30 §
04:00 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 8:00:00 on db2207.codfw.wmnet with reason: Maintenance [production]
03:51 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db2197.codfw.wmnet with reason: Maintenance [production]
03:51 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 8:00:00 on db2197.codfw.wmnet with reason: Maintenance [production]
03:51 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2189 (T370903)', diff saved to https://phabricator.wikimedia.org/P68284 and previous config saved to /var/cache/conftool/dbconfig/20240830-035123-ladsgroup.json [production]
03:36 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2189', diff saved to https://phabricator.wikimedia.org/P68283 and previous config saved to /var/cache/conftool/dbconfig/20240830-033616-ladsgroup.json [production]
03:21 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2189', diff saved to https://phabricator.wikimedia.org/P68282 and previous config saved to /var/cache/conftool/dbconfig/20240830-032109-ladsgroup.json [production]
03:06 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2189 (T370903)', diff saved to https://phabricator.wikimedia.org/P68281 and previous config saved to /var/cache/conftool/dbconfig/20240830-030602-ladsgroup.json [production]
02:58 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db2189 (T370903)', diff saved to https://phabricator.wikimedia.org/P68280 and previous config saved to /var/cache/conftool/dbconfig/20240830-025809-ladsgroup.json [production]
02:58 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db2189.codfw.wmnet with reason: Maintenance [production]
02:57 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 8:00:00 on db2189.codfw.wmnet with reason: Maintenance [production]
02:57 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2175 (T370903)', diff saved to https://phabricator.wikimedia.org/P68279 and previous config saved to /var/cache/conftool/dbconfig/20240830-025747-ladsgroup.json [production]
02:42 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2175', diff saved to https://phabricator.wikimedia.org/P68278 and previous config saved to /var/cache/conftool/dbconfig/20240830-024239-ladsgroup.json [production]
02:27 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2175', diff saved to https://phabricator.wikimedia.org/P68277 and previous config saved to /var/cache/conftool/dbconfig/20240830-022732-ladsgroup.json [production]
02:12 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2175 (T370903)', diff saved to https://phabricator.wikimedia.org/P68276 and previous config saved to /var/cache/conftool/dbconfig/20240830-021225-ladsgroup.json [production]
02:06 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2213 (T371742)', diff saved to https://phabricator.wikimedia.org/P68275 and previous config saved to /var/cache/conftool/dbconfig/20240830-020606-ladsgroup.json [production]
02:03 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db2175 (T370903)', diff saved to https://phabricator.wikimedia.org/P68274 and previous config saved to /var/cache/conftool/dbconfig/20240830-020305-ladsgroup.json [production]
02:02 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db2175.codfw.wmnet with reason: Maintenance [production]
02:02 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 8:00:00 on db2175.codfw.wmnet with reason: Maintenance [production]
02:02 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2148 (T370903)', diff saved to https://phabricator.wikimedia.org/P68273 and previous config saved to /var/cache/conftool/dbconfig/20240830-020243-ladsgroup.json [production]
01:50 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2213', diff saved to https://phabricator.wikimedia.org/P68272 and previous config saved to /var/cache/conftool/dbconfig/20240830-015059-ladsgroup.json [production]
01:47 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2148', diff saved to https://phabricator.wikimedia.org/P68271 and previous config saved to /var/cache/conftool/dbconfig/20240830-014736-ladsgroup.json [production]
01:35 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2213', diff saved to https://phabricator.wikimedia.org/P68270 and previous config saved to /var/cache/conftool/dbconfig/20240830-013551-ladsgroup.json [production]
01:32 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2148', diff saved to https://phabricator.wikimedia.org/P68269 and previous config saved to /var/cache/conftool/dbconfig/20240830-013229-ladsgroup.json [production]
01:20 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2213 (T371742)', diff saved to https://phabricator.wikimedia.org/P68268 and previous config saved to /var/cache/conftool/dbconfig/20240830-012044-ladsgroup.json [production]
01:17 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2148 (T370903)', diff saved to https://phabricator.wikimedia.org/P68267 and previous config saved to /var/cache/conftool/dbconfig/20240830-011721-ladsgroup.json [production]
01:08 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db2148 (T370903)', diff saved to https://phabricator.wikimedia.org/P68266 and previous config saved to /var/cache/conftool/dbconfig/20240830-010823-ladsgroup.json [production]
01:08 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db2148.codfw.wmnet with reason: Maintenance [production]
01:08 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 8:00:00 on db2148.codfw.wmnet with reason: Maintenance [production]
01:08 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2138 (T370903)', diff saved to https://phabricator.wikimedia.org/P68265 and previous config saved to /var/cache/conftool/dbconfig/20240830-010801-ladsgroup.json [production]
00:55 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db2213 (T371742)', diff saved to https://phabricator.wikimedia.org/P68264 and previous config saved to /var/cache/conftool/dbconfig/20240830-005534-ladsgroup.json [production]
00:55 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2213.codfw.wmnet with reason: Maintenance [production]
00:55 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 12:00:00 on db2213.codfw.wmnet with reason: Maintenance [production]
00:55 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2211 (T371742)', diff saved to https://phabricator.wikimedia.org/P68263 and previous config saved to /var/cache/conftool/dbconfig/20240830-005512-ladsgroup.json [production]
00:52 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2138', diff saved to https://phabricator.wikimedia.org/P68262 and previous config saved to /var/cache/conftool/dbconfig/20240830-005254-ladsgroup.json [production]
00:40 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2211', diff saved to https://phabricator.wikimedia.org/P68261 and previous config saved to /var/cache/conftool/dbconfig/20240830-004004-ladsgroup.json [production]
00:37 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2138', diff saved to https://phabricator.wikimedia.org/P68260 and previous config saved to /var/cache/conftool/dbconfig/20240830-003746-ladsgroup.json [production]
00:24 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2211', diff saved to https://phabricator.wikimedia.org/P68259 and previous config saved to /var/cache/conftool/dbconfig/20240830-002457-ladsgroup.json [production]
00:22 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2138 (T370903)', diff saved to https://phabricator.wikimedia.org/P68258 and previous config saved to /var/cache/conftool/dbconfig/20240830-002239-ladsgroup.json [production]
00:13 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db2138 (T370903)', diff saved to https://phabricator.wikimedia.org/P68255 and previous config saved to /var/cache/conftool/dbconfig/20240830-001353-ladsgroup.json [production]
00:13 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db2138.codfw.wmnet with reason: Maintenance [production]
00:13 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 8:00:00 on db2138.codfw.wmnet with reason: Maintenance [production]
00:13 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2126 (T370903)', diff saved to https://phabricator.wikimedia.org/P68254 and previous config saved to /var/cache/conftool/dbconfig/20240830-001331-ladsgroup.json [production]
00:09 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2211 (T371742)', diff saved to https://phabricator.wikimedia.org/P68253 and previous config saved to /var/cache/conftool/dbconfig/20240830-000950-ladsgroup.json [production]
2024-08-29 §
23:58 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2126', diff saved to https://phabricator.wikimedia.org/P68252 and previous config saved to /var/cache/conftool/dbconfig/20240829-235824-ladsgroup.json [production]
23:44 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Depooling db2211 (T371742)', diff saved to https://phabricator.wikimedia.org/P68251 and previous config saved to /var/cache/conftool/dbconfig/20240829-234420-ladsgroup.json [production]
23:44 <ladsgroup@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2211.codfw.wmnet with reason: Maintenance [production]
23:44 <ladsgroup@cumin1002> START - Cookbook sre.hosts.downtime for 12:00:00 on db2211.codfw.wmnet with reason: Maintenance [production]
23:43 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2126', diff saved to https://phabricator.wikimedia.org/P68250 and previous config saved to /var/cache/conftool/dbconfig/20240829-234317-ladsgroup.json [production]
23:33 <jclark@cumin1002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host ml-lab1001.eqiad.wmnet with OS bookworm [production]
23:28 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2126 (T370903)', diff saved to https://phabricator.wikimedia.org/P68249 and previous config saved to /var/cache/conftool/dbconfig/20240829-232810-ladsgroup.json [production]