3101-3150 of 10000 results (75ms)
2022-08-27 ยง
22:17 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 3 days, 0:00:00 on db2149.codfw.wmnet with reason: Sad disk [production]
22:16 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1130 (T316186)', diff saved to https://phabricator.wikimedia.org/P33406 and previous config saved to /var/cache/conftool/dbconfig/20220827-221631-ladsgroup.json [production]
22:11 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1130 (T316186)', diff saved to https://phabricator.wikimedia.org/P33405 and previous config saved to /var/cache/conftool/dbconfig/20220827-221118-ladsgroup.json [production]
22:11 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1130.eqiad.wmnet with reason: Maintenance [production]
22:10 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1130.eqiad.wmnet with reason: Maintenance [production]
20:58 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2156 (T316186)', diff saved to https://phabricator.wikimedia.org/P33404 and previous config saved to /var/cache/conftool/dbconfig/20220827-205809-ladsgroup.json [production]
20:43 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2156', diff saved to https://phabricator.wikimedia.org/P33403 and previous config saved to /var/cache/conftool/dbconfig/20220827-204303-ladsgroup.json [production]
20:27 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2156', diff saved to https://phabricator.wikimedia.org/P33402 and previous config saved to /var/cache/conftool/dbconfig/20220827-202757-ladsgroup.json [production]
20:12 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2156 (T316186)', diff saved to https://phabricator.wikimedia.org/P33401 and previous config saved to /var/cache/conftool/dbconfig/20220827-201250-ladsgroup.json [production]
20:06 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2156 (T316186)', diff saved to https://phabricator.wikimedia.org/P33400 and previous config saved to /var/cache/conftool/dbconfig/20220827-200639-ladsgroup.json [production]
20:06 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on db2094.codfw.wmnet with reason: Maintenance [production]
20:06 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 2 days, 0:00:00 on db2094.codfw.wmnet with reason: Maintenance [production]
20:06 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2156.codfw.wmnet with reason: Maintenance [production]
20:06 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2156.codfw.wmnet with reason: Maintenance [production]
20:05 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2109 (T316186)', diff saved to https://phabricator.wikimedia.org/P33399 and previous config saved to /var/cache/conftool/dbconfig/20220827-200559-ladsgroup.json [production]
19:50 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2109', diff saved to https://phabricator.wikimedia.org/P33398 and previous config saved to /var/cache/conftool/dbconfig/20220827-195053-ladsgroup.json [production]
19:35 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2109', diff saved to https://phabricator.wikimedia.org/P33397 and previous config saved to /var/cache/conftool/dbconfig/20220827-193546-ladsgroup.json [production]
19:20 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2109 (T316186)', diff saved to https://phabricator.wikimedia.org/P33396 and previous config saved to /var/cache/conftool/dbconfig/20220827-192040-ladsgroup.json [production]
19:15 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2109 (T316186)', diff saved to https://phabricator.wikimedia.org/P33395 and previous config saved to /var/cache/conftool/dbconfig/20220827-191515-ladsgroup.json [production]
19:15 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2109.codfw.wmnet with reason: Maintenance [production]
19:14 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2109.codfw.wmnet with reason: Maintenance [production]
19:14 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2149 (T316186)', diff saved to https://phabricator.wikimedia.org/P33394 and previous config saved to /var/cache/conftool/dbconfig/20220827-191450-ladsgroup.json [production]
18:59 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2149', diff saved to https://phabricator.wikimedia.org/P33393 and previous config saved to /var/cache/conftool/dbconfig/20220827-185944-ladsgroup.json [production]
18:44 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2149', diff saved to https://phabricator.wikimedia.org/P33392 and previous config saved to /var/cache/conftool/dbconfig/20220827-184438-ladsgroup.json [production]
18:29 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2149 (T316186)', diff saved to https://phabricator.wikimedia.org/P33391 and previous config saved to /var/cache/conftool/dbconfig/20220827-182931-ladsgroup.json [production]
18:24 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2149 (T316186)', diff saved to https://phabricator.wikimedia.org/P33390 and previous config saved to /var/cache/conftool/dbconfig/20220827-182408-ladsgroup.json [production]
18:24 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2149.codfw.wmnet with reason: Maintenance [production]
18:23 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2149.codfw.wmnet with reason: Maintenance [production]
18:23 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2177 (T316186)', diff saved to https://phabricator.wikimedia.org/P33389 and previous config saved to /var/cache/conftool/dbconfig/20220827-182343-ladsgroup.json [production]
18:08 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2177', diff saved to https://phabricator.wikimedia.org/P33388 and previous config saved to /var/cache/conftool/dbconfig/20220827-180836-ladsgroup.json [production]
17:53 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2177', diff saved to https://phabricator.wikimedia.org/P33387 and previous config saved to /var/cache/conftool/dbconfig/20220827-175330-ladsgroup.json [production]
17:38 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2177 (T316186)', diff saved to https://phabricator.wikimedia.org/P33386 and previous config saved to /var/cache/conftool/dbconfig/20220827-173824-ladsgroup.json [production]
17:33 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2177 (T316186)', diff saved to https://phabricator.wikimedia.org/P33385 and previous config saved to /var/cache/conftool/dbconfig/20220827-173305-ladsgroup.json [production]
17:32 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2177.codfw.wmnet with reason: Maintenance [production]
17:32 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2177.codfw.wmnet with reason: Maintenance [production]
17:32 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2127 (T316186)', diff saved to https://phabricator.wikimedia.org/P33384 and previous config saved to /var/cache/conftool/dbconfig/20220827-173240-ladsgroup.json [production]
17:17 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2127', diff saved to https://phabricator.wikimedia.org/P33383 and previous config saved to /var/cache/conftool/dbconfig/20220827-171734-ladsgroup.json [production]
17:02 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2127', diff saved to https://phabricator.wikimedia.org/P33382 and previous config saved to /var/cache/conftool/dbconfig/20220827-170227-ladsgroup.json [production]
16:47 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2127 (T316186)', diff saved to https://phabricator.wikimedia.org/P33381 and previous config saved to /var/cache/conftool/dbconfig/20220827-164721-ladsgroup.json [production]
16:41 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2127 (T316186)', diff saved to https://phabricator.wikimedia.org/P33380 and previous config saved to /var/cache/conftool/dbconfig/20220827-164156-ladsgroup.json [production]
16:41 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2127.codfw.wmnet with reason: Maintenance [production]
16:41 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2127.codfw.wmnet with reason: Maintenance [production]
16:35 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2139.codfw.wmnet with reason: Maintenance [production]
16:35 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2139.codfw.wmnet with reason: Maintenance [production]
16:35 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1112 (T316186)', diff saved to https://phabricator.wikimedia.org/P33379 and previous config saved to /var/cache/conftool/dbconfig/20220827-163528-ladsgroup.json [production]
16:20 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1112', diff saved to https://phabricator.wikimedia.org/P33378 and previous config saved to /var/cache/conftool/dbconfig/20220827-162022-ladsgroup.json [production]
16:05 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1112', diff saved to https://phabricator.wikimedia.org/P33377 and previous config saved to /var/cache/conftool/dbconfig/20220827-160516-ladsgroup.json [production]
15:50 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1112 (T316186)', diff saved to https://phabricator.wikimedia.org/P33376 and previous config saved to /var/cache/conftool/dbconfig/20220827-155010-ladsgroup.json [production]
15:44 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1112 (T316186)', diff saved to https://phabricator.wikimedia.org/P33375 and previous config saved to /var/cache/conftool/dbconfig/20220827-154452-ladsgroup.json [production]
15:44 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on clouddb[1013,1017,1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance [production]