1251-1300 of 10000 results (84ms)
2022-11-22 §
04:03 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1161.eqiad.wmnet with reason: Maintenance [production]
02:52 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2171:3315 (T323214)', diff saved to https://phabricator.wikimedia.org/P40417 and previous config saved to /var/cache/conftool/dbconfig/20221122-025209-ladsgroup.json [production]
02:52 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2171.codfw.wmnet with reason: Maintenance [production]
02:51 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2171.codfw.wmnet with reason: Maintenance [production]
02:51 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2157 (T323214)', diff saved to https://phabricator.wikimedia.org/P40416 and previous config saved to /var/cache/conftool/dbconfig/20221122-025148-ladsgroup.json [production]
02:36 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2157', diff saved to https://phabricator.wikimedia.org/P40415 and previous config saved to /var/cache/conftool/dbconfig/20221122-023641-ladsgroup.json [production]
02:21 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2157', diff saved to https://phabricator.wikimedia.org/P40414 and previous config saved to /var/cache/conftool/dbconfig/20221122-022134-ladsgroup.json [production]
02:06 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2157 (T323214)', diff saved to https://phabricator.wikimedia.org/P40413 and previous config saved to /var/cache/conftool/dbconfig/20221122-020628-ladsgroup.json [production]
01:59 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1150.eqiad.wmnet with reason: Maintenance [production]
01:59 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1150.eqiad.wmnet with reason: Maintenance [production]
01:59 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1144:3315 (T323214)', diff saved to https://phabricator.wikimedia.org/P40412 and previous config saved to /var/cache/conftool/dbconfig/20221122-015923-ladsgroup.json [production]
01:56 <dzahn@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 14 days, 0:00:00 on phab1004.eqiad.wmnet with reason: T322250 [production]
01:56 <dzahn@cumin2002> START - Cookbook sre.hosts.downtime for 14 days, 0:00:00 on phab1004.eqiad.wmnet with reason: T322250 [production]
01:55 <dzahn@cumin2002> END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for phab1001.eqiad.wmnet [production]
01:55 <dzahn@cumin2002> START - Cookbook sre.hosts.remove-downtime for phab1001.eqiad.wmnet [production]
01:44 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1144:3315', diff saved to and previous config saved to /var/cache/conftool/dbconfig/20221122-014417-ladsgroup.json [production]
01:29 <brennen@deploy1002> Finished deploy [phabricator/deployment@f68dc24]: deploy config changes for phab1004 -> phab1001 revert (duration: 00m 56s) [production]
01:29 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1144:3315', diff saved to and previous config saved to /var/cache/conftool/dbconfig/20221122-012910-ladsgroup.json [production]
01:28 <brennen@deploy1002> Started deploy [phabricator/deployment@f68dc24]: deploy config changes for phab1004 -> phab1001 revert [production]
01:26 <dzahn@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on phab1004.eqiad.wmnet with reason: T322250 [production]
01:26 <dzahn@cumin2002> START - Cookbook sre.hosts.downtime for 2 days, 0:00:00 on phab1004.eqiad.wmnet with reason: T322250 [production]
01:25 <brennen> reverting to phab1001; short phabricator downtime incoming while DNS changes are made (T280597) [production]
01:14 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1144:3315 (T323214)', diff saved to https://phabricator.wikimedia.org/P40411 and previous config saved to /var/cache/conftool/dbconfig/20221122-011404-ladsgroup.json [production]
01:13 <dzahn@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on phab1004.eqiad.wmnet with reason: T322250 [production]
01:13 <dzahn@cumin2002> START - Cookbook sre.hosts.downtime for 2 days, 0:00:00 on phab1004.eqiad.wmnet with reason: T322250 [production]
00:31 <dzahn@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 14 days, 0:00:00 on phab1001.eqiad.wmnet with reason: T322250 [production]
00:31 <dzahn@cumin2002> START - Cookbook sre.hosts.downtime for 14 days, 0:00:00 on phab1001.eqiad.wmnet with reason: T322250 [production]
00:24 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance [production]
00:24 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance [production]
00:24 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1197 (T322618)', diff saved to https://phabricator.wikimedia.org/P40410 and previous config saved to /var/cache/conftool/dbconfig/20221122-002411-ladsgroup.json [production]
00:23 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance [production]
00:22 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance [production]
00:22 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1198 (T322618)', diff saved to https://phabricator.wikimedia.org/P40409 and previous config saved to /var/cache/conftool/dbconfig/20221122-002245-ladsgroup.json [production]
00:09 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1197', diff saved to https://phabricator.wikimedia.org/P40408 and previous config saved to /var/cache/conftool/dbconfig/20221122-000904-ladsgroup.json [production]
00:07 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1198', diff saved to https://phabricator.wikimedia.org/P40407 and previous config saved to /var/cache/conftool/dbconfig/20221122-000739-ladsgroup.json [production]
00:07 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2157 (T323214)', diff saved to https://phabricator.wikimedia.org/P40406 and previous config saved to /var/cache/conftool/dbconfig/20221122-000700-ladsgroup.json [production]
00:06 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2157.codfw.wmnet with reason: Maintenance [production]
00:06 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2157.codfw.wmnet with reason: Maintenance [production]
00:06 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2137:3315 (T323214)', diff saved to https://phabricator.wikimedia.org/P40405 and previous config saved to /var/cache/conftool/dbconfig/20221122-000638-ladsgroup.json [production]
2022-11-21 §
23:53 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1197', diff saved to https://phabricator.wikimedia.org/P40404 and previous config saved to /var/cache/conftool/dbconfig/20221121-235357-ladsgroup.json [production]
23:52 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1198', diff saved to https://phabricator.wikimedia.org/P40403 and previous config saved to /var/cache/conftool/dbconfig/20221121-235232-ladsgroup.json [production]
23:51 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2137:3315', diff saved to https://phabricator.wikimedia.org/P40402 and previous config saved to /var/cache/conftool/dbconfig/20221121-235132-ladsgroup.json [production]
23:38 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1197 (T322618)', diff saved to https://phabricator.wikimedia.org/P40401 and previous config saved to /var/cache/conftool/dbconfig/20221121-233851-ladsgroup.json [production]
23:37 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1198 (T322618)', diff saved to https://phabricator.wikimedia.org/P40400 and previous config saved to /var/cache/conftool/dbconfig/20221121-233726-ladsgroup.json [production]
23:36 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1197 (T322618)', diff saved to https://phabricator.wikimedia.org/P40399 and previous config saved to /var/cache/conftool/dbconfig/20221121-233640-ladsgroup.json [production]
23:36 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1197.eqiad.wmnet with reason: Maintenance [production]
23:36 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2137:3315', diff saved to https://phabricator.wikimedia.org/P40398 and previous config saved to /var/cache/conftool/dbconfig/20221121-233625-ladsgroup.json [production]
23:36 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1197.eqiad.wmnet with reason: Maintenance [production]
23:36 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1188 (T322618)', diff saved to https://phabricator.wikimedia.org/P40397 and previous config saved to /var/cache/conftool/dbconfig/20221121-233619-ladsgroup.json [production]
23:33 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1198 (T322618)', diff saved to https://phabricator.wikimedia.org/P40396 and previous config saved to /var/cache/conftool/dbconfig/20221121-233331-ladsgroup.json [production]