|
2022-11-02
§
|
| 02:24 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1192 (T318605)', diff saved to https://phabricator.wikimedia.org/P37545 and previous config saved to /var/cache/conftool/dbconfig/20221102-022408-ladsgroup.json |
[production] |
| 01:50 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1192 (T318605)', diff saved to https://phabricator.wikimedia.org/P37544 and previous config saved to /var/cache/conftool/dbconfig/20221102-015019-ladsgroup.json |
[production] |
| 01:50 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1192.eqiad.wmnet with reason: Maintenance |
[production] |
| 01:49 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1192.eqiad.wmnet with reason: Maintenance |
[production] |
| 01:49 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1178 (T318605)', diff saved to https://phabricator.wikimedia.org/P37543 and previous config saved to /var/cache/conftool/dbconfig/20221102-014955-ladsgroup.json |
[production] |
| 01:34 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P37542 and previous config saved to /var/cache/conftool/dbconfig/20221102-013447-ladsgroup.json |
[production] |
| 01:19 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P37541 and previous config saved to /var/cache/conftool/dbconfig/20221102-011937-ladsgroup.json |
[production] |
| 01:09 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2181 (T318605)', diff saved to https://phabricator.wikimedia.org/P37540 and previous config saved to /var/cache/conftool/dbconfig/20221102-010958-ladsgroup.json |
[production] |
| 01:04 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1178 (T318605)', diff saved to https://phabricator.wikimedia.org/P37539 and previous config saved to /var/cache/conftool/dbconfig/20221102-010430-ladsgroup.json |
[production] |
| 00:54 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2181', diff saved to https://phabricator.wikimedia.org/P37538 and previous config saved to /var/cache/conftool/dbconfig/20221102-005451-ladsgroup.json |
[production] |
| 00:39 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2181', diff saved to https://phabricator.wikimedia.org/P37537 and previous config saved to /var/cache/conftool/dbconfig/20221102-003941-ladsgroup.json |
[production] |
| 00:29 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1178 (T318605)', diff saved to https://phabricator.wikimedia.org/P37536 and previous config saved to /var/cache/conftool/dbconfig/20221102-002936-ladsgroup.json |
[production] |
| 00:29 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance |
[production] |
| 00:29 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance |
[production] |
| 00:29 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1177 (T318605)', diff saved to https://phabricator.wikimedia.org/P37535 and previous config saved to /var/cache/conftool/dbconfig/20221102-002913-ladsgroup.json |
[production] |
| 00:24 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2181 (T318605)', diff saved to https://phabricator.wikimedia.org/P37534 and previous config saved to /var/cache/conftool/dbconfig/20221102-002433-ladsgroup.json |
[production] |
| 00:14 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P37533 and previous config saved to /var/cache/conftool/dbconfig/20221102-001401-ladsgroup.json |
[production] |
|
2022-11-01
§
|
| 23:58 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P37532 and previous config saved to /var/cache/conftool/dbconfig/20221101-235853-ladsgroup.json |
[production] |
| 23:49 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db2181 (T318605)', diff saved to https://phabricator.wikimedia.org/P37531 and previous config saved to /var/cache/conftool/dbconfig/20221101-234957-ladsgroup.json |
[production] |
| 23:49 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2181.codfw.wmnet with reason: Maintenance |
[production] |
| 23:49 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2181.codfw.wmnet with reason: Maintenance |
[production] |
| 23:49 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2168:3318 (T318605)', diff saved to https://phabricator.wikimedia.org/P37530 and previous config saved to /var/cache/conftool/dbconfig/20221101-234935-ladsgroup.json |
[production] |
| 23:43 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1177 (T318605)', diff saved to https://phabricator.wikimedia.org/P37529 and previous config saved to /var/cache/conftool/dbconfig/20221101-234346-ladsgroup.json |
[production] |
| 23:34 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2168:3318', diff saved to https://phabricator.wikimedia.org/P37528 and previous config saved to /var/cache/conftool/dbconfig/20221101-233427-ladsgroup.json |
[production] |
| 23:19 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2168:3318', diff saved to https://phabricator.wikimedia.org/P37527 and previous config saved to /var/cache/conftool/dbconfig/20221101-231919-ladsgroup.json |
[production] |
| 23:08 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1177 (T318605)', diff saved to https://phabricator.wikimedia.org/P37526 and previous config saved to /var/cache/conftool/dbconfig/20221101-230833-ladsgroup.json |
[production] |
| 23:08 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance |
[production] |
| 23:08 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1177.eqiad.wmnet with reason: Maintenance |
[production] |
| 23:08 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1172 (T318605)', diff saved to https://phabricator.wikimedia.org/P37525 and previous config saved to /var/cache/conftool/dbconfig/20221101-230811-ladsgroup.json |
[production] |
| 23:04 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2168:3318 (T318605)', diff saved to https://phabricator.wikimedia.org/P37524 and previous config saved to /var/cache/conftool/dbconfig/20221101-230411-ladsgroup.json |
[production] |
| 22:55 |
<Emperor> |
depool ms-fe2009 |
[production] |
| 22:53 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P37523 and previous config saved to /var/cache/conftool/dbconfig/20221101-225303-ladsgroup.json |
[production] |
| 22:43 |
<krinkle@deploy1002> |
Finished deploy [integration/docroot@2ddd7d9]: (no justification provided) (duration: 00m 33s) |
[production] |
| 22:43 |
<krinkle@deploy1002> |
Started deploy [integration/docroot@2ddd7d9]: (no justification provided) |
[production] |
| 22:37 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P37522 and previous config saved to /var/cache/conftool/dbconfig/20221101-223754-ladsgroup.json |
[production] |
| 22:32 |
<jhathaway@cumin1001> |
END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host aux-k8s-worker1002.eqiad.wmnet |
[production] |
| 22:32 |
<Emperor> |
rolling restart of eqiad swift front-ends |
[production] |
| 22:29 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db2168:3318 (T318605)', diff saved to https://phabricator.wikimedia.org/P37521 and previous config saved to /var/cache/conftool/dbconfig/20221101-222858-ladsgroup.json |
[production] |
| 22:29 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2168.codfw.wmnet with reason: Maintenance |
[production] |
| 22:29 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2168.codfw.wmnet with reason: Maintenance |
[production] |
| 22:28 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2167:3318 (T318605)', diff saved to https://phabricator.wikimedia.org/P37520 and previous config saved to /var/cache/conftool/dbconfig/20221101-222835-ladsgroup.json |
[production] |
| 22:22 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1172 (T318605)', diff saved to https://phabricator.wikimedia.org/P37519 and previous config saved to /var/cache/conftool/dbconfig/20221101-222247-ladsgroup.json |
[production] |
| 22:20 |
<jhathaway@cumin1001> |
conftool action : set/pooled=false; selector: dnsdisc=swift,name=eqiad |
[production] |
| 22:13 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2167:3318', diff saved to https://phabricator.wikimedia.org/P37518 and previous config saved to /var/cache/conftool/dbconfig/20221101-221328-ladsgroup.json |
[production] |
| 22:09 |
<jhathaway@cumin1001> |
END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) aux-k8s-worker1002.eqiad.wmnet on all recursors |
[production] |
| 22:09 |
<jhathaway@cumin1001> |
START - Cookbook sre.dns.wipe-cache aux-k8s-worker1002.eqiad.wmnet on all recursors |
[production] |
| 22:08 |
<jhathaway@cumin1001> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 21:58 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2167:3318', diff saved to https://phabricator.wikimedia.org/P37517 and previous config saved to /var/cache/conftool/dbconfig/20221101-215820-ladsgroup.json |
[production] |
| 21:53 |
<jhathaway@cumin1001> |
START - Cookbook sre.dns.netbox |
[production] |
| 21:53 |
<jhathaway@cumin1001> |
START - Cookbook sre.ganeti.makevm for new host aux-k8s-worker1002.eqiad.wmnet |
[production] |