|
2024-04-04
ยง
|
| 20:33 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1214 (T355609)', diff saved to https://phabricator.wikimedia.org/P59578 and previous config saved to /var/cache/conftool/dbconfig/20240404-203302-marostegui.json |
[production] |
| 20:26 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2108', diff saved to https://phabricator.wikimedia.org/P59577 and previous config saved to /var/cache/conftool/dbconfig/20240404-202634-arnaudb.json |
[production] |
| 20:17 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1214', diff saved to https://phabricator.wikimedia.org/P59576 and previous config saved to /var/cache/conftool/dbconfig/20240404-201755-marostegui.json |
[production] |
| 20:11 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2108', diff saved to https://phabricator.wikimedia.org/P59575 and previous config saved to /var/cache/conftool/dbconfig/20240404-201126-arnaudb.json |
[production] |
| 20:02 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1214', diff saved to https://phabricator.wikimedia.org/P59574 and previous config saved to /var/cache/conftool/dbconfig/20240404-200247-marostegui.json |
[production] |
| 19:56 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2108 (T360332)', diff saved to https://phabricator.wikimedia.org/P59573 and previous config saved to /var/cache/conftool/dbconfig/20240404-195615-arnaudb.json |
[production] |
| 19:53 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Depooling db2108 (T360332)', diff saved to https://phabricator.wikimedia.org/P59572 and previous config saved to /var/cache/conftool/dbconfig/20240404-195333-arnaudb.json |
[production] |
| 19:53 |
<arnaudb@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2108.codfw.wmnet with reason: Maintenance |
[production] |
| 19:53 |
<arnaudb@cumin1002> |
START - Cookbook sre.hosts.downtime for 12:00:00 on db2108.codfw.wmnet with reason: Maintenance |
[production] |
| 19:52 |
<arnaudb@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2098.codfw.wmnet with reason: Maintenance |
[production] |
| 19:52 |
<arnaudb@cumin1002> |
START - Cookbook sre.hosts.downtime for 12:00:00 on db2098.codfw.wmnet with reason: Maintenance |
[production] |
| 19:51 |
<arnaudb@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on dbstore1008.eqiad.wmnet with reason: Maintenance |
[production] |
| 19:51 |
<arnaudb@cumin1002> |
START - Cookbook sre.hosts.downtime for 12:00:00 on dbstore1008.eqiad.wmnet with reason: Maintenance |
[production] |
| 19:51 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1236 (T360332)', diff saved to https://phabricator.wikimedia.org/P59571 and previous config saved to /var/cache/conftool/dbconfig/20240404-195138-arnaudb.json |
[production] |
| 19:47 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1214 (T355609)', diff saved to https://phabricator.wikimedia.org/P59570 and previous config saved to /var/cache/conftool/dbconfig/20240404-194739-marostegui.json |
[production] |
| 19:36 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1236', diff saved to https://phabricator.wikimedia.org/P59569 and previous config saved to /var/cache/conftool/dbconfig/20240404-193631-arnaudb.json |
[production] |
| 19:21 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1236', diff saved to https://phabricator.wikimedia.org/P59568 and previous config saved to /var/cache/conftool/dbconfig/20240404-192123-arnaudb.json |
[production] |
| 19:11 |
<sukhe@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=cp4052.ulsfo.wmnet,service=(cdn|ats-be) |
[production] |
| 19:10 |
<sukhe@cumin1002> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp4052.ulsfo.wmnet with OS bullseye |
[production] |
| 19:06 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1236 (T360332)', diff saved to https://phabricator.wikimedia.org/P59567 and previous config saved to /var/cache/conftool/dbconfig/20240404-190616-arnaudb.json |
[production] |
| 19:01 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Depooling db1236 (T360332)', diff saved to https://phabricator.wikimedia.org/P59566 and previous config saved to /var/cache/conftool/dbconfig/20240404-190146-arnaudb.json |
[production] |
| 19:01 |
<arnaudb@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1236.eqiad.wmnet with reason: Maintenance |
[production] |
| 19:01 |
<arnaudb@cumin1002> |
START - Cookbook sre.hosts.downtime for 12:00:00 on db1236.eqiad.wmnet with reason: Maintenance |
[production] |
| 19:01 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1227 (T360332)', diff saved to https://phabricator.wikimedia.org/P59565 and previous config saved to /var/cache/conftool/dbconfig/20240404-190123-arnaudb.json |
[production] |
| 18:54 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depooling db1214 (T355609)', diff saved to https://phabricator.wikimedia.org/P59564 and previous config saved to /var/cache/conftool/dbconfig/20240404-185458-marostegui.json |
[production] |
| 18:54 |
<marostegui@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1214.eqiad.wmnet with reason: Maintenance |
[production] |
| 18:54 |
<marostegui@cumin1002> |
START - Cookbook sre.hosts.downtime for 8:00:00 on db1214.eqiad.wmnet with reason: Maintenance |
[production] |
| 18:54 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1211 (T355609)', diff saved to https://phabricator.wikimedia.org/P59563 and previous config saved to /var/cache/conftool/dbconfig/20240404-185436-marostegui.json |
[production] |
| 18:53 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Depooling db1218 (T356166)', diff saved to https://phabricator.wikimedia.org/P59562 and previous config saved to /var/cache/conftool/dbconfig/20240404-185319-marostegui.json |
[production] |
| 18:53 |
<marostegui@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1218.eqiad.wmnet with reason: Maintenance |
[production] |
| 18:53 |
<marostegui@cumin1002> |
START - Cookbook sre.hosts.downtime for 8:00:00 on db1218.eqiad.wmnet with reason: Maintenance |
[production] |
| 18:52 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1207 (T356166)', diff saved to https://phabricator.wikimedia.org/P59561 and previous config saved to /var/cache/conftool/dbconfig/20240404-185256-marostegui.json |
[production] |
| 18:49 |
<sukhe@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cp4052.ulsfo.wmnet with reason: host reimage |
[production] |
| 18:46 |
<sukhe@cumin1002> |
START - Cookbook sre.hosts.downtime for 2:00:00 on cp4052.ulsfo.wmnet with reason: host reimage |
[production] |
| 18:46 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1227', diff saved to https://phabricator.wikimedia.org/P59560 and previous config saved to /var/cache/conftool/dbconfig/20240404-184616-arnaudb.json |
[production] |
| 18:41 |
<denisse@cumin2002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:30:00 on webperf2003.codfw.wmnet,webperf1003.eqiad.wmnet with reason: Downtiming the webperf hosts part of the cergen to CFSSL migration - T360414 |
[production] |
| 18:40 |
<denisse@cumin2002> |
START - Cookbook sre.hosts.downtime for 0:30:00 on webperf2003.codfw.wmnet,webperf1003.eqiad.wmnet with reason: Downtiming the webperf hosts part of the cergen to CFSSL migration - T360414 |
[production] |
| 18:39 |
<denisse@cumin2002> |
END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 0:30:00 on performance.wikimedia.org with reason: Downtiming the webperf hosts part of the cergen to CFSSL migration - T360414 |
[production] |
| 18:39 |
<denisse@cumin2002> |
START - Cookbook sre.hosts.downtime for 0:30:00 on performance.wikimedia.org with reason: Downtiming the webperf hosts part of the cergen to CFSSL migration - T360414 |
[production] |
| 18:39 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1211', diff saved to https://phabricator.wikimedia.org/P59559 and previous config saved to /var/cache/conftool/dbconfig/20240404-183928-marostegui.json |
[production] |
| 18:37 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1207', diff saved to https://phabricator.wikimedia.org/P59558 and previous config saved to /var/cache/conftool/dbconfig/20240404-183748-marostegui.json |
[production] |
| 18:31 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1227', diff saved to https://phabricator.wikimedia.org/P59557 and previous config saved to /var/cache/conftool/dbconfig/20240404-183108-arnaudb.json |
[production] |
| 18:31 |
<denisse> |
Disabling Puppet on the webperf hosts part of the cergen to CFSSL migration - T360414 |
[production] |
| 18:24 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1211', diff saved to https://phabricator.wikimedia.org/P59556 and previous config saved to /var/cache/conftool/dbconfig/20240404-182421-marostegui.json |
[production] |
| 18:24 |
<sukhe@cumin1002> |
START - Cookbook sre.hosts.reimage for host cp4052.ulsfo.wmnet with OS bullseye |
[production] |
| 18:23 |
<sukhe@cumin1002> |
END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host cp4052.ulsfo.wmnet with OS bullseye |
[production] |
| 18:22 |
<marostegui@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1207', diff saved to https://phabricator.wikimedia.org/P59555 and previous config saved to /var/cache/conftool/dbconfig/20240404-182241-marostegui.json |
[production] |
| 18:19 |
<andrew@cumin1002> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudbackup1002-dev.eqiad.wmnet with OS bookworm |
[production] |
| 18:16 |
<sukhe@cumin1002> |
START - Cookbook sre.hosts.reimage for host cp4052.ulsfo.wmnet with OS bullseye |
[production] |
| 18:16 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1227 (T360332)', diff saved to https://phabricator.wikimedia.org/P59554 and previous config saved to /var/cache/conftool/dbconfig/20240404-181601-arnaudb.json |
[production] |