51-100 of 10000 results (71ms)
2024-04-04 ยง
20:41 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2108 (T360332)', diff saved to https://phabricator.wikimedia.org/P59579 and previous config saved to /var/cache/conftool/dbconfig/20240404-204141-arnaudb.json [production]
20:33 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1216.eqiad.wmnet with reason: Maintenance [production]
20:33 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 8:00:00 on db1216.eqiad.wmnet with reason: Maintenance [production]
20:33 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1214 (T355609)', diff saved to https://phabricator.wikimedia.org/P59578 and previous config saved to /var/cache/conftool/dbconfig/20240404-203302-marostegui.json [production]
20:26 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2108', diff saved to https://phabricator.wikimedia.org/P59577 and previous config saved to /var/cache/conftool/dbconfig/20240404-202634-arnaudb.json [production]
20:17 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1214', diff saved to https://phabricator.wikimedia.org/P59576 and previous config saved to /var/cache/conftool/dbconfig/20240404-201755-marostegui.json [production]
20:11 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2108', diff saved to https://phabricator.wikimedia.org/P59575 and previous config saved to /var/cache/conftool/dbconfig/20240404-201126-arnaudb.json [production]
20:02 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1214', diff saved to https://phabricator.wikimedia.org/P59574 and previous config saved to /var/cache/conftool/dbconfig/20240404-200247-marostegui.json [production]
19:56 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2108 (T360332)', diff saved to https://phabricator.wikimedia.org/P59573 and previous config saved to /var/cache/conftool/dbconfig/20240404-195615-arnaudb.json [production]
19:53 <arnaudb@cumin1002> dbctl commit (dc=all): 'Depooling db2108 (T360332)', diff saved to https://phabricator.wikimedia.org/P59572 and previous config saved to /var/cache/conftool/dbconfig/20240404-195333-arnaudb.json [production]
19:53 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2108.codfw.wmnet with reason: Maintenance [production]
19:53 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 12:00:00 on db2108.codfw.wmnet with reason: Maintenance [production]
19:52 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2098.codfw.wmnet with reason: Maintenance [production]
19:52 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 12:00:00 on db2098.codfw.wmnet with reason: Maintenance [production]
19:51 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on dbstore1008.eqiad.wmnet with reason: Maintenance [production]
19:51 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 12:00:00 on dbstore1008.eqiad.wmnet with reason: Maintenance [production]
19:51 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1236 (T360332)', diff saved to https://phabricator.wikimedia.org/P59571 and previous config saved to /var/cache/conftool/dbconfig/20240404-195138-arnaudb.json [production]
19:47 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1214 (T355609)', diff saved to https://phabricator.wikimedia.org/P59570 and previous config saved to /var/cache/conftool/dbconfig/20240404-194739-marostegui.json [production]
19:36 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1236', diff saved to https://phabricator.wikimedia.org/P59569 and previous config saved to /var/cache/conftool/dbconfig/20240404-193631-arnaudb.json [production]
19:21 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1236', diff saved to https://phabricator.wikimedia.org/P59568 and previous config saved to /var/cache/conftool/dbconfig/20240404-192123-arnaudb.json [production]
19:11 <sukhe@puppetmaster1001> conftool action : set/pooled=yes; selector: name=cp4052.ulsfo.wmnet,service=(cdn|ats-be) [production]
19:10 <sukhe@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp4052.ulsfo.wmnet with OS bullseye [production]
19:06 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1236 (T360332)', diff saved to https://phabricator.wikimedia.org/P59567 and previous config saved to /var/cache/conftool/dbconfig/20240404-190616-arnaudb.json [production]
19:01 <arnaudb@cumin1002> dbctl commit (dc=all): 'Depooling db1236 (T360332)', diff saved to https://phabricator.wikimedia.org/P59566 and previous config saved to /var/cache/conftool/dbconfig/20240404-190146-arnaudb.json [production]
19:01 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1236.eqiad.wmnet with reason: Maintenance [production]
19:01 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 12:00:00 on db1236.eqiad.wmnet with reason: Maintenance [production]
19:01 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1227 (T360332)', diff saved to https://phabricator.wikimedia.org/P59565 and previous config saved to /var/cache/conftool/dbconfig/20240404-190123-arnaudb.json [production]
18:54 <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db1214 (T355609)', diff saved to https://phabricator.wikimedia.org/P59564 and previous config saved to /var/cache/conftool/dbconfig/20240404-185458-marostegui.json [production]
18:54 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1214.eqiad.wmnet with reason: Maintenance [production]
18:54 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 8:00:00 on db1214.eqiad.wmnet with reason: Maintenance [production]
18:54 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1211 (T355609)', diff saved to https://phabricator.wikimedia.org/P59563 and previous config saved to /var/cache/conftool/dbconfig/20240404-185436-marostegui.json [production]
18:53 <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db1218 (T356166)', diff saved to https://phabricator.wikimedia.org/P59562 and previous config saved to /var/cache/conftool/dbconfig/20240404-185319-marostegui.json [production]
18:53 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1218.eqiad.wmnet with reason: Maintenance [production]
18:53 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 8:00:00 on db1218.eqiad.wmnet with reason: Maintenance [production]
18:52 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1207 (T356166)', diff saved to https://phabricator.wikimedia.org/P59561 and previous config saved to /var/cache/conftool/dbconfig/20240404-185256-marostegui.json [production]
18:49 <sukhe@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cp4052.ulsfo.wmnet with reason: host reimage [production]
18:46 <sukhe@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on cp4052.ulsfo.wmnet with reason: host reimage [production]
18:46 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1227', diff saved to https://phabricator.wikimedia.org/P59560 and previous config saved to /var/cache/conftool/dbconfig/20240404-184616-arnaudb.json [production]
18:41 <denisse@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:30:00 on webperf2003.codfw.wmnet,webperf1003.eqiad.wmnet with reason: Downtiming the webperf hosts part of the cergen to CFSSL migration - T360414 [production]
18:40 <denisse@cumin2002> START - Cookbook sre.hosts.downtime for 0:30:00 on webperf2003.codfw.wmnet,webperf1003.eqiad.wmnet with reason: Downtiming the webperf hosts part of the cergen to CFSSL migration - T360414 [production]
18:39 <denisse@cumin2002> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 0:30:00 on performance.wikimedia.org with reason: Downtiming the webperf hosts part of the cergen to CFSSL migration - T360414 [production]
18:39 <denisse@cumin2002> START - Cookbook sre.hosts.downtime for 0:30:00 on performance.wikimedia.org with reason: Downtiming the webperf hosts part of the cergen to CFSSL migration - T360414 [production]
18:39 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1211', diff saved to https://phabricator.wikimedia.org/P59559 and previous config saved to /var/cache/conftool/dbconfig/20240404-183928-marostegui.json [production]
18:37 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1207', diff saved to https://phabricator.wikimedia.org/P59558 and previous config saved to /var/cache/conftool/dbconfig/20240404-183748-marostegui.json [production]
18:31 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1227', diff saved to https://phabricator.wikimedia.org/P59557 and previous config saved to /var/cache/conftool/dbconfig/20240404-183108-arnaudb.json [production]
18:31 <denisse> Disabling Puppet on the webperf hosts part of the cergen to CFSSL migration - T360414 [production]
18:24 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1211', diff saved to https://phabricator.wikimedia.org/P59556 and previous config saved to /var/cache/conftool/dbconfig/20240404-182421-marostegui.json [production]
18:24 <sukhe@cumin1002> START - Cookbook sre.hosts.reimage for host cp4052.ulsfo.wmnet with OS bullseye [production]
18:23 <sukhe@cumin1002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host cp4052.ulsfo.wmnet with OS bullseye [production]
18:22 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1207', diff saved to https://phabricator.wikimedia.org/P59555 and previous config saved to /var/cache/conftool/dbconfig/20240404-182241-marostegui.json [production]