1201-1250 of 10000 results (42ms)
2022-03-08 ยง
10:12 <jmm@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cumin2002.codfw.wmnet [production]
10:10 <elukey@cumin1001> START - Cookbook sre.hosts.reboot-single for host ml-serve2001.codfw.wmnet [production]
10:06 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1102.eqiad.wmnet with reason: Maintenance [production]
10:06 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 12:00:00 on db1102.eqiad.wmnet with reason: Maintenance [production]
10:06 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1179 (T300381)', diff saved to https://phabricator.wikimedia.org/P22065 and previous config saved to /var/cache/conftool/dbconfig/20220308-100559-marostegui.json [production]
10:03 <jmm@cumin1001> START - Cookbook sre.hosts.reboot-single for host cumin2002.codfw.wmnet [production]
10:02 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1180', diff saved to https://phabricator.wikimedia.org/P22064 and previous config saved to /var/cache/conftool/dbconfig/20220308-100234-marostegui.json [production]
09:56 <vgutierrez@cumin1001> START - Cookbook sre.hosts.reimage for host cp2035.codfw.wmnet with OS buster [production]
09:50 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1179', diff saved to https://phabricator.wikimedia.org/P22063 and previous config saved to /var/cache/conftool/dbconfig/20220308-095055-marostegui.json [production]
09:47 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1180 (T298294)', diff saved to https://phabricator.wikimedia.org/P22062 and previous config saved to /var/cache/conftool/dbconfig/20220308-094730-marostegui.json [production]
09:46 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1180 (T298294)', diff saved to https://phabricator.wikimedia.org/P22061 and previous config saved to /var/cache/conftool/dbconfig/20220308-094613-marostegui.json [production]
09:46 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1180.eqiad.wmnet with reason: Maintenance [production]
09:46 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 8:00:00 on db1180.eqiad.wmnet with reason: Maintenance [production]
09:46 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1165 (T298294)', diff saved to https://phabricator.wikimedia.org/P22060 and previous config saved to /var/cache/conftool/dbconfig/20220308-094605-marostegui.json [production]
09:43 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1174 (T300775)', diff saved to https://phabricator.wikimedia.org/P22059 and previous config saved to /var/cache/conftool/dbconfig/20220308-094354-marostegui.json [production]
09:43 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on db1174.eqiad.wmnet with reason: Maintenance [production]
09:43 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 2 days, 0:00:00 on db1174.eqiad.wmnet with reason: Maintenance [production]
09:41 <marostegui@cumin1001> dbctl commit (dc=all): 'db1158 (re)pooling @ 100%: After schema change', diff saved to https://phabricator.wikimedia.org/P22058 and previous config saved to /var/cache/conftool/dbconfig/20220308-094155-root.json [production]
09:35 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1179', diff saved to https://phabricator.wikimedia.org/P22057 and previous config saved to /var/cache/conftool/dbconfig/20220308-093550-marostegui.json [production]
09:34 <jayme@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host kubernetes2022.codfw.wmnet [production]
09:31 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1165', diff saved to https://phabricator.wikimedia.org/P22056 and previous config saved to /var/cache/conftool/dbconfig/20220308-093101-marostegui.json [production]
09:27 <jayme@cumin1001> START - Cookbook sre.hosts.reboot-single for host kubernetes2022.codfw.wmnet [production]
09:26 <marostegui@cumin1001> dbctl commit (dc=all): 'db1158 (re)pooling @ 75%: After schema change', diff saved to https://phabricator.wikimedia.org/P22055 and previous config saved to /var/cache/conftool/dbconfig/20220308-092651-root.json [production]
09:26 <jayme@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host kubernetes2021.codfw.wmnet [production]
09:20 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1179 (T300381)', diff saved to https://phabricator.wikimedia.org/P22054 and previous config saved to /var/cache/conftool/dbconfig/20220308-092045-marostegui.json [production]
09:18 <jayme@cumin1001> START - Cookbook sre.hosts.reboot-single for host kubernetes2021.codfw.wmnet [production]
09:15 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1165', diff saved to https://phabricator.wikimedia.org/P22053 and previous config saved to /var/cache/conftool/dbconfig/20220308-091556-marostegui.json [production]
09:11 <marostegui@cumin1001> dbctl commit (dc=all): 'db1158 (re)pooling @ 50%: After schema change', diff saved to https://phabricator.wikimedia.org/P22052 and previous config saved to /var/cache/conftool/dbconfig/20220308-091147-root.json [production]
09:10 <jayme@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host kubernetes2020.codfw.wmnet [production]
09:05 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1179 (T300381)', diff saved to https://phabricator.wikimedia.org/P22051 and previous config saved to /var/cache/conftool/dbconfig/20220308-090531-marostegui.json [production]
09:05 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1179.eqiad.wmnet with reason: Maintenance [production]
09:05 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 12:00:00 on db1179.eqiad.wmnet with reason: Maintenance [production]
09:03 <jayme@cumin1001> START - Cookbook sre.hosts.reboot-single for host kubernetes2020.codfw.wmnet [production]
09:00 <jayme@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host kubernetes2019.codfw.wmnet [production]
09:00 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1165 (T298294)', diff saved to https://phabricator.wikimedia.org/P22050 and previous config saved to /var/cache/conftool/dbconfig/20220308-090051-marostegui.json [production]
08:59 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1165 (T298294)', diff saved to https://phabricator.wikimedia.org/P22049 and previous config saved to /var/cache/conftool/dbconfig/20220308-085934-marostegui.json [production]
08:59 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 16:00:00 on clouddb[1015,1019,1021].eqiad.wmnet,db1155.eqiad.wmnet with reason: Maintenance [production]
08:59 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 16:00:00 on clouddb[1015,1019,1021].eqiad.wmnet,db1155.eqiad.wmnet with reason: Maintenance [production]
08:59 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db1165.eqiad.wmnet with reason: Maintenance [production]
08:59 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 8:00:00 on db1165.eqiad.wmnet with reason: Maintenance [production]
08:59 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1168 (T298294)', diff saved to https://phabricator.wikimedia.org/P22048 and previous config saved to /var/cache/conftool/dbconfig/20220308-085921-marostegui.json [production]
08:56 <marostegui@cumin1001> dbctl commit (dc=all): 'db1158 (re)pooling @ 25%: After schema change', diff saved to https://phabricator.wikimedia.org/P22047 and previous config saved to /var/cache/conftool/dbconfig/20220308-085644-root.json [production]
08:54 <jayme@cumin1001> START - Cookbook sre.hosts.reboot-single for host kubernetes2019.codfw.wmnet [production]
08:44 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1168', diff saved to https://phabricator.wikimedia.org/P22046 and previous config saved to /var/cache/conftool/dbconfig/20220308-084416-marostegui.json [production]
08:41 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance [production]
08:41 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 12:00:00 on dbstore1007.eqiad.wmnet with reason: Maintenance [production]
08:41 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1175 (T300381)', diff saved to https://phabricator.wikimedia.org/P22045 and previous config saved to /var/cache/conftool/dbconfig/20220308-084148-marostegui.json [production]
08:39 <jayme@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host kubernetes2018.codfw.wmnet [production]
08:32 <jayme@cumin1001> START - Cookbook sre.hosts.reboot-single for host kubernetes2018.codfw.wmnet [production]
08:29 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1168', diff saved to https://phabricator.wikimedia.org/P22044 and previous config saved to /var/cache/conftool/dbconfig/20220308-082912-marostegui.json [production]