3251-3300 of 10000 results (84ms)
2023-11-21 §
06:53 <arnaudb@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2147', diff saved to https://phabricator.wikimedia.org/P53648 and previous config saved to /var/cache/conftool/dbconfig/20231121-065333-arnaudb.json [production]
06:38 <arnaudb@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2147 (T348183)', diff saved to https://phabricator.wikimedia.org/P53647 and previous config saved to /var/cache/conftool/dbconfig/20231121-063827-arnaudb.json [production]
02:32 <ejegg> fundraising civicrm upgraded from b3da5d3f to 3a8558e7 [production]
01:35 <arnaudb@cumin1001> dbctl commit (dc=all): 'Depooling db2147 (T348183)', diff saved to https://phabricator.wikimedia.org/P53646 and previous config saved to /var/cache/conftool/dbconfig/20231121-013514-arnaudb.json [production]
01:35 <arnaudb@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2147.codfw.wmnet with reason: Maintenance [production]
01:34 <arnaudb@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2147.codfw.wmnet with reason: Maintenance [production]
2023-11-20 §
22:46 <catrope@deploy2002> Finished scap: Backport for [[gerrit:975596|[parsoid] Fix Parsoid relative links (T350952)]] (duration: 19m 32s) [production]
22:41 <catrope@deploy2002> catrope and cscott: Continuing with sync [production]
22:28 <catrope@deploy2002> catrope and cscott: Backport for [[gerrit:975596|[parsoid] Fix Parsoid relative links (T350952)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
22:27 <catrope@deploy2002> Started scap: Backport for [[gerrit:975596|[parsoid] Fix Parsoid relative links (T350952)]] [production]
22:20 <cmooney@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
22:19 <cmooney@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update entries for cloud hosts. - cmooney@cumin1001" [production]
22:19 <cmooney@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update entries for cloud hosts. - cmooney@cumin1001" [production]
22:17 <cmooney@cumin1001> START - Cookbook sre.dns.netbox [production]
22:15 <catrope@deploy2002> Finished scap: Backport for [[gerrit:975366|Revert "mw.notify: Limit width of overlay to max-width-page-container" (T349622)]] (duration: 17m 40s) [production]
22:09 <catrope@deploy2002> jdlrobson and catrope: Continuing with sync [production]
21:59 <catrope@deploy2002> jdlrobson and catrope: Backport for [[gerrit:975366|Revert "mw.notify: Limit width of overlay to max-width-page-container" (T349622)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
21:58 <catrope@deploy2002> Started scap: Backport for [[gerrit:975366|Revert "mw.notify: Limit width of overlay to max-width-page-container" (T349622)]] [production]
21:38 <catrope@deploy2002> Finished scap: Backport for [[gerrit:975879|Disable MobileFrontend AMC drawer temporarily while erroring (T351669)]] (duration: 22m 11s) [production]
21:32 <catrope@deploy2002> catrope and jdlrobson: Continuing with sync [production]
21:17 <catrope@deploy2002> catrope and jdlrobson: Backport for [[gerrit:975879|Disable MobileFrontend AMC drawer temporarily while erroring (T351669)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
21:16 <catrope@deploy2002> Started scap: Backport for [[gerrit:975879|Disable MobileFrontend AMC drawer temporarily while erroring (T351669)]] [production]
21:12 <catrope@deploy2002> Finished scap: Backport for [[gerrit:973795|Enable action blocks in ruwiki (T351048)]] (duration: 08m 52s) [production]
21:06 <catrope@deploy2002> catrope and stjn: Continuing with sync [production]
21:05 <catrope@deploy2002> catrope and stjn: Backport for [[gerrit:973795|Enable action blocks in ruwiki (T351048)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
21:03 <catrope@deploy2002> Started scap: Backport for [[gerrit:973795|Enable action blocks in ruwiki (T351048)]] [production]
21:02 <eevans@cumin1001> END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for aqs1014.eqiad.wmnet [production]
21:02 <eevans@cumin1001> START - Cookbook sre.hosts.remove-downtime for aqs1014.eqiad.wmnet [production]
21:02 <eevans@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host aqs1014.eqiad.wmnet with OS bullseye [production]
20:40 <eevans@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on aqs1014.eqiad.wmnet with reason: host reimage [production]
20:37 <eevans@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on aqs1014.eqiad.wmnet with reason: host reimage [production]
20:34 <arnaudb@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2139.codfw.wmnet with reason: Maintenance [production]
20:33 <arnaudb@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2139.codfw.wmnet with reason: Maintenance [production]
20:33 <arnaudb@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2138:3314 (T348183)', diff saved to https://phabricator.wikimedia.org/P53645 and previous config saved to /var/cache/conftool/dbconfig/20231120-203337-arnaudb.json [production]
20:21 <eevans@cumin1001> START - Cookbook sre.hosts.reimage for host aqs1014.eqiad.wmnet with OS bullseye [production]
20:21 <eevans@cumin1001> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host aqs1014.eqiad.wmnet with OS bullseye [production]
20:18 <arnaudb@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2138:3314', diff saved to https://phabricator.wikimedia.org/P53644 and previous config saved to /var/cache/conftool/dbconfig/20231120-201831-arnaudb.json [production]
20:10 <eevans@cumin1001> START - Cookbook sre.hosts.reimage for host aqs1014.eqiad.wmnet with OS bullseye [production]
20:08 <eevans@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host aqs1013.eqiad.wmnet with OS bullseye [production]
20:03 <arnaudb@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2138:3314', diff saved to https://phabricator.wikimedia.org/P53643 and previous config saved to /var/cache/conftool/dbconfig/20231120-200324-arnaudb.json [production]
19:59 <brett@cumin2002> END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for acmechief2001.codfw.wmnet [production]
19:59 <brett@cumin2002> START - Cookbook sre.hosts.remove-downtime for acmechief2001.codfw.wmnet [production]
19:50 <eevans@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on aqs1013.eqiad.wmnet with reason: host reimage [production]
19:48 <arnaudb@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2138:3314 (T348183)', diff saved to https://phabricator.wikimedia.org/P53642 and previous config saved to /var/cache/conftool/dbconfig/20231120-194818-arnaudb.json [production]
19:48 <eevans@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on aqs1013.eqiad.wmnet with reason: host reimage [production]
19:36 <eevans@cumin1001> START - Cookbook sre.hosts.reimage for host aqs1013.eqiad.wmnet with OS bullseye [production]
19:21 <sukhe> pool cp4045.ulsfo.wmnet post reboot and puppet 7 upgrade [production]
19:16 <sukhe@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cp4045.ulsfo.wmnet [production]
19:05 <sukhe@cumin2002> START - Cookbook sre.hosts.reboot-single for host cp4045.ulsfo.wmnet [production]
19:04 <cmooney@cumin1001> END (FAIL) - Cookbook sre.dns.netbox (exit_code=99) [production]