2801-2850 of 10000 results (110ms)
2024-06-26 §
03:24 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P65443 and previous config saved to /var/cache/conftool/dbconfig/20240626-032426-marostegui.json [production]
03:09 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P65442 and previous config saved to /var/cache/conftool/dbconfig/20240626-030919-marostegui.json [production]
02:54 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1178 (T364069)', diff saved to https://phabricator.wikimedia.org/P65441 and previous config saved to /var/cache/conftool/dbconfig/20240626-025412-marostegui.json [production]
00:21 <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db2138 (T367856)', diff saved to https://phabricator.wikimedia.org/P65440 and previous config saved to /var/cache/conftool/dbconfig/20240626-002103-marostegui.json [production]
00:20 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2138.codfw.wmnet with reason: Maintenance [production]
00:20 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2138.codfw.wmnet with reason: Maintenance [production]
00:20 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2126 (T367856)', diff saved to https://phabricator.wikimedia.org/P65439 and previous config saved to /var/cache/conftool/dbconfig/20240626-002041-marostegui.json [production]
00:05 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2126', diff saved to https://phabricator.wikimedia.org/P65438 and previous config saved to /var/cache/conftool/dbconfig/20240626-000534-marostegui.json [production]
2024-06-25 §
23:50 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2126', diff saved to https://phabricator.wikimedia.org/P65437 and previous config saved to /var/cache/conftool/dbconfig/20240625-235027-marostegui.json [production]
23:35 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2126 (T367856)', diff saved to https://phabricator.wikimedia.org/P65436 and previous config saved to /var/cache/conftool/dbconfig/20240625-233520-marostegui.json [production]
23:27 <andrew@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt2006-dev.codfw.wmnet with OS bookworm [production]
23:05 <andrew@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirt2006-dev.codfw.wmnet with reason: host reimage [production]
23:02 <andrew@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on cloudvirt2006-dev.codfw.wmnet with reason: host reimage [production]
22:44 <andrew@cumin1002> START - Cookbook sre.hosts.reimage for host cloudvirt2006-dev.codfw.wmnet with OS bookworm [production]
22:43 <brett@puppetmaster1001> conftool action : set/pooled=yes; selector: name=cp5017.eqsin.wmnet [production]
22:42 <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db1178 (T364069)', diff saved to https://phabricator.wikimedia.org/P65435 and previous config saved to /var/cache/conftool/dbconfig/20240625-224249-marostegui.json [production]
22:42 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance [production]
22:42 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1178.eqiad.wmnet with reason: Maintenance [production]
22:42 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1177 (T364069)', diff saved to https://phabricator.wikimedia.org/P65434 and previous config saved to /var/cache/conftool/dbconfig/20240625-224226-marostegui.json [production]
22:33 <andrew@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt2005-dev.codfw.wmnet with OS bookworm [production]
22:27 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P65433 and previous config saved to /var/cache/conftool/dbconfig/20240625-222719-marostegui.json [production]
22:12 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1177', diff saved to https://phabricator.wikimedia.org/P65432 and previous config saved to /var/cache/conftool/dbconfig/20240625-221212-marostegui.json [production]
22:10 <bvibber> a webVideoTranscode job reported 'No space left on device' from a failed ffmpeg run on mw1446 recently [production]
22:09 <andrew@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirt2005-dev.codfw.wmnet with reason: host reimage [production]
22:05 <andrew@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on cloudvirt2005-dev.codfw.wmnet with reason: host reimage [production]
21:57 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1177 (T364069)', diff saved to https://phabricator.wikimedia.org/P65431 and previous config saved to /var/cache/conftool/dbconfig/20240625-215705-marostegui.json [production]
21:47 <andrew@cumin1002> START - Cookbook sre.hosts.reimage for host cloudvirt2005-dev.codfw.wmnet with OS bookworm [production]
20:44 <cjming> end of UTC late backport window [production]
20:41 <cjming@deploy1002> Finished scap: Backport for [[gerrit:1043880|Cleanup: Remove wgNavigationTimingSurveyName (T367128)]] (duration: 08m 29s) [production]
20:36 <cjming@deploy1002> jdlrobson, cjming: Continuing with sync [production]
20:35 <cjming@deploy1002> jdlrobson, cjming: Backport for [[gerrit:1043880|Cleanup: Remove wgNavigationTimingSurveyName (T367128)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
20:32 <cjming@deploy1002> Started scap: Backport for [[gerrit:1043880|Cleanup: Remove wgNavigationTimingSurveyName (T367128)]] [production]
20:31 <cjming@deploy1002> Finished scap: Backport for [[gerrit:1041250|Enable dark mode on more pages (T366378 T367374 T366373 T366520 T366373)]] (duration: 15m 04s) [production]
20:26 <cjming@deploy1002> jdlrobson, cjming: Continuing with sync [production]
20:19 <cjming@deploy1002> jdlrobson, cjming: Backport for [[gerrit:1041250|Enable dark mode on more pages (T366378 T367374 T366373 T366520 T366373)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
20:16 <cjming@deploy1002> Started scap: Backport for [[gerrit:1041250|Enable dark mode on more pages (T366378 T367374 T366373 T366520 T366373)]] [production]
20:14 <cjming@deploy1002> Finished scap: Backport for [[gerrit:1049608|Temporarily disable '4K' 2160p and mid 1440p transcodes (T368433)]] (duration: 08m 36s) [production]
20:11 <Emperor> restart swift-proxy on ms-fe2010 ms-fe1011 T360913 [production]
20:09 <cjming@deploy1002> cjming, bvibber: Continuing with sync [production]
20:08 <cjming@deploy1002> cjming, bvibber: Backport for [[gerrit:1049608|Temporarily disable '4K' 2160p and mid 1440p transcodes (T368433)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
20:05 <cjming@deploy1002> Started scap: Backport for [[gerrit:1049608|Temporarily disable '4K' 2160p and mid 1440p transcodes (T368433)]] [production]
20:03 <brett@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp5017.eqsin.wmnet with OS bullseye [production]
20:01 <hashar@deploy1002> Finished deploy [integration/docroot@1eb5f4c]: remove CollaborationKit T368092 (duration: 00m 07s) [production]
20:01 <hashar@deploy1002> Started deploy [integration/docroot@1eb5f4c]: remove CollaborationKit T368092 [production]
19:29 <marostegui@cumin1002> dbctl commit (dc=all): 'Depooling db2126 (T367856)', diff saved to https://phabricator.wikimedia.org/P65430 and previous config saved to /var/cache/conftool/dbconfig/20240625-192947-marostegui.json [production]
19:29 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on db2187.codfw.wmnet with reason: Maintenance [production]
19:29 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 2 days, 0:00:00 on db2187.codfw.wmnet with reason: Maintenance [production]
19:29 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2126.codfw.wmnet with reason: Maintenance [production]
19:29 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2126.codfw.wmnet with reason: Maintenance [production]
19:29 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2125 (T367856)', diff saved to https://phabricator.wikimedia.org/P65429 and previous config saved to /var/cache/conftool/dbconfig/20240625-192910-marostegui.json [production]