51-100 of 10000 results (75ms)
2023-11-13 ยง
21:29 <arnaudb@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2178', diff saved to https://phabricator.wikimedia.org/P53375 and previous config saved to /var/cache/conftool/dbconfig/20231113-212904-arnaudb.json [production]
21:28 <btullis@deploy2002> helmfile [staging] START helmfile.d/services/datahub: apply on main [production]
21:26 <urbanecm@deploy2002> urbanecm and brion and dani: Continuing with sync [production]
21:25 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt1034.eqiad.wmnet with OS bookworm [production]
21:22 <urbanecm@deploy2002> urbanecm and brion and dani: Backport for [[gerrit:973808|Undeploy pilot survey on metawiki (T349854)]], [[gerrit:973790|Don't change transcode rows during read operations (T152851)]], [[gerrit:973791|Fixes to requeueTranscodes to make it easier to batch-fill (T68722)]], [[gerrit:973793|Only include completed transcodes in .m3u8 playlist (T350996)]] synced to the testservers (https://wikitech.wiki [production]
21:21 <urbanecm@deploy2002> Started scap: Backport for [[gerrit:973808|Undeploy pilot survey on metawiki (T349854)]], [[gerrit:973790|Don't change transcode rows during read operations (T152851)]], [[gerrit:973791|Fixes to requeueTranscodes to make it easier to batch-fill (T68722)]], [[gerrit:973793|Only include completed transcodes in .m3u8 playlist (T350996)]] [production]
21:20 <urbanecm@deploy2002> Sync cancelled. [production]
21:19 <urbanecm@deploy2002> urbanecm and dani: Backport for [[gerrit:973808|Undeploy pilot survey on metawiki (T349854)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
21:19 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts cloudvirt1030.eqiad.wmnet [production]
21:18 <andrew@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
21:18 <andrew@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: cloudvirt1030.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - andrew@cumin1001" [production]
21:18 <andrew@cumin1001> START - Cookbook sre.hosts.reimage for host cloudvirt1035.eqiad.wmnet with OS bookworm [production]
21:18 <urbanecm@deploy2002> Started scap: Backport for [[gerrit:973808|Undeploy pilot survey on metawiki (T349854)]] [production]
21:17 <andrew@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: cloudvirt1030.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - andrew@cumin1001" [production]
21:16 <urbanecm@deploy2002> Finished scap: Backport for [[gerrit:973870|Enable edit check on swwiki (T350921)]], [[gerrit:973814|Fix Reader Demographics 2 survey (T345951)]] (duration: 10m 15s) [production]
21:15 <andrew@cumin1001> START - Cookbook sre.dns.netbox [production]
21:13 <arnaudb@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2178', diff saved to https://phabricator.wikimedia.org/P53374 and previous config saved to /var/cache/conftool/dbconfig/20231113-211358-arnaudb.json [production]
21:11 <urbanecm@deploy2002> dani and kemayo and urbanecm: Continuing with sync [production]
21:11 <andrew@cumin1001> START - Cookbook sre.hosts.decommission for hosts cloudvirt1030.eqiad.wmnet [production]
21:10 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts cloudvirt1029.eqiad.wmnet [production]
21:10 <andrew@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
21:10 <andrew@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: cloudvirt1029.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - andrew@cumin1001" [production]
21:09 <andrew@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: cloudvirt1029.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - andrew@cumin1001" [production]
21:07 <urbanecm@deploy2002> dani and kemayo and urbanecm: Backport for [[gerrit:973870|Enable edit check on swwiki (T350921)]], [[gerrit:973814|Fix Reader Demographics 2 survey (T345951)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
21:06 <andrew@cumin1001> START - Cookbook sre.dns.netbox [production]
21:06 <urbanecm@deploy2002> Started scap: Backport for [[gerrit:973870|Enable edit check on swwiki (T350921)]], [[gerrit:973814|Fix Reader Demographics 2 survey (T345951)]] [production]
21:02 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirt1034.eqiad.wmnet with reason: host reimage [production]
21:02 <andrew@cumin1001> START - Cookbook sre.hosts.decommission for hosts cloudvirt1029.eqiad.wmnet [production]
21:02 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts cloudvirt1028.eqiad.wmnet [production]
21:02 <andrew@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
21:01 <andrew@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: cloudvirt1028.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - andrew@cumin1001" [production]
21:00 <andrew@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: cloudvirt1028.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - andrew@cumin1001" [production]
20:59 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt1033.eqiad.wmnet with OS bookworm [production]
20:59 <andrew@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on cloudvirt1034.eqiad.wmnet with reason: host reimage [production]
20:58 <arnaudb@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2178 (T348183)', diff saved to https://phabricator.wikimedia.org/P53373 and previous config saved to /var/cache/conftool/dbconfig/20231113-205852-arnaudb.json [production]
20:58 <andrew@cumin1001> START - Cookbook sre.dns.netbox [production]
20:53 <andrew@cumin1001> START - Cookbook sre.hosts.decommission for hosts cloudvirt1028.eqiad.wmnet [production]
20:52 <bking@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 14 days, 0:00:00 on wdqs1023.eqiad.wmnet with reason: T347504 [production]
20:52 <bking@cumin1001> START - Cookbook sre.hosts.downtime for 14 days, 0:00:00 on wdqs1023.eqiad.wmnet with reason: T347504 [production]
20:52 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts cloudvirt1027.eqiad.wmnet [production]
20:52 <andrew@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
20:52 <andrew@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: cloudvirt1027.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - andrew@cumin1001" [production]
20:52 <bking@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 14 days, 0:00:00 on wdqs1024.eqiad.wmnet with reason: T347504 [production]
20:52 <bking@cumin1001> START - Cookbook sre.hosts.downtime for 14 days, 0:00:00 on wdqs1024.eqiad.wmnet with reason: T347504 [production]
20:51 <bking@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 14 days, 0:00:00 on wdqs1022.eqiad.wmnet with reason: T347504 [production]
20:51 <bking@cumin1001> START - Cookbook sre.hosts.downtime for 14 days, 0:00:00 on wdqs1022.eqiad.wmnet with reason: T347504 [production]
20:51 <andrew@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: cloudvirt1027.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - andrew@cumin1001" [production]
20:50 <arnaudb@cumin1001> dbctl commit (dc=all): 'Depooling db2178 (T348183)', diff saved to https://phabricator.wikimedia.org/P53372 and previous config saved to /var/cache/conftool/dbconfig/20231113-205032-arnaudb.json [production]
20:50 <arnaudb@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2178.codfw.wmnet with reason: Maintenance [production]
20:50 <arnaudb@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2178.codfw.wmnet with reason: Maintenance [production]