101-150 of 10000 results (24ms)
2026-01-08 ยง
16:41 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1159', diff saved to https://phabricator.wikimedia.org/P86861 and previous config saved to /var/cache/conftool/dbconfig/20260108-164140-marostegui.json [production]
16:31 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1159 (T413525)', diff saved to https://phabricator.wikimedia.org/P86860 and previous config saved to /var/cache/conftool/dbconfig/20260108-163131-marostegui.json [production]
16:18 <marostegui@cumin1003> dbctl commit (dc=all): 'Depooling db1159 (T413525)', diff saved to https://phabricator.wikimedia.org/P86858 and previous config saved to /var/cache/conftool/dbconfig/20260108-161848-marostegui.json [production]
16:18 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1159.eqiad.wmnet with reason: Maintenance [production]
16:14 <elukey@cumin1003> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host wdqs1029.eqiad.wmnet with OS trixie [production]
16:10 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2229 (T413525)', diff saved to https://phabricator.wikimedia.org/P86857 and previous config saved to /var/cache/conftool/dbconfig/20260108-161038-marostegui.json [production]
16:00 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2229', diff saved to https://phabricator.wikimedia.org/P86856 and previous config saved to /var/cache/conftool/dbconfig/20260108-160029-marostegui.json [production]
15:55 <dancy> Upgrading gitlab-runner to v18.5.0 on gitlab-cloud-runners. (T414053) [releng]
15:54 <elukey@cumin1003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wdqs1029.eqiad.wmnet with reason: host reimage [production]
15:50 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2229', diff saved to https://phabricator.wikimedia.org/P86855 and previous config saved to /var/cache/conftool/dbconfig/20260108-155021-marostegui.json [production]
15:48 <elukey@cumin1003> START - Cookbook sre.hosts.downtime for 2:00:00 on wdqs1029.eqiad.wmnet with reason: host reimage [production]
15:44 <andrew@cumin2002> START - Cookbook sre.hosts.reimage for host cloudbackup2003.codfw.wmnet with OS trixie [production]
15:44 <andrew@cumin2002> START - Cookbook sre.hosts.reimage for host cloudbackup1003.eqiad.wmnet with OS trixie [production]
15:40 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2229 (T413525)', diff saved to https://phabricator.wikimedia.org/P86854 and previous config saved to /var/cache/conftool/dbconfig/20260108-154013-marostegui.json [production]
15:29 <elukey@cumin1003> START - Cookbook sre.hosts.reimage for host wdqs1029.eqiad.wmnet with OS trixie [production]
15:24 <marostegui@cumin1003> dbctl commit (dc=all): 'Depooling db2229 (T413525)', diff saved to https://phabricator.wikimedia.org/P86853 and previous config saved to /var/cache/conftool/dbconfig/20260108-152432-marostegui.json [production]
15:24 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2229.codfw.wmnet with reason: Maintenance [production]
15:24 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2224 (T413525)', diff saved to https://phabricator.wikimedia.org/P86852 and previous config saved to /var/cache/conftool/dbconfig/20260108-152407-marostegui.json [production]
15:21 <marostegui@cumin1003> dbctl commit (dc=all): 'Depooling db1190 (T411163 T411164)', diff saved to https://phabricator.wikimedia.org/P86851 and previous config saved to /var/cache/conftool/dbconfig/20260108-152103-marostegui.json [production]
15:20 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3 days, 0:00:00 on db1190.eqiad.wmnet with reason: Maintenance [production]
15:14 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2224', diff saved to https://phabricator.wikimedia.org/P86850 and previous config saved to /var/cache/conftool/dbconfig/20260108-151400-marostegui.json [production]
15:10 <Lucas_WMDE> UTC afternoon backport+config window done [production]
15:09 <mszwarc@deploy2002> Finished scap sync-world: Backport for [[gerrit:1224688|Revert^2 "Silence TransactionProfiler warnings in CheckUserPrivateEventsHandler"]], [[gerrit:1224673|Update API call in edit.js with rvslots (T412762)]] (duration: 09m 11s) [production]
15:05 <mszwarc@deploy2002> ladsgroup, mszwarc: Continuing with sync [production]
15:03 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2224', diff saved to https://phabricator.wikimedia.org/P86849 and previous config saved to /var/cache/conftool/dbconfig/20260108-150351-marostegui.json [production]
15:02 <mszwarc@deploy2002> ladsgroup, mszwarc: Backport for [[gerrit:1224688|Revert^2 "Silence TransactionProfiler warnings in CheckUserPrivateEventsHandler"]], [[gerrit:1224673|Update API call in edit.js with rvslots (T412762)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
15:00 <mszwarc@deploy2002> Started scap sync-world: Backport for [[gerrit:1224688|Revert^2 "Silence TransactionProfiler warnings in CheckUserPrivateEventsHandler"]], [[gerrit:1224673|Update API call in edit.js with rvslots (T412762)]] [production]
14:58 <elukey@cumin1003> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wdqs1029.eqiad.wmnet with OS trixie [production]
14:55 <lucaswerkmeister-wmde@deploy2002> Finished scap sync-world: Backport for [[gerrit:1214986|Enable the MEX / wbui2025 beta feature on wikidata (T403015)]] (duration: 08m 49s) [production]
14:53 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2224 (T413525)', diff saved to https://phabricator.wikimedia.org/P86848 and previous config saved to /var/cache/conftool/dbconfig/20260108-145343-marostegui.json [production]
14:51 <lucaswerkmeister-wmde@deploy2002> lucaswerkmeister-wmde, arthurtaylor: Continuing with sync [production]
14:51 <urbanecm> Add `mszwarc` to `wmf-deployment` on Gerrit (existing deployer, T404697) [production]
14:48 <lucaswerkmeister-wmde@deploy2002> lucaswerkmeister-wmde, arthurtaylor: Backport for [[gerrit:1214986|Enable the MEX / wbui2025 beta feature on wikidata (T403015)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
14:47 <jgreen@cumin1003> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
14:46 <lucaswerkmeister-wmde@deploy2002> Started scap sync-world: Backport for [[gerrit:1214986|Enable the MEX / wbui2025 beta feature on wikidata (T403015)]] [production]
14:45 <jgreen@cumin1003> START - Cookbook sre.dns.netbox [production]
14:43 <mszwarc@deploy2002> Finished scap sync-world: Backport for [[gerrit:1223155|[enwikiquote] Enable block feature for AbuseFilter (T413530)]], [[gerrit:1223159|[ruwiki] Disable setting a cookie for blocked anonymous users (T413737)]], [[gerrit:1224232|[enwikiquote] Add new autopatrolled and patroller usergroups (T413848)]] (duration: 08m 36s) [production]
14:39 <mszwarc@deploy2002> mszwarc, superpes: Continuing with sync [production]
14:37 <marostegui@cumin1003> dbctl commit (dc=all): 'Depooling db2224 (T413525)', diff saved to https://phabricator.wikimedia.org/P86847 and previous config saved to /var/cache/conftool/dbconfig/20260108-143755-marostegui.json [production]
14:37 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2224.codfw.wmnet with reason: Maintenance [production]
14:37 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2217 (T413525)', diff saved to https://phabricator.wikimedia.org/P86846 and previous config saved to /var/cache/conftool/dbconfig/20260108-143730-marostegui.json [production]
14:36 <mszwarc@deploy2002> mszwarc, superpes: Backport for [[gerrit:1223155|[enwikiquote] Enable block feature for AbuseFilter (T413530)]], [[gerrit:1223159|[ruwiki] Disable setting a cookie for blocked anonymous users (T413737)]], [[gerrit:1224232|[enwikiquote] Add new autopatrolled and patroller usergroups (T413848)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified t [production]
14:36 <elukey@cumin1003> START - Cookbook sre.hosts.reimage for host wdqs1029.eqiad.wmnet with OS trixie [production]
14:34 <jgreen@cumin1003> END (FAIL) - Cookbook sre.dns.netbox (exit_code=99) [production]
14:34 <mszwarc@deploy2002> Started scap sync-world: Backport for [[gerrit:1223155|[enwikiquote] Enable block feature for AbuseFilter (T413530)]], [[gerrit:1223159|[ruwiki] Disable setting a cookie for blocked anonymous users (T413737)]], [[gerrit:1224232|[enwikiquote] Add new autopatrolled and patroller usergroups (T413848)]] [production]
14:33 <moritzm> installing systemd bugfix updates from Bookworm point release [production]
14:31 <mszwarc@deploy2002> Finished scap sync-world: Backport for [[gerrit:1224685|Revert "Silence TransactionProfiler warnings in CheckUserPrivateEventsHandler"]] (duration: 07m 34s) [production]
14:27 <mszwarc@deploy2002> mszwarc, trainbranchbot: Continuing with sync [production]
14:27 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2217', diff saved to https://phabricator.wikimedia.org/P86845 and previous config saved to /var/cache/conftool/dbconfig/20260108-142722-marostegui.json [production]
14:26 <jgreen@cumin1003> START - Cookbook sre.dns.netbox [production]