1-50 of 10000 results (93ms)
2026-01-22 §
10:14 <marostegui@cumin1003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host dbproxy2007.codfw.wmnet with OS trixie [production]
09:50 <marostegui@cumin1003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on dbproxy2007.codfw.wmnet with reason: host reimage [production]
09:48 <jgiannelos@deploy2002> helmfile [staging] DONE helmfile.d/services/mobileapps: apply [production]
09:48 <jgiannelos@deploy2002> helmfile [staging] START helmfile.d/services/mobileapps: apply [production]
09:46 <marostegui@cumin1003> START - Cookbook sre.hosts.downtime for 2:00:00 on dbproxy2007.codfw.wmnet with reason: host reimage [production]
09:44 <aklapper@deploy2002> rebuilt and synchronized wikiversions files: group2 to 1.46.0-wmf.12 refs T413803 [production]
09:31 <marostegui@cumin1003> START - Cookbook sre.hosts.reimage for host dbproxy2007.codfw.wmnet with OS trixie [production]
09:26 <aklapper@deploy2002> rebuilt and synchronized wikiversions files: group1 to 1.46.0-wmf.12 refs T413803 [production]
09:18 <aklapper@deploy2002> Finished scap sync-world: Backport for [[gerrit:1229724|Revert "Fix DivisionByZeroError when calculating bitrate" (T415169)]] (duration: 07m 13s) [production]
09:13 <aklapper@deploy2002> jforrester, aklapper: Continuing with sync [production]
09:13 <aklapper@deploy2002> jforrester, aklapper: Backport for [[gerrit:1229724|Revert "Fix DivisionByZeroError when calculating bitrate" (T415169)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
09:10 <aklapper@deploy2002> Started scap sync-world: Backport for [[gerrit:1229724|Revert "Fix DivisionByZeroError when calculating bitrate" (T415169)]] [production]
08:06 <ladsgroup@cumin1003> dbctl commit (dc=all): 'Depooling db2225 (T410589)', diff saved to https://phabricator.wikimedia.org/P87861 and previous config saved to /var/cache/conftool/dbconfig/20260122-080653-ladsgroup.json [production]
08:06 <ladsgroup@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3 days, 0:00:00 on db2225.codfw.wmnet with reason: Maintenance [production]
08:06 <ladsgroup@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2207 (T410589)', diff saved to https://phabricator.wikimedia.org/P87860 and previous config saved to /var/cache/conftool/dbconfig/20260122-080629-ladsgroup.json [production]
07:56 <ladsgroup@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2207', diff saved to https://phabricator.wikimedia.org/P87859 and previous config saved to /var/cache/conftool/dbconfig/20260122-075620-ladsgroup.json [production]
07:46 <ladsgroup@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2207', diff saved to https://phabricator.wikimedia.org/P87858 and previous config saved to /var/cache/conftool/dbconfig/20260122-074612-ladsgroup.json [production]
07:36 <ladsgroup@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2207 (T410589)', diff saved to https://phabricator.wikimedia.org/P87857 and previous config saved to /var/cache/conftool/dbconfig/20260122-073604-ladsgroup.json [production]
07:15 <marostegui@cumin1003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host dbproxy2006.codfw.wmnet with OS trixie [production]
06:55 <marostegui@cumin1003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on dbproxy2006.codfw.wmnet with reason: host reimage [production]
06:51 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3 days, 0:00:00 on db1239.eqiad.wmnet with reason: Maintenance [production]
06:51 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1235 (T411163 T411164)', diff saved to https://phabricator.wikimedia.org/P87856 and previous config saved to /var/cache/conftool/dbconfig/20260122-065138-marostegui.json [production]
06:49 <marostegui@cumin1003> START - Cookbook sre.hosts.downtime for 2:00:00 on dbproxy2006.codfw.wmnet with reason: host reimage [production]
06:41 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1235', diff saved to https://phabricator.wikimedia.org/P87855 and previous config saved to /var/cache/conftool/dbconfig/20260122-064131-marostegui.json [production]
06:33 <marostegui@cumin1003> START - Cookbook sre.hosts.reimage for host dbproxy2006.codfw.wmnet with OS trixie [production]
06:31 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1235', diff saved to https://phabricator.wikimedia.org/P87854 and previous config saved to /var/cache/conftool/dbconfig/20260122-063122-marostegui.json [production]
06:21 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1235 (T411163 T411164)', diff saved to https://phabricator.wikimedia.org/P87853 and previous config saved to /var/cache/conftool/dbconfig/20260122-062114-marostegui.json [production]
06:06 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3 days, 0:00:00 on db1240.eqiad.wmnet with reason: long schema change [production]
06:05 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3 days, 0:00:00 on db1239.eqiad.wmnet with reason: long schema change [production]
04:19 <marostegui@cumin1003> dbctl commit (dc=all): 'Depooling db2163 (T411163 T411164)', diff saved to https://phabricator.wikimedia.org/P87852 and previous config saved to /var/cache/conftool/dbconfig/20260122-041956-marostegui.json [production]
04:19 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3 days, 0:00:00 on db2163.codfw.wmnet with reason: Maintenance [production]
04:19 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2154 (T411163 T411164)', diff saved to https://phabricator.wikimedia.org/P87851 and previous config saved to /var/cache/conftool/dbconfig/20260122-041931-marostegui.json [production]
04:09 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2154', diff saved to https://phabricator.wikimedia.org/P87850 and previous config saved to /var/cache/conftool/dbconfig/20260122-040922-marostegui.json [production]
03:59 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2154', diff saved to https://phabricator.wikimedia.org/P87849 and previous config saved to /var/cache/conftool/dbconfig/20260122-035914-marostegui.json [production]
03:49 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2154 (T411163 T411164)', diff saved to https://phabricator.wikimedia.org/P87848 and previous config saved to /var/cache/conftool/dbconfig/20260122-034905-marostegui.json [production]
03:43 <marostegui@cumin1003> dbctl commit (dc=all): 'Depooling db1172 (T411163 T411164)', diff saved to https://phabricator.wikimedia.org/P87847 and previous config saved to /var/cache/conftool/dbconfig/20260122-034302-marostegui.json [production]
03:42 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3 days, 0:00:00 on db1172.eqiad.wmnet with reason: Maintenance [production]
01:58 <larssandergreen> civicrm upgraded from 02a558a2 to 24bca0d2 [production]
01:47 <pt1979@cumin1003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mwlog1003.eqiad.wmnet with OS bookworm [production]
01:47 <pt1979@cumin1003> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - pt1979@cumin1003" [production]
01:46 <pt1979@cumin1003> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - pt1979@cumin1003" [production]
01:27 <pt1979@cumin1003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mwlog1003.eqiad.wmnet with reason: host reimage [production]
01:23 <pt1979@cumin1003> START - Cookbook sre.hosts.downtime for 2:00:00 on mwlog1003.eqiad.wmnet with reason: host reimage [production]
01:07 <zabe@deploy2002> Finished scap sync-world: Backport for [[gerrit:1229607|Start reading from il_target_id on small wikis (T413669)]] (duration: 07m 12s) [production]
01:02 <zabe@deploy2002> zabe: Continuing with sync [production]
01:02 <zabe@deploy2002> zabe: Backport for [[gerrit:1229607|Start reading from il_target_id on small wikis (T413669)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
00:59 <zabe@deploy2002> Started scap sync-world: Backport for [[gerrit:1229607|Start reading from il_target_id on small wikis (T413669)]] [production]
2026-01-21 §
23:40 <eileen> config revision changed from 5a1bd9b8 to 8d6fc61d [production]
23:00 <eileen> config revision changed from 485f8e69 to 5a1bd9b8 [production]
21:14 <catrope@deploy2002> Finished scap sync-world: Backport for [[gerrit:1229219|Enable OATHAuth passkey features in production (T415146)]], [[gerrit:1229680|Don't directly serialize PublicKeyCredential objects]], [[gerrit:1229679|Don't directly serialize PublicKeyCredential objects]] (duration: 09m 42s) [production]