51-100 of 10000 results (95ms)
2026-01-08 ยง
20:06 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1230', diff saved to https://phabricator.wikimedia.org/P86892 and previous config saved to /var/cache/conftool/dbconfig/20260108-200635-marostegui.json [production]
20:04 <zabe@deploy2002> zabe: Continuing with sync [production]
19:56 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1230 (T413525)', diff saved to https://phabricator.wikimedia.org/P86891 and previous config saved to /var/cache/conftool/dbconfig/20260108-195627-marostegui.json [production]
19:55 <zabe@deploy2002> zabe: Backport for [[gerrit:1224679|MimeAnalyzer: Fix syntax error in MAJOR_MIME_TYPES array (T414077)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
19:53 <zabe@deploy2002> Started scap sync-world: Backport for [[gerrit:1224679|MimeAnalyzer: Fix syntax error in MAJOR_MIME_TYPES array (T414077)]] [production]
19:50 <zabe@deploy2002> Finished scap sync-world: Backport for [[gerrit:1224781|fixed typo of word initial in the WmfConfig.php file (T201491)]], [[gerrit:1223289|Enable phan on more php files]] (duration: 08m 09s) [production]
19:47 <marostegui@cumin1003> dbctl commit (dc=all): 'Depooling db1230 (T413525)', diff saved to https://phabricator.wikimedia.org/P86889 and previous config saved to /var/cache/conftool/dbconfig/20260108-194731-marostegui.json [production]
19:47 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1230.eqiad.wmnet with reason: Maintenance [production]
19:47 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1216.eqiad.wmnet with reason: Maintenance [production]
19:46 <zabe@deploy2002> zabe, ebenezerrao: Continuing with sync [production]
19:46 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1207 (T413525)', diff saved to https://phabricator.wikimedia.org/P86888 and previous config saved to /var/cache/conftool/dbconfig/20260108-194649-marostegui.json [production]
19:44 <zabe@deploy2002> zabe, ebenezerrao: Backport for [[gerrit:1224781|fixed typo of word initial in the WmfConfig.php file (T201491)]], [[gerrit:1223289|Enable phan on more php files]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
19:42 <zabe@deploy2002> Started scap sync-world: Backport for [[gerrit:1224781|fixed typo of word initial in the WmfConfig.php file (T201491)]], [[gerrit:1223289|Enable phan on more php files]] [production]
19:42 <andrew@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudbackup1003.eqiad.wmnet with OS trixie [production]
19:42 <andrew@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudbackup2003.codfw.wmnet with OS trixie [production]
19:36 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1207', diff saved to https://phabricator.wikimedia.org/P86887 and previous config saved to /var/cache/conftool/dbconfig/20260108-193641-marostegui.json [production]
19:26 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1207', diff saved to https://phabricator.wikimedia.org/P86886 and previous config saved to /var/cache/conftool/dbconfig/20260108-192633-marostegui.json [production]
19:24 <dduvall@deploy2002> rebuilt and synchronized wikiversions files: group2 to 1.46.0-wmf.10 refs T408280 [production]
19:16 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1207 (T413525)', diff saved to https://phabricator.wikimedia.org/P86885 and previous config saved to /var/cache/conftool/dbconfig/20260108-191624-marostegui.json [production]
19:07 <marostegui@cumin1003> dbctl commit (dc=all): 'Depooling db1207 (T413525)', diff saved to https://phabricator.wikimedia.org/P86884 and previous config saved to /var/cache/conftool/dbconfig/20260108-190727-marostegui.json [production]
19:07 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1207.eqiad.wmnet with reason: Maintenance [production]
19:07 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1200 (T413525)', diff saved to https://phabricator.wikimedia.org/P86883 and previous config saved to /var/cache/conftool/dbconfig/20260108-190702-marostegui.json [production]
18:57 <eileen> civicrm upgraded from b454e4ba to 2689332e [production]
18:56 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1200', diff saved to https://phabricator.wikimedia.org/P86882 and previous config saved to /var/cache/conftool/dbconfig/20260108-185654-marostegui.json [production]
18:46 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1200', diff saved to https://phabricator.wikimedia.org/P86881 and previous config saved to /var/cache/conftool/dbconfig/20260108-184645-marostegui.json [production]
18:36 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1200 (T413525)', diff saved to https://phabricator.wikimedia.org/P86880 and previous config saved to /var/cache/conftool/dbconfig/20260108-183637-marostegui.json [production]
18:26 <marostegui@cumin1003> dbctl commit (dc=all): 'Depooling db1200 (T413525)', diff saved to https://phabricator.wikimedia.org/P86879 and previous config saved to /var/cache/conftool/dbconfig/20260108-182641-marostegui.json [production]
18:26 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1200.eqiad.wmnet with reason: Maintenance [production]
18:26 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1185 (T413525)', diff saved to https://phabricator.wikimedia.org/P86878 and previous config saved to /var/cache/conftool/dbconfig/20260108-182627-marostegui.json [production]
18:16 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1185', diff saved to https://phabricator.wikimedia.org/P86877 and previous config saved to /var/cache/conftool/dbconfig/20260108-181619-marostegui.json [production]
18:15 <bd808@deploy2002> helmfile [codfw] DONE helmfile.d/services/developer-portal: apply [production]
18:14 <bd808@deploy2002> helmfile [codfw] START helmfile.d/services/developer-portal: apply [production]
18:12 <bd808@deploy2002> helmfile [eqiad] DONE helmfile.d/services/developer-portal: apply [production]
18:12 <bd808@deploy2002> helmfile [eqiad] START helmfile.d/services/developer-portal: apply [production]
18:11 <bd808@deploy2002> helmfile [staging] DONE helmfile.d/services/developer-portal: apply [production]
18:11 <bd808@deploy2002> helmfile [staging] START helmfile.d/services/developer-portal: apply [production]
18:06 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1185', diff saved to https://phabricator.wikimedia.org/P86876 and previous config saved to /var/cache/conftool/dbconfig/20260108-180611-marostegui.json [production]
17:56 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1185 (T413525)', diff saved to https://phabricator.wikimedia.org/P86875 and previous config saved to /var/cache/conftool/dbconfig/20260108-175602-marostegui.json [production]
17:50 <ladsgroup@deploy2002> Finished scap sync-world: Backport for [[gerrit:1224071|Only generate 120,250 thumbs (temporary) (T408062 T412971)]] (duration: 17m 34s) [production]
17:46 <ladsgroup@deploy2002> mvernon, ladsgroup: Continuing with sync [production]
17:46 <marostegui@cumin1003> dbctl commit (dc=all): 'Depooling db1185 (T413525)', diff saved to https://phabricator.wikimedia.org/P86874 and previous config saved to /var/cache/conftool/dbconfig/20260108-174606-marostegui.json [production]
17:45 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1185.eqiad.wmnet with reason: Maintenance [production]
17:45 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1161 (T413525)', diff saved to https://phabricator.wikimedia.org/P86873 and previous config saved to /var/cache/conftool/dbconfig/20260108-174542-marostegui.json [production]
17:35 <ladsgroup@deploy2002> mvernon, ladsgroup: Backport for [[gerrit:1224071|Only generate 120,250 thumbs (temporary) (T408062 T412971)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
17:35 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1161', diff saved to https://phabricator.wikimedia.org/P86872 and previous config saved to /var/cache/conftool/dbconfig/20260108-173534-marostegui.json [production]
17:32 <ladsgroup@deploy2002> Started scap sync-world: Backport for [[gerrit:1224071|Only generate 120,250 thumbs (temporary) (T408062 T412971)]] [production]
17:32 <andrew@cumin2002> START - Cookbook sre.hosts.reimage for host cloudbackup2003.codfw.wmnet with OS trixie [production]
17:31 <andrew@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudbackup2003.codfw.wmnet with OS trixie [production]
17:31 <jynus> restarted wmf_auto_restart_prometheus-mysqld-exporter.service @ db2231 [production]
17:25 <marostegui@cumin1003> dbctl commit (dc=all): 'Depooling db2155 (T411163 T411164)', diff saved to https://phabricator.wikimedia.org/P86871 and previous config saved to /var/cache/conftool/dbconfig/20260108-172551-marostegui.json [production]