301-350 of 10000 results (31ms)
2026-01-12 ยง
15:51 <marostegui@cumin1003> dbctl commit (dc=all): 'Depooling db1206 (T413525)', diff saved to https://phabricator.wikimedia.org/P87375 and previous config saved to /var/cache/conftool/dbconfig/20260112-155113-marostegui.json [production]
15:51 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1206.eqiad.wmnet with reason: Maintenance [production]
15:50 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1196 (T413525)', diff saved to https://phabricator.wikimedia.org/P87374 and previous config saved to /var/cache/conftool/dbconfig/20260112-155049-marostegui.json [production]
15:40 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1196', diff saved to https://phabricator.wikimedia.org/P87373 and previous config saved to /var/cache/conftool/dbconfig/20260112-154041-marostegui.json [production]
15:40 <ejegg> donorwiki upgraded from 02dbb1e2 to 3ffc70f0 [production]
15:39 <elukey@cumin1003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wdqs1031.eqiad.wmnet with reason: host reimage [production]
15:35 <elukey@cumin1003> START - Cookbook sre.hosts.downtime for 2:00:00 on wdqs1031.eqiad.wmnet with reason: host reimage [production]
15:35 <andrew@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudlb2004-dev.codfw.wmnet with OS trixie [production]
15:33 <zabe@deploy2002> Finished scap sync-world: Backport for [[gerrit:1225088|Disable updates for Special:GloballyUnusedFiles (T414202)]], [[gerrit:1225118|Stop updating Deadendpages and Lonelypages on commons (T371662)]], [[gerrit:1225587|Correctly disable Special:Deadendpages on commons (T371662)]] (duration: 06m 47s) [production]
15:30 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1196', diff saved to https://phabricator.wikimedia.org/P87371 and previous config saved to /var/cache/conftool/dbconfig/20260112-153033-marostegui.json [production]
15:29 <zabe@deploy2002> zabe: Continuing with sync [production]
15:28 <zabe@deploy2002> zabe: Backport for [[gerrit:1225088|Disable updates for Special:GloballyUnusedFiles (T414202)]], [[gerrit:1225118|Stop updating Deadendpages and Lonelypages on commons (T371662)]], [[gerrit:1225587|Correctly disable Special:Deadendpages on commons (T371662)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
15:26 <zabe@deploy2002> Started scap sync-world: Backport for [[gerrit:1225088|Disable updates for Special:GloballyUnusedFiles (T414202)]], [[gerrit:1225118|Stop updating Deadendpages and Lonelypages on commons (T371662)]], [[gerrit:1225587|Correctly disable Special:Deadendpages on commons (T371662)]] [production]
15:20 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1196 (T413525)', diff saved to https://phabricator.wikimedia.org/P87370 and previous config saved to /var/cache/conftool/dbconfig/20260112-152025-marostegui.json [production]
15:20 <zabe@deploy2002> Sync cancelled. [production]
15:19 <btullis@cumin1003> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host an-test-druid1001.eqiad.wmnet with OS bookworm [production]
15:19 <marostegui@cumin1003> dbctl commit (dc=all): 'Depooling db2173 (T413525)', diff saved to https://phabricator.wikimedia.org/P87369 and previous config saved to /var/cache/conftool/dbconfig/20260112-151934-marostegui.json [production]
15:19 <btullis@cumin1003> START - Cookbook sre.hosts.reimage for host an-test-druid1001.eqiad.wmnet with OS bookworm [production]
15:19 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2173.codfw.wmnet with reason: Maintenance [production]
15:19 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2170 (T413525)', diff saved to https://phabricator.wikimedia.org/P87368 and previous config saved to /var/cache/conftool/dbconfig/20260112-151908-marostegui.json [production]
15:17 <elukey@cumin1003> START - Cookbook sre.hosts.reimage for host wdqs1031.eqiad.wmnet with OS trixie [production]
15:16 <btullis@cumin1003> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host an-test-druid1001.eqiad.wmnet with OS bookworm [production]
15:16 <btullis@cumin1003> START - Cookbook sre.hosts.reimage for host an-test-druid1001.eqiad.wmnet with OS bookworm [production]
15:14 <elukey@cumin1003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wdqs1030.eqiad.wmnet with OS trixie [production]
15:13 <andrew@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudlb2004-dev.codfw.wmnet with reason: host reimage [production]
15:12 <zabe@deploy2002> mwscript-k8s job started: foreachwikiindblist all refreshImageMetadata.php --mediatype AUDIO --mime unknown/mpeg --oldimage --force # T414259 [production]
15:10 <zabe@deploy2002> zabe: Backport for [[gerrit:1225088|Disable updates for Special:GloballyUnusedFiles (T414202)]], [[gerrit:1225118|Stop updating Deadendpages and Lonelypages on commons (T371662)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
15:09 <andrew@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on cloudlb2004-dev.codfw.wmnet with reason: host reimage [production]
15:09 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2170', diff saved to https://phabricator.wikimedia.org/P87367 and previous config saved to /var/cache/conftool/dbconfig/20260112-150900-marostegui.json [production]
15:08 <zabe@deploy2002> Started scap sync-world: Backport for [[gerrit:1225088|Disable updates for Special:GloballyUnusedFiles (T414202)]], [[gerrit:1225118|Stop updating Deadendpages and Lonelypages on commons (T371662)]] [production]
15:06 <ejegg> fundraising civicrm upgraded from 197629c6 to 158b60a0 [production]
15:06 <zabe@deploy2002> Finished scap sync-world: Backport for [[gerrit:1225165|Stop setting $wgBlockTargetMigrationStage (T355034)]] (duration: 06m 43s) [production]
15:05 <zabe@deploy2002> mwscript-k8s job started: foreachwikiindblist all refreshImageMetadata.php --mediatype AUDIO --mime unknown/mpeg --force # T414259 [production]
15:02 <zabe@deploy2002> zabe: Continuing with sync [production]
15:01 <zabe@deploy2002> zabe: Backport for [[gerrit:1225165|Stop setting $wgBlockTargetMigrationStage (T355034)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
14:59 <elukey@cumin1003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wdqs1030.eqiad.wmnet with reason: host reimage [production]
14:59 <zabe@deploy2002> Started scap sync-world: Backport for [[gerrit:1225165|Stop setting $wgBlockTargetMigrationStage (T355034)]] [production]
14:58 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2170', diff saved to https://phabricator.wikimedia.org/P87366 and previous config saved to /var/cache/conftool/dbconfig/20260112-145852-marostegui.json [production]
14:52 <zabe@deploy2002> mwscript-k8s job started: foreachwikiindblist all refreshImageMetadata.php --mediatype AUDIO --mime unknown/flac --oldimage --force # T414259 [production]
14:51 <elukey@cumin1003> START - Cookbook sre.hosts.downtime for 2:00:00 on wdqs1030.eqiad.wmnet with reason: host reimage [production]
14:51 <zabe@deploy2002> mwscript-k8s job started: foreachwikiindblist all refreshImageMetadata.php --mediatype AUDIO --mime unknown/flac --oldimages --force # T414259 [production]
14:50 <zabe> empty machinevision-tester group on commons # T372004 [production]
14:50 <zabe> empty mediasearch-tester group on commons # T372004 [production]
14:49 <andrew@cumin2002> START - Cookbook sre.hosts.reimage for host cloudlb2004-dev.codfw.wmnet with OS trixie [production]
14:48 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db2170 (T413525)', diff saved to https://phabricator.wikimedia.org/P87365 and previous config saved to /var/cache/conftool/dbconfig/20260112-144843-marostegui.json [production]
14:46 <marostegui@cumin1003> dbctl commit (dc=all): 'Depooling db1196 (T413525)', diff saved to https://phabricator.wikimedia.org/P87364 and previous config saved to /var/cache/conftool/dbconfig/20260112-144602-marostegui.json [production]
14:45 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1013,1017].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance [production]
14:45 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1196.eqiad.wmnet with reason: Maintenance [production]
14:45 <marostegui@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1195 (T413525)', diff saved to https://phabricator.wikimedia.org/P87363 and previous config saved to /var/cache/conftool/dbconfig/20260112-144518-marostegui.json [production]
14:43 <zabe@deploy2002> mwscript-k8s job started: foreachwikiindblist all refreshImageMetadata.php --mediatype AUDIO --mime unknown/flac --force # T414259 [production]