151-200 of 10000 results (42ms)
2026-06-29 ยง
14:07 <cwilliams@cumin1003> dbmaint on s4@eqiad T429893 [production]
14:07 <cwilliams@cumin1003> START - Cookbook sre.mysql.major-upgrade [production]
14:05 <aikochou@deploy1003> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'articletopic-outlink' for release 'main' . [production]
14:04 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component builds-api [toolsbeta]
13:57 <ozge@deploy1003> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-editquality-reverted' for release 'main' . [production]
13:55 <aikochou@deploy1003> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'articletopic-outlink' for release 'main' . [production]
13:54 <ozge@deploy1003> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-editquality-goodfaith' for release 'main' . [production]
13:52 <ozge@deploy1003> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-editquality-damaging' for release 'main' . [production]
13:52 <dcaro@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component builds-api [toolsbeta]
13:51 <ozge@deploy1003> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-drafttopic' for release 'main' . [production]
13:49 <ozge@deploy1003> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-draftquality' for release 'main' . [production]
13:49 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component builds-api [toolsbeta]
13:49 <Tran> Deployed patch for T427287 [production]
13:43 <dcaro@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api [tools]
13:39 <aikochou@deploy1003> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'article-models' for release 'main' . [production]
13:33 <aikochou@deploy1003> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'article-models' for release 'main' . [production]
13:31 <stran@deploy1003> Scap cancelled without rolling back. [production]
13:31 <ozge@deploy1003> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-articletopic' for release 'main' . [production]
13:30 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component jobs-api [tools]
13:29 <dcaro@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api [toolsbeta]
13:27 <ozge@deploy1003> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-articlequality' for release 'main' . [production]
13:22 <cwilliams@cumin1003> END (PASS) - Cookbook sre.mysql.major-upgrade (exit_code=0) [production]
13:22 <cwilliams@cumin1003> END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1252: Migration of db1252.eqiad.wmnet completed [production]
13:17 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component jobs-api [toolsbeta]
13:17 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) on deployment codfw1dev for all services [admin]
13:15 <stran@deploy1003> stran, vadymts1: Continuing with deployment [production]
13:15 <andrewbogott> rm -rf /srv/paws/files-to-remove/* on paws-nfs-2 [paws]
13:14 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.restart_openstack on deployment codfw1dev for all services [admin]
13:13 <stran@deploy1003> stran, vadymts1: Backport for [[gerrit:1306056|User groups changes for English Wikiversity (T430416)]], [[gerrit:1305811|hrwiki: Add to wgCiteResponsiveReferences (T430182)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
13:11 <stran@deploy1003> Started scap sync-world: Backport for [[gerrit:1306056|User groups changes for English Wikiversity (T430416)]], [[gerrit:1305811|hrwiki: Add to wgCiteResponsiveReferences (T430182)]] [production]
13:10 <dcaro@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api [tools]
13:05 <dpogorzelski@deploy1003> helmfile [codfw] DONE helmfile.d/services/rest-gateway: sync [production]
13:05 <dpogorzelski@deploy1003> helmfile [codfw] START helmfile.d/services/rest-gateway: sync [production]
13:04 <dpogorzelski@deploy1003> helmfile [eqiad] DONE helmfile.d/services/rest-gateway: sync [production]
13:04 <dpogorzelski@deploy1003> helmfile [eqiad] START helmfile.d/services/rest-gateway: sync [production]
12:57 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component jobs-api [tools]
12:56 <dcaro@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component jobs-api [toolsbeta]
12:46 <atsuko@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-content-history-reconcile-enrich: apply [production]
12:46 <atsuko@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-content-history-reconcile-enrich: apply [production]
12:44 <atsuko@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-growthbook: apply [production]
12:44 <atsuko@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-growthbook: apply [production]
12:44 <dcaro@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component jobs-api [toolsbeta]
12:41 <atsuko@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/postgresql-growthbook-next: apply [production]
12:41 <atsuko@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/postgresql-growthbook-next: apply [production]
12:37 <cwilliams@cumin1003> START - Cookbook sre.mysql.pool pool db1252: Migration of db1252.eqiad.wmnet completed [production]
12:35 <atsuko@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/spark-history: apply [production]
12:34 <atsuko@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/spark-history: apply [production]
12:34 <atsuko@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/datasets-config: apply [production]
12:34 <atsuko@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/datasets-config: apply [production]
12:33 <dbrant@deploy1003> helmfile [codfw] DONE helmfile.d/services/wikifeeds: apply [production]