2401-2450 of 10000 results (31ms)
2024-11-04 ยง
12:41 <raymond-ndibe@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-10 from 1.27.16 to 1.28.14 (T362867) [tools]
12:41 <raymond-ndibe@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-nfs-1 from 1.27.16 to 1.28.14 (T362867) [tools]
12:40 <raymond-ndibe@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-nfs-1 from 1.27.16 to 1.28.14 (T362867) [tools]
12:40 <raymond-ndibe@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-108 from 1.27.16 to 1.28.14 (T362867) [tools]
12:39 <raymond-ndibe@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-108 from 1.27.16 to 1.28.14 (T362867) [tools]
12:39 <raymond-ndibe@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-107 from 1.27.16 to 1.28.14 (T362867) [tools]
12:38 <raymond-ndibe@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-107 from 1.27.16 to 1.28.14 (T362867) [tools]
12:38 <raymond-ndibe@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-106 from 1.27.16 to 1.28.14 (T362867) [tools]
12:38 <sstefanova@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component builds-api [toolsbeta]
12:36 <raymond-ndibe@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-106 from 1.27.16 to 1.28.14 (T362867) [tools]
12:36 <raymond-ndibe@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-105 from 1.27.16 to 1.28.14 (T362867) [tools]
12:35 <raymond-ndibe@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-105 from 1.27.16 to 1.28.14 (T362867) [tools]
12:35 <raymond-ndibe@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-103 from 1.27.16 to 1.28.14 (T362867) [tools]
12:35 <elukey@cumin1002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti1052.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
12:34 <isaranto@deploy2002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-articlequality' for release 'main' . [production]
12:34 <raymond-ndibe@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-103 from 1.27.16 to 1.28.14 (T362867) [tools]
12:34 <raymond-ndibe@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-worker-102 from 1.27.16 to 1.28.14 (T362867) [tools]
12:33 <raymond-ndibe@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-worker-102 from 1.27.16 to 1.28.14 (T362867) [tools]
12:24 <elukey@cumin1002> START - Cookbook sre.hosts.provision for host ganeti1052.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
12:22 <raymond-ndibe@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-control-9 from 1.27.16 to 1.28.14 (T362867) [tools]
12:22 <isaranto@deploy2002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-draftquality' for release 'main' . [production]
12:22 <sstefanova@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component envvars-api [tools]
12:22 <isaranto@deploy2002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-drafttopic' for release 'main' . [production]
12:20 <isaranto@deploy2002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-editquality-reverted' for release 'main' . [production]
12:19 <isaranto@deploy2002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-editquality-goodfaith' for release 'main' . [production]
12:19 <isaranto@deploy2002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-editquality-damaging' for release 'main' . [production]
12:16 <sstefanova@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component envvars-api [tools]
12:13 <raymond-ndibe@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-control-9 from 1.27.16 to 1.28.14 (T362867) [tools]
12:11 <sstefanova@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component envvars-api [tools]
12:11 <jmm@cumin2002> END (FAIL) - Cookbook sre.ganeti.addnode (exit_code=99) for new host ganeti1039.eqiad.wmnet to cluster eqiad and group B [production]
12:11 <jmm@cumin2002> START - Cookbook sre.ganeti.addnode for new host ganeti1039.eqiad.wmnet to cluster eqiad and group B [production]
12:10 <isaranto@deploy2002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-articletopic' for release 'main' . [production]
12:08 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti1039.eqiad.wmnet [production]
12:08 <elukey@cumin1002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti1051.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
12:06 <sstefanova@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component envvars-api [tools]
12:03 <raymond-ndibe@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-control-8 from 1.27.16 to 1.28.14 (T362867) [tools]
12:01 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host ganeti1039.eqiad.wmnet [production]
11:59 <sstefanova@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component envvars-api [tools]
11:58 <elukey@cumin1002> START - Cookbook sre.hosts.provision for host ganeti1051.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
11:58 <sstefanova@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component envvars-api [tools]
11:57 <raymond-ndibe@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-control-8 from 1.27.16 to 1.28.14 (T362867) [tools]
11:56 <elukey@cumin1002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti1050.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
11:55 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2227 (T376905)', diff saved to https://phabricator.wikimedia.org/P70859 and previous config saved to /var/cache/conftool/dbconfig/20241104-115514-ladsgroup.json [production]
11:51 <sstefanova@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.component.deploy (exit_code=0) for component envvars-api [toolsbeta]
11:49 <raymond-ndibe@cloudcumin1001> END (PASS) - Cookbook wmcs.toolforge.k8s.worker.upgrade (exit_code=0) for node tools-k8s-control-7 from 1.27.16 to 1.28.14 (T362867) [tools]
11:45 <elukey@cumin1002> START - Cookbook sre.hosts.provision for host ganeti1050.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
11:45 <sstefanova@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component envvars-api [toolsbeta]
11:44 <elukey@cumin1002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti1049.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
11:42 <hashar> Restarted CI Jenkins to update the Collapsible Sections plugin due to T378864 # T378327 [releng]
11:42 <raymond-ndibe@cloudcumin1001> START - Cookbook wmcs.toolforge.k8s.worker.upgrade for node tools-k8s-control-7 from 1.27.16 to 1.28.14 (T362867) [tools]