101-150 of 10000 results (59ms)
2024-03-01 ยง
18:50 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1186.eqiad.wmnet with reason: Maintenance [production]
18:50 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 12:00:00 on db1186.eqiad.wmnet with reason: Maintenance [production]
18:12 <taavi@cumin1002> dbctl commit (dc=all): 'depool db1169 T358892', diff saved to https://phabricator.wikimedia.org/P58287 and previous config saved to /var/cache/conftool/dbconfig/20240301-181221-taavi.json [production]
17:58 <dancy@deploy2002> Finished deploy [cassandra/logstash-logback-encoder@162f72f]: (no justification provided) (duration: 00m 08s) [production]
17:58 <dancy@deploy2002> Started deploy [cassandra/logstash-logback-encoder@162f72f]: (no justification provided) [production]
16:54 <claime> Pooled and uncordoned mw1384.eqiad.wmnet mw1432.eqiad.wmnet mw1433.eqiad.wmnet - T351074 [production]
16:52 <cgoubert@cumin2002> conftool action : set/weight=10:pooled=yes; selector: name=(mw1384.eqiad.wmnet|mw1432.eqiad.wmnet|mw1433.eqiad.wmnet),cluster=kubernetes,service=kubesvc [production]
16:46 <claime> Running homer 'cr*eqiad*' commit 'T351074' [production]
16:46 <cgoubert@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mw1384.eqiad.wmnet with OS bullseye [production]
16:43 <cgoubert@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mw1432.eqiad.wmnet with OS bullseye [production]
16:40 <cgoubert@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mw1433.eqiad.wmnet with OS bullseye [production]
16:27 <cgoubert@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw1384.eqiad.wmnet with reason: host reimage [production]
16:24 <cgoubert@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw1432.eqiad.wmnet with reason: host reimage [production]
16:22 <cgoubert@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mw1433.eqiad.wmnet with reason: host reimage [production]
16:20 <cgoubert@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on mw1384.eqiad.wmnet with reason: host reimage [production]
16:20 <cgoubert@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on mw1432.eqiad.wmnet with reason: host reimage [production]
16:19 <cgoubert@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on mw1433.eqiad.wmnet with reason: host reimage [production]
16:17 <elukey@deploy2002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-editquality-damaging' for release 'main' . [production]
16:16 <elukey@deploy2002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-editquality-reverted' for release 'main' . [production]
16:16 <elukey@deploy2002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-editquality-goodfaith' for release 'main' . [production]
16:16 <elukey@deploy2002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-drafttopic' for release 'main' . [production]
16:15 <elukey@deploy2002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-draftquality' for release 'main' . [production]
16:15 <elukey@deploy2002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-articletopic' for release 'main' . [production]
16:07 <cgoubert@cumin2002> START - Cookbook sre.hosts.reimage for host mw1432.eqiad.wmnet with OS bullseye [production]
16:06 <cgoubert@cumin2002> START - Cookbook sre.hosts.reimage for host mw1384.eqiad.wmnet with OS bullseye [production]
16:06 <cgoubert@cumin2002> START - Cookbook sre.hosts.reimage for host mw1433.eqiad.wmnet with OS bullseye [production]
16:05 <dancy@deploy2002> Finished deploy [analytics/refinery@6e8f25b]: (no justification provided) (duration: 00m 03s) [production]
16:05 <dancy@deploy2002> Started deploy [analytics/refinery@6e8f25b]: (no justification provided) [production]
16:04 <elukey@deploy2002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-articlequality' for release 'main' . [production]
16:03 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4 days, 0:00:00 on db2117.codfw.wmnet with reason: Silence for maintenance [production]
16:03 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 4 days, 0:00:00 on db2117.codfw.wmnet with reason: Silence for maintenance [production]
15:57 <claime> Depooling mw1384.eqiad.wmnet,mw1432.eqiad.wmnet,mw1433.eqiad.wmnet for move to k8s - T351074 [production]
15:51 <elukey@deploy2002> helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. [production]
15:51 <elukey@deploy2002> helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. [production]
14:57 <elukey@deploy2002> helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. [production]
14:57 <elukey@deploy2002> helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. [production]
14:54 <jiji@deploy2002> helmfile [staging] DONE helmfile.d/services/mw-mcrouter: apply [production]
14:53 <jiji@deploy2002> helmfile [staging] START helmfile.d/services/mw-mcrouter: apply [production]
14:52 <jiji@deploy2002> helmfile [staging] DONE helmfile.d/services/mw-mcrouter: apply [production]
14:52 <jiji@deploy2002> helmfile [staging] START helmfile.d/services/mw-mcrouter: apply [production]
14:52 <jiji@deploy2002> helmfile [staging] DONE helmfile.d/services/mw-mcrouter: apply [production]
14:52 <jiji@deploy2002> helmfile [staging] START helmfile.d/services/mw-mcrouter: apply [production]
14:28 <jnuche@deploy2002> Finished deploy [releng/jenkins-deploy@4421d2c] (releasing): (no justification provided) (duration: 00m 38s) [production]
14:28 <cgoubert@deploy2002> helmfile [eqiad] DONE helmfile.d/services/mw-api-int: apply [production]
14:28 <jnuche@deploy2002> Started deploy [releng/jenkins-deploy@4421d2c] (releasing): (no justification provided) [production]
14:28 <cgoubert@deploy2002> helmfile [eqiad] START helmfile.d/services/mw-api-int: apply [production]
14:08 <claime> Pooled and uncordoned mw1387.eqiad.wmnet mw1389.eqiad.wmnet mw1391.eqiad.wmnet mw1393.eqiad.wmnet mw1395.eqiad.wmnet mw1397.eqiad.wmnet - T351074 [production]
14:08 <cgoubert@cumin2002> conftool action : set/weight=10:pooled=yes; selector: name=(mw1387.eqiad.wmnet|mw1389.eqiad.wmnet|mw1391.eqiad.wmnet|mw1393.eqiad.wmnet|mw1395.eqiad.wmnet|mw1397.eqiad.wmnet),cluster=kubernetes,service=kubesvc [production]
14:00 <claime> Running homer 'cr*eqiad*' commit 'T351074' [production]
13:59 <cgoubert@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mw1393.eqiad.wmnet with OS bullseye [production]