4401-4450 of 10000 results (90ms)
2023-07-18 ยง
11:20 <btullis@cumin1001> START - Cookbook sre.hosts.reimage for host analytics1074.eqiad.wmnet with OS bullseye [production]
11:20 <btullis@cumin1001> START - Cookbook sre.hosts.reimage for host analytics1073.eqiad.wmnet with OS bullseye [production]
11:15 <mvernon@cumin2002> START - Cookbook sre.hosts.reboot-single for host thanos-be2004.codfw.wmnet [production]
11:15 <mvernon@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host thanos-be2003.codfw.wmnet [production]
11:13 <btullis@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host analytics1071.eqiad.wmnet with OS bullseye [production]
11:06 <mvernon@cumin2002> START - Cookbook sre.hosts.reboot-single for host thanos-be2003.codfw.wmnet [production]
11:03 <mvernon@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host thanos-be2002.codfw.wmnet [production]
10:57 <isaranto@deploy1002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-editquality-reverted' for release 'main' . [production]
10:56 <isaranto@deploy1002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-editquality-goodfaith' for release 'main' . [production]
10:55 <isaranto@deploy1002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-editquality-damaging' for release 'main' . [production]
10:55 <isaranto@deploy1002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-drafttopic' for release 'main' . [production]
10:55 <isaranto@deploy1002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-draftquality' for release 'main' . [production]
10:55 <isaranto@deploy1002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-articletopic' for release 'main' . [production]
10:54 <isaranto@deploy1002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-articlequality' for release 'main' . [production]
10:49 <mvernon@cumin2002> START - Cookbook sre.hosts.reboot-single for host thanos-be2002.codfw.wmnet [production]
10:48 <mvernon@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host thanos-be2001.codfw.wmnet [production]
10:42 <topranks> repool esams after successful move of cr3-knams to new rack T337997 [production]
10:41 <isaranto@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-editquality-reverted' for release 'main' . [production]
10:40 <isaranto@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-editquality-goodfaith' for release 'main' . [production]
10:39 <isaranto@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-editquality-damaging' for release 'main' . [production]
10:39 <isaranto@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-drafttopic' for release 'main' . [production]
10:38 <isaranto@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-draftquality' for release 'main' . [production]
10:38 <isaranto@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-articletopic' for release 'main' . [production]
10:38 <isaranto@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-articlequality' for release 'main' . [production]
10:35 <mvernon@cumin2002> START - Cookbook sre.hosts.reboot-single for host thanos-be2001.codfw.wmnet [production]
10:32 <mvernon@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host thanos-be1004.eqiad.wmnet [production]
10:24 <mvernon@cumin2002> START - Cookbook sre.hosts.reboot-single for host thanos-be1004.eqiad.wmnet [production]
10:24 <mvernon@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host thanos-be1003.eqiad.wmnet [production]
10:24 <cmooney@cumin1001> END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for cr3-knams,cr3-knams IPv6 [production]
10:24 <cmooney@cumin1001> START - Cookbook sre.hosts.remove-downtime for cr3-knams,cr3-knams IPv6 [production]
10:20 <btullis@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on analytics1071.eqiad.wmnet with reason: host reimage [production]
10:17 <btullis@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on analytics1071.eqiad.wmnet with reason: host reimage [production]
10:11 <mvernon@cumin2002> START - Cookbook sre.hosts.reboot-single for host thanos-be1003.eqiad.wmnet [production]
10:10 <mvernon@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host thanos-be1002.eqiad.wmnet [production]
10:05 <mvernon@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host dborch1001.wikimedia.org [production]
10:02 <fabfur> fix last entry: correct CR is https://gerrit.wikimedia.org/r/939242 [production]
10:02 <mvernon@cumin2002> START - Cookbook sre.hosts.reboot-single for host thanos-be1002.eqiad.wmnet [production]
10:02 <fabfur> enable puppet on A:cp-esams for https://gerrit.wikimedia.org/r/939235 (T340983) (hosts will run puppet with the usual schedule) [production]
10:02 <mvernon@cumin1001> START - Cookbook sre.hosts.reboot-single for host dborch1001.wikimedia.org [production]
10:00 <btullis@cumin1001> START - Cookbook sre.hosts.reimage for host analytics1071.eqiad.wmnet with OS bullseye [production]
09:52 <fabfur> disable puppet on A:cp-esams to apply https://gerrit.wikimedia.org/r/c/operations/puppet/+/939242 (T340983) [production]
09:51 <arturo> deploying https://gerrit.wikimedia.org/r/c/operations/homer/public/+/938819 via homer to cr-eqiad & cr-codfw [production]
09:34 <mvernon@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1075.eqiad.wmnet [production]
09:30 <isaranto@deploy1002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-editquality-reverted' for release 'main' . [production]
09:30 <isaranto@deploy1002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-editquality-goodfaith' for release 'main' . [production]
09:30 <isaranto@deploy1002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-editquality-damaging' for release 'main' . [production]
09:30 <isaranto@deploy1002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-drafttopic' for release 'main' . [production]
09:30 <isaranto@deploy1002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-draftquality' for release 'main' . [production]
09:30 <isaranto@deploy1002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-articletopic' for release 'main' . [production]
09:29 <isaranto@deploy1002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-articlequality' for release 'main' . [production]