2601-2650 of 10000 results (52ms)
2022-05-10 ยง
11:56 <elukey@deploy1002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-editquality-reverted' for release 'main' . [production]
11:55 <elukey@deploy1002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-draftquality' for release 'main' . [production]
11:53 <elukey@deploy1002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-articlequality' for release 'main' . [production]
11:52 <elukey@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-editquality-reverted' for release 'main' . [production]
11:51 <klausman@cumin2002> START - Cookbook sre.hosts.reimage for host ores2004.codfw.wmnet with OS buster [production]
11:50 <elukey@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-draftquality' for release 'main' . [production]
11:48 <elukey@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-articlequality' for release 'main' . [production]
11:16 <btullis> purged bpfcc-tools from kubernetes1013 [production]
11:16 <btullis> purged bpfcc-tools from kubernetes1013 [production]
11:06 <btullis> temporarily install bpfcc-tools on kubernetes1013 [production]
11:05 <btullis> temporarily install bpfcc-tools on kubernetes1013 (T306181) [production]
10:44 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ldap-replica1004.wikimedia.org [production]
10:42 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host ldap-replica1004.wikimedia.org [production]
10:38 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ldap-replica1003.wikimedia.org [production]
10:36 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host ldap-replica1003.wikimedia.org [production]
10:34 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ldap-replica2006.wikimedia.org [production]
10:30 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host ldap-replica2006.wikimedia.org [production]
10:29 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ldap-replica2005.wikimedia.org [production]
10:27 <mvernon@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ms-be2053.codfw.wmnet with OS bullseye [production]
10:26 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host ldap-replica2005.wikimedia.org [production]
10:19 <filippo@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host prometheus1006.eqiad.wmnet [production]
10:17 <filippo@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host prometheus2006.codfw.wmnet [production]
10:11 <filippo@cumin1001> START - Cookbook sre.hosts.reboot-single for host prometheus2006.codfw.wmnet [production]
10:11 <filippo@cumin1001> START - Cookbook sre.hosts.reboot-single for host prometheus1006.eqiad.wmnet [production]
10:07 <filippo@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host prometheus2005.codfw.wmnet [production]
10:05 <filippo@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host prometheus1005.eqiad.wmnet [production]
10:00 <filippo@cumin1001> START - Cookbook sre.hosts.reboot-single for host prometheus2005.codfw.wmnet [production]
10:00 <ayounsi@deploy1002> Finished deploy [netbox/deploy@7bbf659]: Upgrade Netbox-dev2002 to 2.11 (duration: 01m 41s) [production]
09:59 <filippo@cumin1001> START - Cookbook sre.hosts.reboot-single for host prometheus1005.eqiad.wmnet [production]
09:58 <ayounsi@deploy1002> Started deploy [netbox/deploy@7bbf659]: Upgrade Netbox-dev2002 to 2.11 [production]
09:56 <klausman@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ores2003.codfw.wmnet with OS buster [production]
09:53 <klausman@deploy1002> Finished deploy [ores/deploy@98a1b2e]: (no justification provided) (duration: 00m 06s) [production]
09:53 <klausman@deploy1002> Started deploy [ores/deploy@98a1b2e]: (no justification provided) [production]
09:42 <filippo@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host centrallog2002.codfw.wmnet [production]
09:37 <filippo@cumin1001> START - Cookbook sre.hosts.reboot-single for host centrallog2002.codfw.wmnet [production]
09:28 <mvernon@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ms-be2053.codfw.wmnet with reason: host reimage [production]
09:25 <mvernon@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on ms-be2053.codfw.wmnet with reason: host reimage [production]
09:22 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host netflow1002.eqiad.wmnet [production]
09:21 <klausman@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ores2003.codfw.wmnet with reason: host reimage [production]
09:20 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host netflow1002.eqiad.wmnet [production]
09:19 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host netflow2002.codfw.wmnet [production]
09:18 <klausman@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on ores2003.codfw.wmnet with reason: host reimage [production]
09:18 <marostegui@cumin1001> dbctl commit (dc=all): 'Increase traffic on db1172 to test 10.6 T307546', diff saved to https://phabricator.wikimedia.org/P27776 and previous config saved to /var/cache/conftool/dbconfig/20220510-091812-marostegui.json [production]
09:14 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host netflow2002.codfw.wmnet [production]
09:14 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host rpki1001.eqiad.wmnet [production]
09:12 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host rpki1001.eqiad.wmnet [production]
09:10 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host rpki2002.codfw.wmnet [production]
09:07 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host rpki2002.codfw.wmnet [production]
08:57 <mvernon@cumin1001> START - Cookbook sre.hosts.reimage for host ms-be2053.codfw.wmnet with OS bullseye [production]
08:53 <klausman@cumin2002> START - Cookbook sre.hosts.reimage for host ores2003.codfw.wmnet with OS buster [production]