351-400 of 10000 results (40ms)
2021-12-07 ยง
22:07 <andrew@cumin1001> START - Cookbook sre.hosts.reimage for host cloudvirt1028.eqiad.wmnet with OS buster [production]
22:06 <andrew@cumin1001> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=93) for host cloudvirt1028.eqiad.wmnet with OS buster [production]
21:26 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
21:25 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
21:23 <dancy@deploy1002> Started scap: testwikis wikis to 1.38.0-wmf.12 refs T293953 [production]
21:18 <andrew@cumin1001> START - Cookbook sre.hosts.reimage for host cloudvirt1028.eqiad.wmnet with OS buster [production]
19:58 <ebernhardson@deploy1002> Finished deploy [wdqs/wdqs@c21117f] (wcqs): Deploy version 0.3.95 to wcqs (duration: 01m 48s) [production]
19:56 <ebernhardson@deploy1002> Started deploy [wdqs/wdqs@c21117f] (wcqs): Deploy version 0.3.95 to wcqs [production]
19:56 <cmooney@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudvirt1028.eqiad.wmnet with OS buster [production]
19:49 <eileen> revision civicrm 311382de -> c47cf762 [production]
19:26 <legoktm> upgrading sacp to 4.1.0 everywhere (T296867) [production]
19:26 <cmooney@cumin1001> START - Cookbook sre.hosts.reimage for host cloudvirt1028.eqiad.wmnet with OS buster [production]
19:18 <herron> graphite1004.mgmt: racadm serveraction powercycle [production]
19:13 <ebernhardson> start outage recovery for commonswiki against eqiad cirrus cluster after snapshot restore [production]
18:47 <jgiannelos@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'tegola-vector-tiles' for release 'main' . [production]
18:46 <jgiannelos@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'tegola-vector-tiles' for release 'main' . [production]
18:45 <jgiannelos@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'tegola-vector-tiles' for release 'main' . [production]
18:38 <jgiannelos@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'proton' for release 'production' . [production]
18:33 <jgiannelos@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'proton' for release 'production' . [production]
18:27 <jgiannelos@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'proton' for release 'production' . [production]
17:54 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host rpki2002.codfw.wmnet [production]
17:51 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host rpki2002.codfw.wmnet [production]
17:41 <herron> graphite1004.mgmt: racadm serveraction powercycle [production]
17:40 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host rpki1001.eqiad.wmnet [production]
17:36 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host rpki1001.eqiad.wmnet [production]
17:36 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host rpki1001.eqiad.wmnet [production]
17:35 <root@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'miscweb' for release 'main' . [production]
17:34 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host rpki1001.eqiad.wmnet [production]
17:33 <root@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'miscweb' for release 'main' . [production]
17:32 <root@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'miscweb' for release 'main' . [production]
17:31 <root@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'miscweb' for release 'main' . [production]
17:27 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
17:27 <root@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'miscweb' for release 'main' . [production]
17:26 <mwdebug-deploy@deploy1002> helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
17:25 <root@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'miscweb' for release 'main' . [production]
17:19 <root@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'miscweb' for release 'main' . [production]
16:25 <Amir1> deleting broken flaggedtemplates rows on dewiki (T297094) [production]
16:14 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host copernicium.wikimedia.org [production]
16:08 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host copernicium.wikimedia.org [production]
16:07 <root@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'miscweb' for release 'main' . [production]
16:07 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host failoid2002.codfw.wmnet [production]
16:04 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host failoid2002.codfw.wmnet [production]
16:02 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ldap-replica1003.wikimedia.org [production]
16:00 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host ldap-replica1003.wikimedia.org [production]
15:55 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ldap-replica2005.wikimedia.org [production]
15:53 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host ldap-replica2005.wikimedia.org [production]
15:53 <mwdebug-deploy@deploy1002> helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . [production]
15:52 <kormat@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2084.codfw.wmnet with reason: Reracking T296930 [production]
15:52 <kormat@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2084.codfw.wmnet with reason: Reracking T296930 [production]
15:50 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ldap-replica2006.wikimedia.org [production]