9201-9250 of 10000 results (64ms)
2021-03-02 ยง
13:44 <akosiaris@cumin1001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host kubemaster1002.eqiad.wmnet [production]
13:42 <akosiaris@cumin1001> START - Cookbook sre.ganeti.makevm for new host kubestagemaster1001.eqiad.wmnet [production]
13:25 <akosiaris@cumin1001> START - Cookbook sre.ganeti.makevm for new host kubemaster1002.eqiad.wmnet [production]
13:24 <akosiaris@cumin1001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host kubemaster1001.eqiad.wmnet [production]
13:16 <akosiaris@cumin1001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host kubemaster2001.codfw.wmnet [production]
13:13 <akosiaris@deploy1001> helmfile [eqiad] Ran 'sync' command on namespace 'kube-system' for release 'calico-policy-controller' . [production]
13:10 <akosiaris@deploy1001> helmfile [staging] Ran 'sync' command on namespace 'kube-system' for release 'calico-policy-controller' . [production]
13:08 <akosiaris@cumin1001> START - Cookbook sre.ganeti.makevm for new host kubemaster1001.eqiad.wmnet [production]
12:53 <akosiaris@cumin1001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host kubemaster2002.codfw.wmnet [production]
12:53 <akosiaris@deploy1001> helmfile [codfw] Ran 'sync' command on namespace 'kube-system' for release 'calico-policy-controller' . [production]
12:46 <akosiaris@cumin1001> START - Cookbook sre.ganeti.makevm for new host kubemaster2001.codfw.wmnet [production]
12:44 <aborrero@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cloudvirt1012.eqiad.wmnet [production]
12:43 <akosiaris@cumin1001> END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) for new host kubemaster2001.codfw.wmnet [production]
12:39 <aborrero@cumin1001> START - Cookbook sre.hosts.reboot-single for host cloudvirt1012.eqiad.wmnet [production]
12:32 <jayme@cumin1001> END (FAIL) - Cookbook sre.discovery.service-route (exit_code=99) [production]
12:32 <jayme@cumin1001> START - Cookbook sre.discovery.service-route [production]
12:28 <jayme@cumin1001> conftool action : set/pooled=false; selector: dnsdisc=helm-charts,name=codfw [production]
12:23 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: 29952b404b3fe9c235da86df0ffb86b725845473: vector: Stage 3 of WVUI search treatment A/B test (T249297) (duration: 01m 08s) [production]
12:21 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: 5674d2ab64c2833e15ac8a90696fcde529e58dca: Enable SectionTranslation in testwiki (T275596) (duration: 01m 09s) [production]
12:13 <jayme@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host kubestagetcd2003.codfw.wmnet [production]
12:12 <jayme@cumin1001> START - Cookbook sre.hosts.reboot-single for host kubestagetcd2003.codfw.wmnet [production]
12:12 <mbsantos@deploy1002> Finished deploy [tilerator/deploy@8d3d81c]: (no justification provided) (duration: 00m 15s) [production]
12:11 <mbsantos@deploy1002> Started deploy [tilerator/deploy@8d3d81c]: (no justification provided) [production]
12:07 <jayme@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host kubestagetcd2002.codfw.wmnet [production]
12:06 <urbanecm@deploy1002> Synchronized wmf-config/InitialiseSettings.php: af89965e80a77e92e78e3948e0678460decd7718: Remove test2wiki from wgContentTranslationAsBetaFeature (duration: 01m 38s) [production]
12:02 <jayme@cumin1001> START - Cookbook sre.hosts.reboot-single for host kubestagetcd2002.codfw.wmnet [production]
11:59 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1084 to clone db1164 T258361', diff saved to https://phabricator.wikimedia.org/P14554 and previous config saved to /var/cache/conftool/dbconfig/20210302-115959-marostegui.json [production]
11:53 <jayme@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host kubestagetcd2001.codfw.wmnet [production]
11:53 <mbsantos@deploy1002> Finished deploy [tilerator/deploy@937deb5]: (no justification provided) (duration: 00m 03s) [production]
11:53 <mbsantos@deploy1002> Started deploy [tilerator/deploy@937deb5]: (no justification provided) [production]
11:49 <jayme@cumin1001> START - Cookbook sre.hosts.reboot-single for host kubestagetcd2001.codfw.wmnet [production]
11:47 <jayme@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host kubestagemaster2001.codfw.wmnet [production]
11:40 <jayme@cumin1001> START - Cookbook sre.hosts.reboot-single for host kubestagemaster2001.codfw.wmnet [production]
11:16 <akosiaris@cumin1001> START - Cookbook sre.ganeti.makevm for new host kubemaster2002.codfw.wmnet [production]
11:16 <akosiaris@cumin1001> START - Cookbook sre.ganeti.makevm for new host kubemaster2001.codfw.wmnet [production]
11:12 <hnowlan@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'api-gateway' for release 'staging' . [production]
11:11 <hnowlan@deploy1002> helmfile [staging] Ran 'sync' command on namespace 'api-gateway' for release 'production' . [production]
10:37 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host mc1028.eqiad.wmnet [production]
10:31 <jiji@cumin1001> START - Cookbook sre.hosts.reboot-single for host mc1028.eqiad.wmnet [production]
10:29 <effie> upgrade memcached on mc2024, mc1028 [production]
10:21 <elukey@cumin1001> END (PASS) - Cookbook sre.hadoop.init-hadoop-workers (exit_code=0) for hosts an-worker1119.eqiad.wmnet [production]
10:18 <elukey@cumin1001> START - Cookbook sre.hadoop.init-hadoop-workers for hosts an-worker1119.eqiad.wmnet [production]
10:12 <elukey@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1119.eqiad.wmnet with reason: REIMAGE [production]
10:09 <elukey@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1119.eqiad.wmnet with reason: REIMAGE [production]
10:05 <volans@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on sretest1002.eqiad.wmnet with reason: REIMAGE [production]
10:03 <volans@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on sretest1002.eqiad.wmnet with reason: REIMAGE [production]
09:54 <elukey@cumin1001> END (PASS) - Cookbook sre.hadoop.init-hadoop-workers (exit_code=0) for hosts an-worker[1130-1131].eqiad.wmnet [production]
09:52 <elukey@cumin1001> START - Cookbook sre.hadoop.init-hadoop-workers for hosts an-worker[1130-1131].eqiad.wmnet [production]
09:46 <liw@deploy1002> Finished scap: testwikis wikis to 1.36.0-wmf.33 (duration: 36m 20s) [production]
09:43 <elukey@cumin1001> END (PASS) - Cookbook sre.hadoop.init-hadoop-workers (exit_code=0) for hosts an-worker[1124-1128].eqiad.wmnet [production]