3901-3950 of 10000 results (35ms)
2020-05-12 ยง
15:48 <andrew@cumin1001> START - Cookbook sre.hosts.downtime [production]
15:48 <andrew@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
15:48 <andrew@cumin1001> START - Cookbook sre.hosts.downtime [production]
15:34 <filippo@cumin1001> conftool action : set/pooled=true; selector: dnsdisc=thanos-query [production]
15:15 <akosiaris@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'kube-system' for release 'calico-policy-controller' . [production]
15:15 <akosiaris@deploy1001> helmfile [EQIAD] Ran 'sync' command on namespace 'kube-system' for release 'calico-policy-controller' . [production]
15:14 <akosiaris@deploy1001> helmfile [CODFW] Ran 'sync' command on namespace 'kube-system' for release 'calico-policy-controller' . [production]
15:13 <sukhe@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
15:13 <sukhe@cumin1001> START - Cookbook sre.hosts.downtime [production]
15:12 <dzahn@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) [production]
15:12 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime [production]
15:09 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
15:09 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]
15:07 <kormat@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
15:05 <kormat@cumin1001> START - Cookbook sre.hosts.downtime [production]
15:04 <moritzm> installing 4.9.118 Linux updates on Buster nodes (reboots happening later) [production]
15:02 <moritzm> upgrading contint2001 to openjdk-8 u252 [production]
15:01 <godog> bounce pybal on lvs2010 and lvs2009 - T252186 [production]
14:40 <moritzm> imported openjdk-8 u252 forward port for buster-wikimedia component/jdk8 [production]
14:39 <ema> rolling thumbor upgrade to 2.8-1+deb10u1 T252509 T219569 T236240 [production]
14:39 <andrewbogott> rebuilding cloudcontrol1003 and 1004 [production]
14:38 <hashar> 1.35.0-wmf.22 is on test wikis. Will be pushed to group0 later today during the american window (19:00 - 21:00 UTC) # T249964 [production]
14:34 <ema> thumbor2001: repool [production]
14:33 <otto@deploy1001> Synchronized wmf-config/InitialiseSettings.php: EventLogging to EventGate: - Test everywhere, SearchSatisfaction on testwiki only - T249261 (duration: 01m 06s) [production]
14:33 <ema> thumbor2001: upgrade python-thumbor-wikimedia to 2.8-1+deb10u1 T252509 T219569 T236240 [production]
14:23 <moritzm> installing Java security updates on WDQS hosts [production]
14:20 <hashar@deploy1001> Finished scap: testwikis wikis to 1.35.0-wmf.32 (duration: 72m 04s) [production]
14:05 <aborrero@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
14:05 <aborrero@cumin1001> START - Cookbook sre.hosts.downtime [production]
14:05 <aborrero@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
14:05 <aborrero@cumin1001> START - Cookbook sre.hosts.downtime [production]
14:02 <dzahn@cumin1001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) [production]
14:00 <ema> thumbor2001: depool due to minor bug in 2.7-1+deb10u1 T252509 T219569 T236240 [production]
13:54 <ema> thumbor2001: pool thumbor 2.7-1+deb10u1 for prod traffic T252509 T219569 T236240 [production]
13:50 <ema> thumbor2001: upgrade python-thumbor-wikimedia to 2.7-1+deb10u1 T252509 T219569 T236240 [production]
13:42 <jbond42> disable puppet on all CP hosts to deploy https://gerrit.wikimedia.org/r/c/operations/puppet/+/583342 [production]
13:36 <kormat> reimaging pc2007 to buster T252182 [production]
13:36 <moritzm> rebooting netflow* hosts for kernel update [production]
13:36 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
13:36 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]
13:33 <vgutierrez> rolling upgrade of ATS to version 8.0.7-1wm5 - T249335 [production]
13:31 <moritzm> rebooting deneb for kernel update [production]
13:30 <jmm@cumin2001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
13:30 <jmm@cumin2001> START - Cookbook sre.hosts.downtime [production]
13:24 <dzahn@cumin1001> START - Cookbook sre.ganeti.makevm [production]
13:24 <dzahn@cumin1001> END (FAIL) - Cookbook sre.ganeti.makevm (exit_code=99) [production]
13:24 <dzahn@cumin1001> START - Cookbook sre.ganeti.makevm [production]
13:08 <hashar@deploy1001> Started scap: testwikis wikis to 1.35.0-wmf.32 [production]
13:05 <hashar@deploy1001> Pruned MediaWiki: 1.35.0-wmf.28 (duration: 23m 47s) [production]
12:37 <moritzm> installing iputils update from Buster point release [production]