9501-9550 of 10000 results (90ms)
2020-02-12 ยง
13:21 <marostegui> Restart wikibugs as phab comments aren't showing up on irc - T241109 [production]
13:20 <vgutierrez@cumin1001> START - Cookbook sre.hosts.downtime [production]
13:18 <jynus> setting up db1140 under maintenance (upgrade, reboot, disable alerts) [production]
13:15 <vgutierrez> disabling KA between ats-tls and varnish-fe on cp4031 - T244464 [production]
13:10 <moritzm> upgrading debdeploy fleet-wide to 0.0.99.13 [production]
13:08 <moritzm> uploaded libapache2-mod-auth-cas 1.2-1~deb8u1 for jessie-wikimedia to apt.wikimedia.org [production]
13:05 <vgutierrez> depool cp20[10,18] and reimage as buster - T242093 [production]
13:05 <vgutierrez> pool cp20[12,20] running buster - T242093 [production]
12:55 <vgutierrez@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
12:53 <vgutierrez@cumin1001> START - Cookbook sre.hosts.downtime [production]
12:53 <vgutierrez@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
12:53 <XioNoX> cr1-eqsin RE failover - T244944 [production]
12:50 <vgutierrez@cumin1001> START - Cookbook sre.hosts.downtime [production]
12:35 <vgutierrez> depool cp20[12,20] and reimage as buster - T242093 [production]
12:34 <vgutierrez> pool cp20[13,22] running buster - T242093 [production]
12:26 <vgutierrez@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
12:24 <vgutierrez@cumin1001> START - Cookbook sre.hosts.downtime [production]
12:21 <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:571705|Triple the factor of WDQS lag to maxlag for Wikidata (T244722)]], take II, the cache issue (duration: 01m 03s) [production]
12:19 <vgutierrez@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
12:19 <ladsgroup@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:571705|Triple the factor of WDQS lag to maxlag for Wikidata (T244722)]] (duration: 01m 04s) [production]
12:17 <vgutierrez@cumin1001> START - Cookbook sre.hosts.downtime [production]
12:12 <kartik@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit|571412|Enable ContentTranslation out of beta in bs and mk WPs (T244139, T244140)]] (duration: 01m 15s) [production]
12:08 <vgutierrez> depool cp2013 and reimage as buster - T242093 [production]
12:06 <vgutierrez> pool cp2016 running buster - T242093 [production]
12:01 <vgutierrez> depool cp20[16,22] and reimage as buster - T242093 [production]
11:57 <vgutierrez> pool cp20[19,24] running buster - T242093 [production]
11:53 <akosiaris> mangle sessionstore on mw1331 so that it is unreachable. Testing for T243106 [production]
11:49 <vgutierrez> repooling cp40[26,32] [production]
11:39 <vgutierrez> pool cp3050 running buster - T242093 [production]
11:37 <vgutierrez> depooling cp[4026,4032] [production]
11:35 <vgutierrez@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
11:33 <vgutierrez@cumin1001> START - Cookbook sre.hosts.downtime [production]
11:32 <vgutierrez@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
11:30 <vgutierrez@cumin1001> START - Cookbook sre.hosts.downtime [production]
11:19 <vgutierrez@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
11:18 <vgutierrez> depool cp2024 and reimage as buster - T242093 [production]
11:17 <vgutierrez> pool cp2025 running buster - T242093 [production]
11:16 <vgutierrez@cumin1001> START - Cookbook sre.hosts.downtime [production]
11:15 <vgutierrez> depool cp2016 and reimage as buster - T242093 [production]
11:14 <vgutierrez> pool cp2019 running buster - T242093 [production]
11:11 <moritzm> reimage logstash2026 to test new standard RAID0 partman recipe [production]
11:05 <vgutierrez@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
11:03 <vgutierrez@cumin1001> START - Cookbook sre.hosts.downtime [production]
11:03 <vgutierrez@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
11:00 <vgutierrez@cumin1001> START - Cookbook sre.hosts.downtime [production]
10:50 <vgutierrez> depool cp3050 and reimage as buster - T242093 [production]
10:49 <vgutierrez> pool cp30[51,52] running buster - T242093 [production]
10:45 <vgutierrez> depool cp20[19,25] and reimage as buster - T242093 [production]
10:42 <vgutierrez> pool cp2026 running buster - T242093 [production]
10:36 <vgutierrez> pool cp2023 running buster - T242093 [production]