5251-5300 of 10000 results (71ms)
2018-09-06 ยง
20:58 <ppchelko@deploy1001> Started deploy [restbase/deploy@53bc0a6]: Revert bumping Parsoid content-type filter T194190, take 5 [production]
20:56 <ppchelko@deploy1001> Finished deploy [restbase/deploy@53bc0a6]: Revert bumping Parsoid content-type filter T194190, take 4 (duration: 04m 47s) [production]
20:51 <ppchelko@deploy1001> Started deploy [restbase/deploy@53bc0a6]: Revert bumping Parsoid content-type filter T194190, take 4 [production]
20:51 <ppchelko@deploy1001> Finished deploy [restbase/deploy@53bc0a6]: Revert bumping Parsoid content-type filter T194190, take 3 (duration: 07m 16s) [production]
20:44 <ppchelko@deploy1001> Started deploy [restbase/deploy@53bc0a6]: Revert bumping Parsoid content-type filter T194190, take 3 [production]
20:43 <ppchelko@deploy1001> Finished deploy [restbase/deploy@53bc0a6]: Revert bumping Parsoid content-type filter T194190, take 2 (duration: 09m 26s) [production]
20:34 <ppchelko@deploy1001> Started deploy [restbase/deploy@53bc0a6]: Revert bumping Parsoid content-type filter T194190, take 2 [production]
20:34 <ppchelko@deploy1001> Finished deploy [restbase/deploy@53bc0a6]: Revert bumping Parsoid content-type filter T194190 (duration: 03m 56s) [production]
20:30 <ppchelko@deploy1001> Started deploy [restbase/deploy@53bc0a6]: Revert bumping Parsoid content-type filter T194190 [production]
18:27 <ebernhardson@deploy1001> Finished deploy [search/mjolnir/deploy@d3e2c23]: repair msearch daemon cli args (duration: 03m 56s) [production]
18:23 <ebernhardson@deploy1001> Started deploy [search/mjolnir/deploy@d3e2c23]: repair msearch daemon cli args [production]
18:20 <krinkle@deploy1001> Synchronized php-1.32.0-wmf.20/extensions/Flow/: Ia0112ae62e6b - T203647 (duration: 01m 02s) [production]
18:12 <krinkle@deploy1001> Synchronized php-1.32.0-wmf.20/includes/: I31a97d0168 - T203583 (duration: 01m 13s) [production]
17:56 <thcipriani@deploy1001> rebuilt and synchronized wikiversions files: Canary machines back to wmf.20 [production]
17:45 <thcipriani@deploy1001> scap failed: average error rate on 6/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/2cc7028226a539553178454fc2f14459 for details) [production]
17:34 <volans> uploaded spicerack_0.0.7-1{,+deb9u1} to apt.wikimedia.org {jessie,stretch}-wikimedia - T199079 [production]
17:20 <jynus_> dropping old logs from dbproxy1003 T201021 [production]
17:15 <ebernhardson@deploy1001> Finished deploy [search/mjolnir/deploy@735235b]: re-try bump to master (duration: 12m 46s) [production]
17:02 <ebernhardson@deploy1001> Started deploy [search/mjolnir/deploy@735235b]: re-try bump to master [production]
17:00 <ebernhardson@deploy1001> Finished deploy [search/mjolnir/deploy@735235b]: new cli flags for msearch daemon, bump kafka-python dep to 1.4.x (duration: 16m 19s) [production]
16:44 <ebernhardson@deploy1001> Started deploy [search/mjolnir/deploy@735235b]: new cli flags for msearch daemon, bump kafka-python dep to 1.4.x [production]
16:44 <ebernhardson@deploy1001> deploy aborted: new cli flags for msearch daemon, bump kafka-python dep to 1.4.x (duration: 00m 07s) [production]
16:43 <ebernhardson@deploy1001> Started deploy [search/mjolnir/deploy@735235b]: new cli flags for msearch daemon, bump kafka-python dep to 1.4.x [production]
16:40 <volans> upgraded spicerack to version 0.0.6 on sarin/neodymium - T199079 [production]
16:38 <volans> uploaded spicerack_0.0.6-1{,+deb9u1} to apt.wikimedia.org {jessie,stretch}-wikimedia - T199079 [production]
15:41 <END> (PASS) - Cookbook sre.switchdc.services.02-restore-ttl (exit_code=0) (switchdc/oblivian@neodymium) [production]
15:41 <START> - Cookbook sre.switchdc.services.02-restore-ttl (switchdc/oblivian@neodymium) [production]
15:37 <END> (PASS) - Cookbook sre.switchdc.services.01-switch-dc (exit_code=0) (switchdc/oblivian@neodymium) [production]
15:37 <Switching> services pdfrender: eqiad => codfw (switchdc/oblivian@neodymium) [production]
15:37 <START> - Cookbook sre.switchdc.services.01-switch-dc (switchdc/oblivian@neodymium) [production]
15:36 <END> (PASS) - Cookbook sre.switchdc.services.00-reduce-ttl-and-sleep (exit_code=0) (switchdc/oblivian@neodymium) [production]
15:31 <START> - Cookbook sre.switchdc.services.00-reduce-ttl-and-sleep (switchdc/oblivian@neodymium) [production]
15:30 <END> (PASS) - Cookbook sre.switchdc.services.02-restore-ttl (exit_code=0) (switchdc/oblivian@neodymium) [production]
15:30 <START> - Cookbook sre.switchdc.services.02-restore-ttl (switchdc/oblivian@neodymium) [production]
15:30 <END> (PASS) - Cookbook sre.switchdc.services.01-switch-dc (exit_code=0) (switchdc/oblivian@neodymium) [production]
15:30 <Switching> services pdfrender: codfw => eqiad (switchdc/oblivian@neodymium) [production]
15:30 <START> - Cookbook sre.switchdc.services.01-switch-dc (switchdc/oblivian@neodymium) [production]
15:04 <END> (PASS) - Cookbook sre.switchdc.services.00-reduce-ttl-and-sleep (exit_code=0) (switchdc/oblivian@neodymium) [production]
14:59 <START> - Cookbook sre.switchdc.services.00-reduce-ttl-and-sleep (switchdc/oblivian@neodymium) [production]
14:52 <END> (PASS) - Cookbook sre.switchdc.services.02-restore-ttl (exit_code=0) (switchdc/oblivian@neodymium) [production]
14:52 <START> - Cookbook sre.switchdc.services.02-restore-ttl (switchdc/oblivian@neodymium) [production]
14:52 <END> (PASS) - Cookbook sre.switchdc.services.01-switch-dc (exit_code=0) (switchdc/oblivian@neodymium) [production]
14:52 <Switching> services pdfrender: eqiad => codfw (switchdc/oblivian@neodymium) [production]
14:51 <START> - Cookbook sre.switchdc.services.01-switch-dc (switchdc/oblivian@neodymium) [production]
14:51 <END> (PASS) - Cookbook sre.switchdc.services.00-reduce-ttl-and-sleep (exit_code=0) (switchdc/oblivian@neodymium) [production]
14:46 <START> - Cookbook sre.switchdc.services.00-reduce-ttl-and-sleep (switchdc/oblivian@neodymium) [production]
14:38 <elukey> reboot kafka2001 (eventbus codfw host) for kernel + openjdk-8 upgrades [production]
13:46 <moritzm> reboots of mc hosts in codfw completed [production]
13:45 <elukey> reboot kafka100[2-6] for kernel + openjdk-8 upgrades [production]
13:24 <elukey> reboot kafka-jumbo1001 for openjdk-8 + kernel security upgrades [production]