| 2018-09-07
      
      § | 
    
  | 14:27 | <moritzm> | installing libtirpc security updates on trusty | [production] | 
            
  | 12:35 | <elukey> | reboot kafka200[2,3] (eventbus codfw) for kernel + openjdk-8 upgrades | [production] | 
            
  | 10:03 | <Amir1> | ladsgroup@mwmaint1001:~$ mwscript extensions/CentralAuth/maintenance/deleteLocalPasswords.php --wiki=fawiki --user Ladsgroup --prefix (T201009) | [production] | 
            
  | 09:38 | <hashar@deploy1001> | Synchronized php-1.32.0-wmf.20/extensions/UniversalLanguageSelector: Revert "Simplify by using native JavaScript instead of jQuery" - T203750 (duration: 00m 55s) | [production] | 
            
  | 08:49 | <ema> | passive checks awol on einsteinium, restarting icinga -- T196336 | [production] | 
            
  | 08:45 | <jynus> | reloading apache with bad config for tendril for testing (small downtime) | [production] | 
            
  | 08:33 | <marostegui> | Rebooting haproxies to pick up new config after all the tests - T201021 | [production] | 
            
  | 08:16 | <banyek> | genarting false alert about https auth on dbmonitor1001 | [production] | 
            
  | 07:24 | <moritzm> | rebooting mw2270-mw2290 for kernel security updates | [production] | 
            
  | 06:51 | <moritzm> | rebooting mw2240-mw2269 for kernel security updates | [production] | 
            
  | 06:49 | <moritzm> | rebooting mw2240-mw2269 for kernel security updates | [production] | 
            
  | 04:58 | <marostegui> | Disable puppet on dbproxy1006 for logging testing - T201021 | [production] | 
            
  | 00:21 | <thcipriani@deploy1001> | Finished scap: SWAT: [[gerrit:458586|Add italian translation]] T203297 [[gerrit:458609|Improve German translation]] [[gerrit:458617|German translation: Replace "Vertreter" by "EU-Abgeordnete"]] (duration: 34m 38s) | [production] | 
            
  
    | 2018-09-06
      
      § | 
    
  | 23:46 | <thcipriani@deploy1001> | Started scap: SWAT: [[gerrit:458586|Add italian translation]] T203297 [[gerrit:458609|Improve German translation]] [[gerrit:458617|German translation: Replace "Vertreter" by "EU-Abgeordnete"]] | [production] | 
            
  | 23:43 | <thcipriani@deploy1001> | Synchronized php-1.32.0-wmf.20/extensions/Score/includes/Score.php: SWAT: [[gerrit:458558|Add checks for length of generated audio files]] T203560 (duration: 00m 58s) | [production] | 
            
  | 23:16 | <ejegg> | updated CiviCRM from 7918924e73 to 52873b5dcc | [production] | 
            
  | 22:16 | <ebernhardson> | restart mjolnir-kafka-bulk-daemon service to pickup earlier deploy. | [production] | 
            
  | 21:43 | <RoanKattouw> | Deleting all user_properties rows with up_property='pagetriage-lastuse' (T202175) | [production] | 
            
  | 21:32 | <krinkle@deploy1001> | Synchronized php-1.32.0-wmf.20/includes/MovePage.php: T203661 - I9ebdcbc566b (duration: 00m 57s) | [production] | 
            
  | 21:19 | <krinkle@deploy1001> | Synchronized php-1.32.0-wmf.20/includes/exception/MWExceptionHandler.php: I3f35a519b50ae (duration: 00m 58s) | [production] | 
            
  | 21:02 | <ppchelko@deploy1001> | Finished deploy [restbase/deploy@53bc0a6]: Revert bumping Parsoid content-type filter T194190, take 5 (duration: 04m 01s) | [production] | 
            
  | 20:58 | <ppchelko@deploy1001> | Started deploy [restbase/deploy@53bc0a6]: Revert bumping Parsoid content-type filter T194190, take 5 | [production] | 
            
  | 20:56 | <ppchelko@deploy1001> | Finished deploy [restbase/deploy@53bc0a6]: Revert bumping Parsoid content-type filter T194190, take 4 (duration: 04m 47s) | [production] | 
            
  | 20:51 | <ppchelko@deploy1001> | Started deploy [restbase/deploy@53bc0a6]: Revert bumping Parsoid content-type filter T194190, take 4 | [production] | 
            
  | 20:51 | <ppchelko@deploy1001> | Finished deploy [restbase/deploy@53bc0a6]: Revert bumping Parsoid content-type filter T194190, take 3 (duration: 07m 16s) | [production] | 
            
  | 20:44 | <ppchelko@deploy1001> | Started deploy [restbase/deploy@53bc0a6]: Revert bumping Parsoid content-type filter T194190, take 3 | [production] | 
            
  | 20:43 | <ppchelko@deploy1001> | Finished deploy [restbase/deploy@53bc0a6]: Revert bumping Parsoid content-type filter T194190, take 2 (duration: 09m 26s) | [production] | 
            
  | 20:34 | <ppchelko@deploy1001> | Started deploy [restbase/deploy@53bc0a6]: Revert bumping Parsoid content-type filter T194190, take 2 | [production] | 
            
  | 20:34 | <ppchelko@deploy1001> | Finished deploy [restbase/deploy@53bc0a6]: Revert bumping Parsoid content-type filter T194190 (duration: 03m 56s) | [production] | 
            
  | 20:30 | <ppchelko@deploy1001> | Started deploy [restbase/deploy@53bc0a6]: Revert bumping Parsoid content-type filter T194190 | [production] | 
            
  | 18:27 | <ebernhardson@deploy1001> | Finished deploy [search/mjolnir/deploy@d3e2c23]: repair msearch daemon cli args (duration: 03m 56s) | [production] | 
            
  | 18:23 | <ebernhardson@deploy1001> | Started deploy [search/mjolnir/deploy@d3e2c23]: repair msearch daemon cli args | [production] | 
            
  | 18:20 | <krinkle@deploy1001> | Synchronized php-1.32.0-wmf.20/extensions/Flow/: Ia0112ae62e6b - T203647 (duration: 01m 02s) | [production] | 
            
  | 18:12 | <krinkle@deploy1001> | Synchronized php-1.32.0-wmf.20/includes/: I31a97d0168 - T203583 (duration: 01m 13s) | [production] | 
            
  | 17:56 | <thcipriani@deploy1001> | rebuilt and synchronized wikiversions files: Canary machines back to wmf.20 | [production] | 
            
  | 17:45 | <thcipriani@deploy1001> | scap failed: average error rate on 6/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/2cc7028226a539553178454fc2f14459 for details) | [production] | 
            
  | 17:34 | <volans> | uploaded spicerack_0.0.7-1{,+deb9u1} to apt.wikimedia.org {jessie,stretch}-wikimedia - T199079 | [production] | 
            
  | 17:20 | <jynus_> | dropping old logs from dbproxy1003 T201021 | [production] | 
            
  | 17:15 | <ebernhardson@deploy1001> | Finished deploy [search/mjolnir/deploy@735235b]: re-try bump to master (duration: 12m 46s) | [production] | 
            
  | 17:02 | <ebernhardson@deploy1001> | Started deploy [search/mjolnir/deploy@735235b]: re-try bump to master | [production] | 
            
  | 17:00 | <ebernhardson@deploy1001> | Finished deploy [search/mjolnir/deploy@735235b]: new cli flags for msearch daemon, bump kafka-python dep to 1.4.x (duration: 16m 19s) | [production] | 
            
  | 16:44 | <ebernhardson@deploy1001> | Started deploy [search/mjolnir/deploy@735235b]: new cli flags for msearch daemon, bump kafka-python dep to 1.4.x | [production] | 
            
  | 16:44 | <ebernhardson@deploy1001> | deploy aborted: new cli flags for msearch daemon, bump kafka-python dep to 1.4.x (duration: 00m 07s) | [production] | 
            
  | 16:43 | <ebernhardson@deploy1001> | Started deploy [search/mjolnir/deploy@735235b]: new cli flags for msearch daemon, bump kafka-python dep to 1.4.x | [production] | 
            
  | 16:40 | <volans> | upgraded spicerack to version 0.0.6 on sarin/neodymium - T199079 | [production] | 
            
  | 16:38 | <volans> | uploaded spicerack_0.0.6-1{,+deb9u1} to apt.wikimedia.org {jessie,stretch}-wikimedia - T199079 | [production] | 
            
  | 15:41 | <END> | (PASS) - Cookbook sre.switchdc.services.02-restore-ttl (exit_code=0) (switchdc/oblivian@neodymium) | [production] | 
            
  | 15:41 | <START> | - Cookbook sre.switchdc.services.02-restore-ttl (switchdc/oblivian@neodymium) | [production] | 
            
  | 15:37 | <END> | (PASS) - Cookbook sre.switchdc.services.01-switch-dc (exit_code=0) (switchdc/oblivian@neodymium) | [production] | 
            
  | 15:37 | <Switching> | services pdfrender: eqiad => codfw (switchdc/oblivian@neodymium) | [production] |