| 
      
        2018-09-07
      
      §
     | 
  
    
  | 14:29 | 
  <moritzm> | 
  installing PHP security updates on krypton | 
  [production] | 
            
  | 14:27 | 
  <moritzm> | 
  installing libtirpc security updates on trusty | 
  [production] | 
            
  | 12:35 | 
  <elukey> | 
  reboot kafka200[2,3] (eventbus codfw) for kernel + openjdk-8 upgrades | 
  [production] | 
            
  | 10:03 | 
  <Amir1> | 
  ladsgroup@mwmaint1001:~$ mwscript extensions/CentralAuth/maintenance/deleteLocalPasswords.php --wiki=fawiki --user Ladsgroup --prefix (T201009) | 
  [production] | 
            
  | 09:38 | 
  <hashar@deploy1001> | 
  Synchronized php-1.32.0-wmf.20/extensions/UniversalLanguageSelector: Revert "Simplify by using native JavaScript instead of jQuery" - T203750 (duration: 00m 55s) | 
  [production] | 
            
  | 08:49 | 
  <ema> | 
  passive checks awol on einsteinium, restarting icinga -- T196336 | 
  [production] | 
            
  | 08:45 | 
  <jynus> | 
  reloading apache with bad config for tendril for testing (small downtime) | 
  [production] | 
            
  | 08:33 | 
  <marostegui> | 
  Rebooting haproxies to pick up new config after all the tests - T201021 | 
  [production] | 
            
  | 08:16 | 
  <banyek> | 
  genarting false alert about https auth on dbmonitor1001 | 
  [production] | 
            
  | 07:24 | 
  <moritzm> | 
  rebooting mw2270-mw2290 for kernel security updates | 
  [production] | 
            
  | 06:51 | 
  <moritzm> | 
  rebooting mw2240-mw2269 for kernel security updates | 
  [production] | 
            
  | 06:49 | 
  <moritzm> | 
  rebooting mw2240-mw2269 for kernel security updates | 
  [production] | 
            
  | 04:58 | 
  <marostegui> | 
  Disable puppet on dbproxy1006 for logging testing - T201021 | 
  [production] | 
            
  | 00:21 | 
  <thcipriani@deploy1001> | 
  Finished scap: SWAT: [[gerrit:458586|Add italian translation]] T203297 [[gerrit:458609|Improve German translation]] [[gerrit:458617|German translation: Replace "Vertreter" by "EU-Abgeordnete"]] (duration: 34m 38s) | 
  [production] | 
            
  
    | 
      
        2018-09-06
      
      §
     | 
  
    
  | 23:46 | 
  <thcipriani@deploy1001> | 
  Started scap: SWAT: [[gerrit:458586|Add italian translation]] T203297 [[gerrit:458609|Improve German translation]] [[gerrit:458617|German translation: Replace "Vertreter" by "EU-Abgeordnete"]] | 
  [production] | 
            
  | 23:43 | 
  <thcipriani@deploy1001> | 
  Synchronized php-1.32.0-wmf.20/extensions/Score/includes/Score.php: SWAT: [[gerrit:458558|Add checks for length of generated audio files]] T203560 (duration: 00m 58s) | 
  [production] | 
            
  | 23:16 | 
  <ejegg> | 
  updated CiviCRM from 7918924e73 to 52873b5dcc | 
  [production] | 
            
  | 22:16 | 
  <ebernhardson> | 
  restart mjolnir-kafka-bulk-daemon service to pickup earlier deploy. | 
  [production] | 
            
  | 21:43 | 
  <RoanKattouw> | 
  Deleting all user_properties rows with up_property='pagetriage-lastuse' (T202175) | 
  [production] | 
            
  | 21:32 | 
  <krinkle@deploy1001> | 
  Synchronized php-1.32.0-wmf.20/includes/MovePage.php: T203661 - I9ebdcbc566b (duration: 00m 57s) | 
  [production] | 
            
  | 21:19 | 
  <krinkle@deploy1001> | 
  Synchronized php-1.32.0-wmf.20/includes/exception/MWExceptionHandler.php: I3f35a519b50ae (duration: 00m 58s) | 
  [production] | 
            
  | 21:02 | 
  <ppchelko@deploy1001> | 
  Finished deploy [restbase/deploy@53bc0a6]: Revert bumping Parsoid content-type filter T194190, take 5 (duration: 04m 01s) | 
  [production] | 
            
  | 20:58 | 
  <ppchelko@deploy1001> | 
  Started deploy [restbase/deploy@53bc0a6]: Revert bumping Parsoid content-type filter T194190, take 5 | 
  [production] | 
            
  | 20:56 | 
  <ppchelko@deploy1001> | 
  Finished deploy [restbase/deploy@53bc0a6]: Revert bumping Parsoid content-type filter T194190, take 4 (duration: 04m 47s) | 
  [production] | 
            
  | 20:51 | 
  <ppchelko@deploy1001> | 
  Started deploy [restbase/deploy@53bc0a6]: Revert bumping Parsoid content-type filter T194190, take 4 | 
  [production] | 
            
  | 20:51 | 
  <ppchelko@deploy1001> | 
  Finished deploy [restbase/deploy@53bc0a6]: Revert bumping Parsoid content-type filter T194190, take 3 (duration: 07m 16s) | 
  [production] | 
            
  | 20:44 | 
  <ppchelko@deploy1001> | 
  Started deploy [restbase/deploy@53bc0a6]: Revert bumping Parsoid content-type filter T194190, take 3 | 
  [production] | 
            
  | 20:43 | 
  <ppchelko@deploy1001> | 
  Finished deploy [restbase/deploy@53bc0a6]: Revert bumping Parsoid content-type filter T194190, take 2 (duration: 09m 26s) | 
  [production] | 
            
  | 20:34 | 
  <ppchelko@deploy1001> | 
  Started deploy [restbase/deploy@53bc0a6]: Revert bumping Parsoid content-type filter T194190, take 2 | 
  [production] | 
            
  | 20:34 | 
  <ppchelko@deploy1001> | 
  Finished deploy [restbase/deploy@53bc0a6]: Revert bumping Parsoid content-type filter T194190 (duration: 03m 56s) | 
  [production] | 
            
  | 20:30 | 
  <ppchelko@deploy1001> | 
  Started deploy [restbase/deploy@53bc0a6]: Revert bumping Parsoid content-type filter T194190 | 
  [production] | 
            
  | 18:27 | 
  <ebernhardson@deploy1001> | 
  Finished deploy [search/mjolnir/deploy@d3e2c23]: repair msearch daemon cli args (duration: 03m 56s) | 
  [production] | 
            
  | 18:23 | 
  <ebernhardson@deploy1001> | 
  Started deploy [search/mjolnir/deploy@d3e2c23]: repair msearch daemon cli args | 
  [production] | 
            
  | 18:20 | 
  <krinkle@deploy1001> | 
  Synchronized php-1.32.0-wmf.20/extensions/Flow/: Ia0112ae62e6b - T203647 (duration: 01m 02s) | 
  [production] | 
            
  | 18:12 | 
  <krinkle@deploy1001> | 
  Synchronized php-1.32.0-wmf.20/includes/: I31a97d0168 - T203583 (duration: 01m 13s) | 
  [production] | 
            
  | 17:56 | 
  <thcipriani@deploy1001> | 
  rebuilt and synchronized wikiversions files: Canary machines back to wmf.20 | 
  [production] | 
            
  | 17:45 | 
  <thcipriani@deploy1001> | 
  scap failed: average error rate on 6/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/2cc7028226a539553178454fc2f14459 for details) | 
  [production] | 
            
  | 17:34 | 
  <volans> | 
  uploaded spicerack_0.0.7-1{,+deb9u1} to apt.wikimedia.org {jessie,stretch}-wikimedia - T199079 | 
  [production] | 
            
  | 17:20 | 
  <jynus_> | 
  dropping old logs from dbproxy1003 T201021 | 
  [production] | 
            
  | 17:15 | 
  <ebernhardson@deploy1001> | 
  Finished deploy [search/mjolnir/deploy@735235b]: re-try bump to master (duration: 12m 46s) | 
  [production] | 
            
  | 17:02 | 
  <ebernhardson@deploy1001> | 
  Started deploy [search/mjolnir/deploy@735235b]: re-try bump to master | 
  [production] | 
            
  | 17:00 | 
  <ebernhardson@deploy1001> | 
  Finished deploy [search/mjolnir/deploy@735235b]: new cli flags for msearch daemon, bump kafka-python dep to 1.4.x (duration: 16m 19s) | 
  [production] | 
            
  | 16:44 | 
  <ebernhardson@deploy1001> | 
  Started deploy [search/mjolnir/deploy@735235b]: new cli flags for msearch daemon, bump kafka-python dep to 1.4.x | 
  [production] | 
            
  | 16:44 | 
  <ebernhardson@deploy1001> | 
  deploy aborted: new cli flags for msearch daemon, bump kafka-python dep to 1.4.x (duration: 00m 07s) | 
  [production] | 
            
  | 16:43 | 
  <ebernhardson@deploy1001> | 
  Started deploy [search/mjolnir/deploy@735235b]: new cli flags for msearch daemon, bump kafka-python dep to 1.4.x | 
  [production] | 
            
  | 16:40 | 
  <volans> | 
  upgraded spicerack to version 0.0.6 on sarin/neodymium - T199079 | 
  [production] | 
            
  | 16:38 | 
  <volans> | 
  uploaded spicerack_0.0.6-1{,+deb9u1} to apt.wikimedia.org {jessie,stretch}-wikimedia - T199079 | 
  [production] | 
            
  | 15:41 | 
  <END> | 
  (PASS) - Cookbook sre.switchdc.services.02-restore-ttl (exit_code=0) (switchdc/oblivian@neodymium) | 
  [production] | 
            
  | 15:41 | 
  <START> | 
  - Cookbook sre.switchdc.services.02-restore-ttl (switchdc/oblivian@neodymium) | 
  [production] | 
            
  | 15:37 | 
  <END> | 
  (PASS) - Cookbook sre.switchdc.services.01-switch-dc (exit_code=0) (switchdc/oblivian@neodymium) | 
  [production] |