| 
      
        2018-08-27
      
      §
     | 
  
    
  | 08:17 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1104 (duration: 00m 52s) | 
  [production] | 
            
  | 08:05 | 
  <gehel> | 
  restarting kartotherian / tilerator on maps* for nodejs upgrade | 
  [production] | 
            
  | 07:59 | 
  <moritzm> | 
  installing nodejs security updates on maps* hosts | 
  [production] | 
            
  | 07:44 | 
  <elukey> | 
  force remount of /mnt/hdfs on stat1005 (transport not connected errors) | 
  [production] | 
            
  | 07:42 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1104 (duration: 00m 50s) | 
  [production] | 
            
  | 07:33 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1101:3318 (duration: 00m 48s) | 
  [production] | 
            
  | 07:10 | 
  <moritzm> | 
  reimaging wtp2014-wtp2016 to stretch | 
  [production] | 
            
  | 06:53 | 
  <marostegui> | 
  Deploy schema change on labtestwiki - T89737 | 
  [production] | 
            
  | 06:52 | 
  <moritzm> | 
  installing discover updates from stretch 9.5 point release | 
  [production] | 
            
  | 06:50 | 
  <marostegui> | 
  Deploy schema change on labtestwiki - T187089 | 
  [production] | 
            
  | 06:38 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1101:3318 (duration: 00m 48s) | 
  [production] | 
            
  | 06:34 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1099:3318 (duration: 00m 49s) | 
  [production] | 
            
  | 05:31 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1099:3318 (duration: 00m 50s) | 
  [production] | 
            
  | 05:16 | 
  <moritzm> | 
  reimaging wtp2011-wtp2013 to stretch | 
  [production] | 
            
  | 05:14 | 
  <marostegui> | 
  Deploy schema change on db1066 (s2 primary master) | 
  [production] | 
            
  | 02:46 | 
  <l10nupdate@deploy1001> | 
  ResourceLoader cache refresh completed at Mon Aug 27 02:46:56 UTC 2018 (duration 10m 12s) | 
  [production] | 
            
  | 02:36 | 
  <l10nupdate@deploy1001> | 
  scap sync-l10n completed (1.32.0-wmf.18) (duration: 15m 13s) | 
  [production] | 
            
  
    | 
      
        2018-08-24
      
      §
     | 
  
    
  | 22:07 | 
  <krinkle@deploy1001> | 
  Synchronized php-1.32.0-wmf.18/extensions/WikimediaEvents/: I17452c980f2588bbd (duration: 00m 50s) | 
  [production] | 
            
  | 17:44 | 
  <James_F> | 
  scap sync-dir php-1.32.0-wmf.18/extensions/WikimediaEvents/ No-op bump this code to avoid dirty deploy repo | 
  [production] | 
            
  | 17:44 | 
  <James_F> | 
  scap sync-file php-1.32.0-wmf.18/extensions/Echo/includes/gateway/UserNotificationGateway.php Hot-deploy of Ifdfa93059 to fix T202672 log error | 
  [production] | 
            
  | 17:44 | 
  <James_F> | 
  scap sync-file php-1.32.0-wmf.18/extensions/WikiEditor/modules/jquery.wikiEditor.dialogs.config.js Hot-deploy of I364ac118255 to fix missing icon | 
  [production] | 
            
  | 15:07 | 
  <marostegui> | 
  Deploy schema change on dbstore1002:s8 | 
  [production] | 
            
  | 13:45 | 
  <marostegui> | 
  Deploy schema change on s8 codfw master with replication (this will generate lag on s8 codfw) | 
  [production] | 
            
  | 12:33 | 
  <akosiaris> | 
  upload apertium-kaz-tat to apt.wikimedia.org/jessie-wikimedia/main T199962 | 
  [production] | 
            
  | 12:33 | 
  <akosiaris> | 
  upload apertium-eo-fr to apt.wikimedia.org/jessie-wikimedia/main T199962 | 
  [production] | 
            
  | 11:58 | 
  <elukey> | 
  upgrade nodejs packages on aqs* for security upgrade (rolling restart of aqs daemon included) | 
  [production] | 
            
  | 11:19 | 
  <ema> | 
  powercycle cp3030, stuck rebooting T200445 | 
  [production] | 
            
  | 10:47 | 
  <moritzm> | 
  reimage wtp2008-wtp2010 to stretch | 
  [production] | 
            
  | 09:28 | 
  <moritzm> | 
  reimage wtp2005-wtp2007 to stretch | 
  [production] | 
            
  | 09:09 | 
  <marostegui> | 
  Restart MySQL on db2035 (codfw s2 master) for mysql upgrade and pick up new mysql options after merging https://gerrit.wikimedia.org/r/455103 | 
  [production] | 
            
  | 09:04 | 
  <marostegui> | 
  Deploy schema change on db1095:3312 | 
  [production] | 
            
  | 07:56 | 
  <marostegui> | 
  Deploy schema change on db1122 | 
  [production] | 
            
  | 07:53 | 
  <moritzm> | 
  reimage wtp2002-wtp2004 to stretch | 
  [production] | 
            
  | 06:52 | 
  <marostegui> | 
  Deploy schema change on db1074 with replication (this will generate lag on s2 labs) | 
  [production] | 
            
  | 06:38 | 
  <moritzm> | 
  reimage wtp2001 to stretch | 
  [production] | 
            
  | 06:09 | 
  <moritzm> | 
  imported php-igbinary, libzip5, php-apcu-bc to thirdparty/php72 (T202704) | 
  [production] | 
            
  | 05:26 | 
  <marostegui> | 
  Deploy schema change on db1076 | 
  [production] | 
            
  | 00:21 | 
  <mutante> | 
  ms-be2043 - mount -a ; re-enabled puppet, running puppet | 
  [production] | 
            
  
    | 
      
        2018-08-23
      
      §
     | 
  
    
  | 23:47 | 
  <reedy@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: T202548 (duration: 01m 01s) | 
  [production] | 
            
  | 23:43 | 
  <Reedy> | 
  created wikilove tables on zh_yuewiki T202548 | 
  [production] | 
            
  | 23:42 | 
  <reedy@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: T201236 (duration: 00m 55s) | 
  [production] | 
            
  | 23:37 | 
  <reedy@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: T202599 (duration: 00m 53s) | 
  [production] | 
            
  | 23:34 | 
  <reedy@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: T202599 (duration: 00m 49s) | 
  [production] | 
            
  | 23:28 | 
  <Reedy> | 
  running `mwscript updateCollation.php --wiki=trwiki --previous-collation=uppercase` T184943 | 
  [production] | 
            
  | 23:28 | 
  <reedy@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: T184943 (duration: 00m 48s) | 
  [production] | 
            
  | 23:23 | 
  <reedy@deploy1001> | 
  Synchronized wmf-config/flaggedrevs.php: T202139 (duration: 00m 48s) | 
  [production] | 
            
  | 23:19 | 
  <reedy@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: T202680 (duration: 00m 48s) | 
  [production] | 
            
  | 23:16 | 
  <reedy@deploy1001> | 
  scap failed: average error rate on 3/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/2cc7028226a539553178454fc2f14459 for details) | 
  [production] | 
            
  | 23:16 | 
  <reedy@deploy1001> | 
  sync-file aborted: (no justification provided) (duration: 00m 01s) | 
  [production] |