| 
      
        2018-06-27
      
      §
     | 
  
    
  | 08:32 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1074 for alter table (duration: 00m 57s) | 
  [production] | 
            
  | 08:26 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1090:3312 after alter table (duration: 00m 57s) | 
  [production] | 
            
  | 07:58 | 
  <vgutierrez> | 
  Reinstall acamar & achernar as spare systems | 
  [production] | 
            
  | 07:50 | 
  <vgutierrez@puppetmaster1001> | 
  conftool action : set/pooled=yes; selector: name=dns4001.wikimedia.org | 
  [production] | 
            
  | 07:38 | 
  <vgutierrez@puppetmaster1001> | 
  conftool action : set/pooled=no; selector: name=dns4001.wikimedia.org | 
  [production] | 
            
  | 07:37 | 
  <vgutierrez> | 
  Depool dns4001 for server restart - T198215 | 
  [production] | 
            
  | 05:08 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1090:3312 for alter table (duration: 01m 06s) | 
  [production] | 
            
  | 05:08 | 
  <marostegui> | 
  Deploy schema change on db1090:3312 T191316 T192926 T89737 T195193 | 
  [production] | 
            
  | 04:47 | 
  <mholloway-shell@deploy1001> | 
  Finished deploy [mobileapps/deploy@2207b66]: Update mobileapps to d7221ba (duration: 05m 42s) | 
  [production] | 
            
  | 04:42 | 
  <mholloway-shell@deploy1001> | 
  Started deploy [mobileapps/deploy@2207b66]: Update mobileapps to d7221ba | 
  [production] | 
            
  | 03:04 | 
  <l10nupdate@deploy1001> | 
  ResourceLoader cache refresh completed at Wed Jun 27 03:04:07 UTC 2018 (duration 10m 32s) | 
  [production] | 
            
  | 02:53 | 
  <l10nupdate@deploy1001> | 
  scap sync-l10n completed (1.32.0-wmf.10) (duration: 15m 53s) | 
  [production] | 
            
  | 02:20 | 
  <l10nupdate@deploy1001> | 
  scap sync-l10n completed (1.32.0-wmf.8) (duration: 08m 33s) | 
  [production] | 
            
  
    | 
      
        2018-06-26
      
      §
     | 
  
    
  | 21:26 | 
  <dduvall@deploy1001> | 
  rebuilt and synchronized wikiversions files: Group0 to 1.32.0-wmf.10 | 
  [production] | 
            
  | 21:18 | 
  <dduvall@deploy1001> | 
  Finished scap: testwiki to php-1.32.0-wmf.10 and rebuild l10n cache (duration: 22m 50s) | 
  [production] | 
            
  | 20:55 | 
  <dduvall@deploy1001> | 
  Started scap: testwiki to php-1.32.0-wmf.10 and rebuild l10n cache | 
  [production] | 
            
  | 20:47 | 
  <marxarelli> | 
  removing orphaned /srv/mediawiki/php-1.31.0-wmf.28 dir on deploy1001 | 
  [production] | 
            
  | 20:38 | 
  <volans> | 
  regenerated debian installer for jessie 8.11 point release (released 3 days ago) | 
  [production] | 
            
  | 20:34 | 
  <marxarelli> | 
  removed /srv/mediawiki/php-1.32.0-wmf.4 on mwdebug2002 to free disk space | 
  [production] | 
            
  | 20:22 | 
  <marxarelli> | 
  manually cleaning up on mwdebug2002 due to full disk and scap failures | 
  [production] | 
            
  | 20:21 | 
  <dduvall@deploy1001> | 
  Pruned MediaWiki: 1.32.0-wmf.4 [keeping static files] (duration: 03m 21s) | 
  [production] | 
            
  | 19:43 | 
  <marxarelli> | 
  cdb-rebuild failed on mwdebug2002 due to lack of disk space | 
  [production] | 
            
  | 19:41 | 
  <dduvall@deploy1001> | 
  Finished scap: testwiki to php-1.32.0-wmf.10 and rebuild l10n cache (duration: 39m 11s) | 
  [production] | 
            
  | 19:02 | 
  <dduvall@deploy1001> | 
  Started scap: testwiki to php-1.32.0-wmf.10 and rebuild l10n cache | 
  [production] | 
            
  | 18:39 | 
  <marxarelli> | 
  setting $wmgUseMwEmbedSupport = false in php-1.32.0-wmf.10/LocalSettings.php to extension registry exception (see T197918 and T191056) | 
  [production] | 
            
  | 18:14 | 
  <marxarelli> | 
  scap sync for testwiki is currently failing on l10update due to MwEmbedSupport's deprecation but no change yet to mediawiki/extensions to remove the extension. blocking train on T197918 | 
  [production] | 
            
  | 18:01 | 
  <dduvall@deploy1001> | 
  scap failed: CalledProcessError Command '/usr/local/bin/mwscript rebuildLocalisationCache.php --wiki="testwiki" --outdir="/tmp/scap_l10n_2905568126" --threads=30 --lang en  --quiet' returned non-zero exit status 255 (duration: 02m 23s) | 
  [production] | 
            
  | 17:59 | 
  <dduvall@deploy1001> | 
  Started scap: testwiki to php-1.32.0-wmf.10 and rebuild l10n cache | 
  [production] | 
            
  | 17:16 | 
  <marxarelli> | 
  starting branch cut for 1.32-wmf.10 | 
  [production] | 
            
  | 16:21 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1105:3312 after alter table (duration: 00m 57s) | 
  [production] | 
            
  | 14:43 | 
  <elukey> | 
  rm syslog.1.gz puppet.log.1.gz on tegment to fix cronspam | 
  [production] | 
            
  | 14:19 | 
  <ottomata> | 
  Re-enable job and change-prop Kafka topic mirroring from main-eqiad -> main-codfw | 
  [production] | 
            
  | 13:37 | 
  <hashar> | 
  European swat completed | 
  [production] | 
            
  | 13:36 | 
  <hashar@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: Enable zh-my variant on zhwiki - T193983 (duration: 00m 57s) | 
  [production] | 
            
  | 13:31 | 
  <hashar@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: Use uploaded HD logos for ruwikiquote - T197508 (duration: 00m 57s) | 
  [production] | 
            
  | 13:29 | 
  <hashar> | 
  purgeList.php  for the various ruwikiquote logos | 
  [production] | 
            
  | 13:27 | 
  <hashar@deploy1001> | 
  Synchronized static/images/project-logos: Change logo resources for ruwikiquote - T197508 (duration: 00m 56s) | 
  [production] | 
            
  | 13:25 | 
  <hashar@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: Enable TemplateStyles on ruwikiquote - T197526 (duration: 00m 57s) | 
  [production] | 
            
  | 13:22 | 
  <dcausse> | 
  elastic@eqiad: forcemerge (only_expunge_deletes=true) on wikidatawiki_content | 
  [production] | 
            
  | 13:21 | 
  <dcausse> | 
  elastic@eqiad deleting stale index viwiki_general_1525301649 | 
  [production] | 
            
  | 13:19 | 
  <hashar@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: Create a few of namespace aliases for ruwiktionary - T197565 (duration: 00m 57s) | 
  [production] | 
            
  | 13:06 | 
  <hashar@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: Add some namespace aliases to ruwikiquote - T197058 (duration: 00m 56s) | 
  [production] | 
            
  | 12:54 | 
  <chasemp> | 
  labcontrol1003 aptitude install ldap-utils python-ldappool | 
  [production] | 
            
  | 12:15 | 
  <marostegui> | 
  Deploy schema change on db1105:3312 T191316 T192926 T89737 T195193 | 
  [production] | 
            
  | 12:14 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1105:3312 for alter table (duration: 00m 57s) | 
  [production] | 
            
  | 12:07 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1103:3312 after alter table (duration: 00m 57s) | 
  [production] | 
            
  | 11:09 | 
  <godog> | 
  roll-upgrade thumbor in codfw/eqiad | 
  [production] | 
            
  | 11:08 | 
  <godog> | 
  upload thumbor 2.0-2 to apt.wikimedia.org | 
  [production] | 
            
  | 10:40 | 
  <godog> | 
  upgrade thumbor to 2.0-1 on thumbor1001 | 
  [production] | 
            
  | 10:10 | 
  <godog> | 
  test upgrade thumbor 2.0 on thumbor2001 | 
  [production] |