| 
      
        2018-05-23
      
      §
     | 
  
    
  | 08:18 | 
  <addshore> | 
  addshore@wasat:~$ mwscript extensions/CirrusSearch/maintenance/updateSearchIndexConfig.php --wiki testwikidatawiki --reindexAndRemoveOk --indexIdentifier=now | 
  [production] | 
            
  | 08:18 | 
  <addshore> | 
  addshore@terbium:~$ mwscript extensions/CirrusSearch/maintenance/updateSearchIndexConfig.php --wiki testwikidatawiki --reindexAndRemoveOk --indexIdentifier=now | 
  [production] | 
            
  | 08:17 | 
  <addshore@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: [[gerrit:434592|testwikidata: Add Lexeme NS to wgNamespacesToBeSearchedDefault]] (duration: 01m 19s) | 
  [production] | 
            
  | 08:06 | 
  <addshore> | 
  addshore@wasat:~$ mwscript extensions/CirrusSearch/maintenance/updateSearchIndexConfig.php --wiki testwikidatawiki --reindexAndRemoveOk --indexIdentifier=now --reindexProcesses=10 | 
  [production] | 
            
  | 08:00 | 
  <moritzm> | 
  upgrading remaining app servers in eqiad to HHVM 3,18.5+dfsg-1+wmf8+deb9u1 | 
  [production] | 
            
  | 07:54 | 
  <addshore> | 
  addshore@terbium:~$ mwscript extensions/CirrusSearch/maintenance/updateSearchIndexConfig.php --wiki testwikidatawiki --reindexAndRemoveOk --indexIdentifier=now | 
  [production] | 
            
  | 07:52 | 
  <addshore@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: [[gerrit:434591|testwikidata: Add Property NS to wgNamespacesToBeSearchedDefault]] (duration: 01m 20s) | 
  [production] | 
            
  | 07:45 | 
  <addshore@tin> | 
  Synchronized php-1.32.0-wmf.5/extensions/WikibaseLexeme: [[gerrit:434602|Use ISO code und for missing language code]] (duration: 01m 30s) | 
  [production] | 
            
  | 07:44 | 
  <addshore@tin> | 
  Synchronized php-1.32.0-wmf.4/extensions/WikibaseLexeme: [[gerrit:434601|Use ISO code und for missing language code]] (duration: 01m 31s) | 
  [production] | 
            
  | 07:38 | 
  <moritzm> | 
  upgrading remaining API servers in eqiad to HHVM 3,18.5+dfsg-1+wmf8+deb9u1 | 
  [production] | 
            
  | 07:29 | 
  <elukey> | 
  upload druid debs 0.11.0-3 to stretch-wikimedia | 
  [production] | 
            
  | 06:52 | 
  <moritzm> | 
  upgrading mw1299-mw1306 (job runners) to HHVM 3,18.5+dfsg-1+wmf8+deb9u1 | 
  [production] | 
            
  | 06:49 | 
  <marostegui> | 
  Re-add indexes on wb_terms on db1092 - T194273 | 
  [production] | 
            
  | 06:44 | 
  <elukey> | 
  restart zookeeper on druid100[4-6] for openjdk-8 upgrades | 
  [production] | 
            
  | 06:44 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1092 for alter table (duration: 01m 20s) | 
  [production] | 
            
  | 06:22 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1092 after alter table (duration: 01m 20s) | 
  [production] | 
            
  | 05:43 | 
  <marostegui> | 
  Deploy schema change on db1092 - T191519 T188299 T190148 T194273 T194270 | 
  [production] | 
            
  | 05:43 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1092 for alter table (duration: 01m 27s) | 
  [production] | 
            
  | 05:21 | 
  <marostegui> | 
  Deploy schema change on dbstore1002:s8 - T191519 T188299 T190148 T194273 T194270 | 
  [production] | 
            
  | 05:12 | 
  <marostegui> | 
  Deploy schema change on s8 codfw primary master (db2045), this will generate lag on codfw - T194273 | 
  [production] | 
            
  | 05:06 | 
  <marostegui> | 
  Deploy schema change on s3 primary master (db1075) - T191519 T188299 T190148 | 
  [production] | 
            
  | 04:04 | 
  <l10nupdate@tin> | 
  ResourceLoader cache refresh completed at Wed May 23 04:04:43 UTC 2018 (duration 7m 22s) | 
  [production] | 
            
  | 03:57 | 
  <l10nupdate@tin> | 
  scap sync-l10n completed (1.32.0-wmf.5) (duration: 16m 39s) | 
  [production] | 
            
  | 02:58 | 
  <l10nupdate@tin> | 
  scap sync-l10n completed (1.32.0-wmf.4) (duration: 10m 54s) | 
  [production] | 
            
  
    | 
      
        2018-05-22
      
      §
     | 
  
    
  | 23:29 | 
  <maxsem@tin> | 
  Synchronized dblists/categories-rdf.dblist: https://gerrit.wikimedia.org/r/#/c/433740/ (duration: 01m 17s) | 
  [production] | 
            
  | 22:59 | 
  <twentyafterfour> | 
  train for group0 1.32.0-wmf.5 completed. Tune in tomorrow for more excitement! | 
  [production] | 
            
  | 22:58 | 
  <twentyafterfour@tin> | 
  rebuilt and synchronized wikiversions files: group0 wikis to 1.32.0-wmf.5  refs T191051 | 
  [production] | 
            
  | 22:54 | 
  <twentyafterfour@tin> | 
  Finished scap: testwikis wikis to 1.32.0-wmf.5  refs T191051 (duration: 76m 01s) | 
  [production] | 
            
  | 21:38 | 
  <twentyafterfour@tin> | 
  Started scap: testwikis wikis to 1.32.0-wmf.5  refs T191051 | 
  [production] | 
            
  | 21:13 | 
  <twentyafterfour@tin> | 
  scap failed: CalledProcessError Command '/usr/local/bin/mwscript rebuildLocalisationCache.php --wiki="testwiki" --outdir="/tmp/scap_l10n_2833418486" --threads=10 --lang en  --quiet' returned non-zero exit status 255 (duration: 39m 54s) | 
  [production] | 
            
  | 21:12 | 
  <ejegg> | 
  updated CiviCRM from 5b8c868a00 to 4d797fc592 | 
  [production] | 
            
  | 20:33 | 
  <twentyafterfour@tin> | 
  Started scap: testwikis wikis to 1.32.0-wmf.5  refs T191051 | 
  [production] | 
            
  | 19:45 | 
  <pnorman@tin> | 
  Finished deploy [tilerator/deploy@18faaa6]: Update tilerator, fix variable substitution (duration: 02m 46s) | 
  [production] | 
            
  | 19:43 | 
  <pnorman@tin> | 
  Started deploy [tilerator/deploy@18faaa6]: Update tilerator, fix variable substitution | 
  [production] | 
            
  | 19:39 | 
  <pnorman@tin> | 
  Finished deploy [tilerator/deploy@18faaa6]: (no justification provided) (duration: 00m 29s) | 
  [production] | 
            
  | 19:38 | 
  <pnorman@tin> | 
  Started deploy [tilerator/deploy@18faaa6]: (no justification provided) | 
  [production] | 
            
  | 19:35 | 
  <pnorman@tin> | 
  Finished deploy [tilerator/deploy@18faaa6]: (no justification provided) (duration: 00m 25s) | 
  [production] | 
            
  | 19:35 | 
  <pnorman@tin> | 
  Started deploy [tilerator/deploy@18faaa6]: (no justification provided) | 
  [production] | 
            
  | 19:34 | 
  <pnorman@tin> | 
  Finished deploy [tilerator/deploy@a4a3fc7]: (no justification provided) (duration: 46m 19s) | 
  [production] | 
            
  | 19:17 | 
  <twentyafterfour@tin> | 
  Synchronized php-1.32.0-wmf.4/extensions/ContentTranslation/: sync https://gerrit.wikimedia.org/r/#/c/434529/ to fix T194810 (duration: 01m 02s) | 
  [production] | 
            
  | 19:14 | 
  <twentyafterfour@tin> | 
  rebuilt and synchronized wikiversions files: group2 wikis to 1.32.0-wmf.4  refs T191050 | 
  [production] | 
            
  | 19:03 | 
  <twentyafterfour> | 
  Today's train: Promoting group2 wikis to 1.32.0-wmf.4 followed by group0 to 1.32.0-wmf.5 | 
  [production] | 
            
  | 18:48 | 
  <pnorman@tin> | 
  Started deploy [tilerator/deploy@a4a3fc7]: (no justification provided) | 
  [production] | 
            
  | 18:44 | 
  <pnorman> | 
  tilerator deploying a4a3fc7b2694a469cc26a6d0cc440d4d46fa9485 | 
  [production] | 
            
  | 17:03 | 
  <marostegui> | 
  Deploy schema change on s8 codfw primary master (db2045), this will generate lag on codfw - T194270 | 
  [production] | 
            
  | 16:52 | 
  <elukey> | 
  restart zookeeper on druid100[1,3] to complete the openjdk-8 upgrade | 
  [production] | 
            
  | 16:51 | 
  <addshore> | 
  addshore@terbium:~$ mwscript extensions/CirrusSearch/maintenance/updateSearchIndexConfig.php --wiki testwikidatawiki --reindexAndRemoveOk --indexIdentifier=now | 
  [production] | 
            
  | 16:43 | 
  <elukey> | 
  upload druid debs 0.11.0-3 to jessie-wikimedia | 
  [production] | 
            
  | 16:25 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-codfw.php: Remove db2064 from config - T195228 (duration: 01m 18s) | 
  [production] | 
            
  | 16:24 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Remove db2064 from config - T195228 (duration: 01m 19s) | 
  [production] |