| 
      
        2018-03-29
      
      §
     | 
  
    
  | 23:37 | 
  <ebernhardson@tin> | 
  Synchronized php-1.31.0-wmf.26/extensions/WikimediaEvents/modules/all/ext.wikimediaEvents.searchSatisfaction.js: SWAT: T187148: Start cirrus AB test (duration: 01m 16s) | 
  [production] | 
            
  | 23:12 | 
  <ebernhardson@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: T187148: Configure 5 buckets for cirrus AB test (duration: 01m 17s) | 
  [production] | 
            
  | 22:10 | 
  <andrew@tin> | 
  Finished deploy [horizon/deploy@14d3e7d]: Updating Horizon with possible fix for T189706 (duration: 03m 16s) | 
  [production] | 
            
  | 22:06 | 
  <andrew@tin> | 
  Started deploy [horizon/deploy@14d3e7d]: Updating Horizon with possible fix for T189706 | 
  [production] | 
            
  | 20:07 | 
  <robh> | 
  shuttdown cp2022 for hw testing | 
  [production] | 
            
  | 18:49 | 
  <maxsem@tin> | 
  Synchronized php-1.31.0-wmf.27/skins/MinervaNeue: https://gerrit.wikimedia.org/r/#/c/423012/ (duration: 01m 17s) | 
  [production] | 
            
  | 18:27 | 
  <maxsem@tin> | 
  Synchronized php-1.31.0-wmf.26/includes/: Shorten summary length to 500 (duration: 02m 06s) | 
  [production] | 
            
  | 18:22 | 
  <maxsem@tin> | 
  Synchronized php-1.31.0-wmf.27/includes/: Shorten summary length to 500 (duration: 02m 14s) | 
  [production] | 
            
  | 17:55 | 
  <dcausse> | 
  pausing restarts of elastic@codfw (6 nodes left) | 
  [production] | 
            
  | 17:35 | 
  <mobrovac@tin> | 
  Finished deploy [restbase/deploy@af592d6]: Add bawikibooks - T191033 (duration: 30m 35s) | 
  [production] | 
            
  | 17:30 | 
  <demon@tin> | 
  Synchronized docroot/wwwportal/w/search-redirect.php: removing symlink indirection (duration: 01m 16s) | 
  [production] | 
            
  | 17:05 | 
  <mobrovac@tin> | 
  Started deploy [restbase/deploy@af592d6]: Add bawikibooks - T191033 | 
  [production] | 
            
  | 14:54 | 
  <mobrovac@tin> | 
  Synchronized wmf-config/jobqueue.php: Cleanup: Use only EventBus for refreshLinks - T185052 (duration: 01m 18s) | 
  [production] | 
            
  | 14:00 | 
  <moritzm> | 
  restarting parsoid and related service on ruthenium to pick up openssl update | 
  [production] | 
            
  | 13:52 | 
  <dcausse> | 
  reverted and rebased tin for undeployed patch due to scap issues (https://gerrit.wikimedia.org/r/#/c/422906/ https://gerrit.wikimedia.org/r/#/c/422929/) | 
  [production] | 
            
  | 13:34 | 
  <dcausse> | 
  aborted scap sync-dir php-1.31.0-wmf.27/extensions/CirrusSearch/ (was taking too much time at: waiting on sync-masters, ok: 1, left: 1) | 
  [production] | 
            
  | 12:54 | 
  <moritzm> | 
  installing ICU security updates on trusty | 
  [production] | 
            
  | 12:29 | 
  <dcausse> | 
  recreating replicas for skwiki_content in elastic@codfw due to stalled shard recovery | 
  [production] | 
            
  | 12:18 | 
  <ariel@tin> | 
  Finished deploy [dumps/dumps@982cebd]: ability to configure production of recombined metacurrent page content file (duration: 00m 02s) | 
  [production] | 
            
  | 12:18 | 
  <ariel@tin> | 
  Started deploy [dumps/dumps@982cebd]: ability to configure production of recombined metacurrent page content file | 
  [production] | 
            
  | 11:02 | 
  <ariel@tin> | 
  Finished deploy [dumps/dumps@96ba844]: cleanup 'latest' links, rss files from old runs (duration: 00m 04s) | 
  [production] | 
            
  | 11:02 | 
  <ariel@tin> | 
  Started deploy [dumps/dumps@96ba844]: cleanup 'latest' links, rss files from old runs | 
  [production] | 
            
  | 10:50 | 
  <dcausse> | 
  restarting elastic@codfw for JVM and plugin upgrade (T189239) | 
  [production] | 
            
  | 09:16 | 
  <elukey> | 
  roll restart aqs on aqs100* for icu/openssl upgrades | 
  [production] | 
            
  | 08:18 | 
  <akosiaris> | 
  T189075 upload apertium_3.5.1-1+wmf1 to apt.wikimedia.org/jessie-wikimedia/main | 
  [production] | 
            
  | 08:18 | 
  <moritzm> | 
  installing OpenJDK security updates on elastic* hosts (along with current version of the search plugins package) | 
  [production] | 
            
  | 08:07 | 
  <elukey> | 
  roll restart of cassandra on aqs* for openjdk-8 upgrades | 
  [production] | 
            
  | 07:20 | 
  <moritzm> | 
  installing openssl security updates | 
  [production] | 
            
  | 07:18 | 
  <ema> | 
  reboot cache@eqiad for retpoline kernel updates: T188092 | 
  [production] | 
            
  | 04:35 | 
  <twentyafterfour> | 
  ran scap pull on deploy1001 | 
  [production] | 
            
  | 02:00 | 
  <l10nupdate@tin> | 
  LocalisationUpdate failed: git pull of core failed | 
  [production] | 
            
  
    | 
      
        2018-03-28
      
      §
     | 
  
    
  | 23:50 | 
  <eileen> | 
  update  civicrm revision changed from 9478ca39f1 to d6855cd281 (further security module updates, engage import dedupe) | 
  [production] | 
            
  | 23:38 | 
  <ebernhardson@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: T187148: Configure next Cirrus AB test (duration: 01m 16s) | 
  [production] | 
            
  | 23:18 | 
  <ebernhardson@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: T184969: Enable PageAssessments on trwiki (duration: 01m 09s) | 
  [production] | 
            
  | 23:13 | 
  <MaxSem> | 
  created PageAssessments tables on trwiki | 
  [production] | 
            
  | 22:17 | 
  <twentyafterfour@tin> | 
  Synchronized php: group1 wikis to 1.31.0-wmf.26 (duration: 01m 18s) | 
  [production] | 
            
  | 22:16 | 
  <twentyafterfour@tin> | 
  rebuilt and synchronized wikiversions files: group1 wikis to 1.31.0-wmf.26 | 
  [production] | 
            
  | 22:13 | 
  <twentyafterfour> | 
  deploy of 1.31.0-wmf.27 resulted in a lot of SlowTimer errors for SlowTimer [10000ms] at runtime/ext_mysql: slow query: SELECT MASTER_GTID_WAIT(...) | 
  [production] | 
            
  | 22:12 | 
  <eileen> | 
  civicrm revision changed from 3f6028b24f to 9478ca39f1 (drupal security update) | 
  [production] | 
            
  | 22:09 | 
  <twentyafterfour@tin> | 
  rebuilt and synchronized wikiversions files: sync https://gerrit.wikimedia.org/r/#/c/422563/ group1 wikis to 1.31.0-wmf.27 refs T183966 T190960 | 
  [production] | 
            
  | 22:08 | 
  <twentyafterfour> | 
  rolling forward group1 to 1.31.0-wmf.27 refs T183966 T190960 | 
  [production] | 
            
  | 22:05 | 
  <twentyafterfour@tin> | 
  Synchronized php-1.31.0-wmf.27/includes/: sync https://gerrit.wikimedia.org/r/#/c/422565/ (duration: 02m 15s) | 
  [production] | 
            
  | 22:03 | 
  <twentyafterfour> | 
  syncing https://gerrit.wikimedia.org/r/#/c/422565/ refs T190960 T183966 | 
  [production] | 
            
  | 21:53 | 
  <mutante> | 
  deploy1001 - revoking old puppet certs and signing new ones | 
  [production] | 
            
  | 21:42 | 
  <twentyafterfour> | 
  getting the train back on track, group1 wikis to 1.31.0-wmf.27 | 
  [production] | 
            
  | 20:51 | 
  <XenoRyet> | 
  updated civicrm from 85c89c7d0a to 3f6028b24f | 
  [production] | 
            
  | 20:50 | 
  <bsitzmann@tin> | 
  Finished deploy [mobileapps/deploy@6a0d877]: Update mobileapps to a5833a0 (duration: 05m 36s) | 
  [production] | 
            
  | 20:44 | 
  <bsitzmann@tin> | 
  Started deploy [mobileapps/deploy@6a0d877]: Update mobileapps to a5833a0 | 
  [production] | 
            
  | 20:12 | 
  <mlitn@tin> | 
  Finished deploy [3d2png/deploy@c447488]: Updating 3d2png (duration: 02m 26s) | 
  [production] | 
            
  | 20:09 | 
  <mlitn@tin> | 
  Started deploy [3d2png/deploy@c447488]: Updating 3d2png | 
  [production] |