| 
      
        2017-07-10
      
      ยง
     | 
  
    
  | 16:14 | 
  <nuria@tin> | 
  Finished deploy [eventlogging/analytics@5e16da1]: (no justification provided) (duration: 00m 04s) | 
  [production] | 
            
  | 16:14 | 
  <nuria@tin> | 
  Started deploy [eventlogging/analytics@5e16da1]: (no justification provided) | 
  [production] | 
            
  | 15:56 | 
  <elukey@tin> | 
  Finished deploy [analytics/refinery@6da2774]: Update stat1002 with the last refinery deployment (duration: 00m 04s) | 
  [production] | 
            
  | 15:55 | 
  <elukey@tin> | 
  Started deploy [analytics/refinery@6da2774]: Update stat1002 with the last refinery deployment | 
  [production] | 
            
  | 15:47 | 
  <milimetric@tin> | 
  Finished deploy [analytics/refinery@6da2774]: Update Sqoop fix python error (duration: 00m 07s) | 
  [production] | 
            
  | 15:47 | 
  <milimetric@tin> | 
  Started deploy [analytics/refinery@6da2774]: Update Sqoop fix python error | 
  [production] | 
            
  | 15:43 | 
  <milimetric@tin> | 
  Finished deploy [analytics/refinery@6da2774]: Update Sqoop fix python error (duration: 01m 36s) | 
  [production] | 
            
  | 15:42 | 
  <milimetric@tin> | 
  Started deploy [analytics/refinery@6da2774]: Update Sqoop fix python error | 
  [production] | 
            
  | 15:37 | 
  <milimetric@tin> | 
  Finished deploy [analytics/refinery@6da2774]: Update Sqoop fix python error (duration: 02m 06s) | 
  [production] | 
            
  | 15:35 | 
  <milimetric@tin> | 
  Started deploy [analytics/refinery@6da2774]: Update Sqoop fix python error | 
  [production] | 
            
  | 15:24 | 
  <andrewbogott> | 
  adding two new hosts (labvirt1014 and labvirt1015) to the nova-compute scheduling pool.  Possible nodepool side-effects, maybe good ones? | 
  [production] | 
            
  | 15:14 | 
  <marostegui> | 
  Drop ukwikimedia_p views from labsdb hosts - T169488 | 
  [production] | 
            
  | 15:00 | 
  <moritzm> | 
  installing apache security updates on app server canaries | 
  [production] | 
            
  | 14:40 | 
  <andrewbogott> | 
  rebooting labvirt1015-1018 for kernel updates | 
  [production] | 
            
  | 14:23 | 
  <marostegui> | 
  Deploy alter table on db1059 - T168661 | 
  [production] | 
            
  | 14:22 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1059 - T168661 (duration: 00m 41s) | 
  [production] | 
            
  | 14:20 | 
  <marostegui@tin> | 
  scap failed: average error rate on 1/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/3888cca979647b9381a7739b0bdbc88e for details) | 
  [production] | 
            
  | 14:14 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Repool db1091 - T168661 (duration: 00m 42s) | 
  [production] | 
            
  | 14:13 | 
  <moritzm> | 
  reimaging mw2118 (video scaler) to jessie | 
  [production] | 
            
  | 14:12 | 
  <TabbyCat> | 
  mwscript updateCollation.php --wiki=frwiktionary --previous-collation=uppercase is being running by zfilipin to finish T169810 | 
  [production] | 
            
  | 14:11 | 
  <zeljkof> | 
  EU SWAT finished | 
  [production] | 
            
  | 14:01 | 
  <dcausse> | 
  elastic@eqiad unbanning elastic1018 & elastic1021 | 
  [production] | 
            
  | 13:49 | 
  <chasemp> | 
  labstore2003:~# umount -fl /srv/backup/tools (for T169774 recovery) | 
  [production] | 
            
  | 13:44 | 
  <zfilipin@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:363740|Set $wgCategoryCollation to uca-default for fr.wiktionary (T169810)]] (duration: 00m 42s) | 
  [production] | 
            
  | 13:43 | 
  <dcausse> | 
  elastic@eqiad banning elastic1018 & elastic1021 to rebalance heavy shards | 
  [production] | 
            
  | 13:32 | 
  <zfilipin@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:364131|Add import sources for specieswiki (T170094)]] (duration: 00m 43s) | 
  [production] | 
            
  | 13:29 | 
  <moritzm> | 
  installing graphite2 security updates (image lib) | 
  [production] | 
            
  | 13:28 | 
  <marostegui> | 
  Disable puppet on db1102 to run check_private_data - T153743 | 
  [production] | 
            
  | 13:20 | 
  <zfilipin@tin> | 
  Synchronized wmf-config/CirrusSearch-common.php: SWAT: [[gerrit:364183|[cirrus] Enable the token_count_router only for chinese (T169498)]] (duration: 00m 43s) | 
  [production] | 
            
  | 13:06 | 
  <milimetric@tin> | 
  Finished deploy [analytics/refinery@c22eb93]: Update Sqoop with better parallelism (duration: 02m 54s) | 
  [production] | 
            
  | 13:04 | 
  <milimetric@tin> | 
  Started deploy [analytics/refinery@c22eb93]: Update Sqoop with better parallelism | 
  [production] | 
            
  | 12:58 | 
  <marostegui> | 
  Run redact_sanitarium on s2 and s6 - db1102 - T153743 | 
  [production] | 
            
  | 12:26 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: db1079 to become master for sanitarium3 - T153743 (duration: 00m 41s) | 
  [production] | 
            
  | 12:23 | 
  <marostegui@tin> | 
  scap failed: average error rate on 2/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/3888cca979647b9381a7739b0bdbc88e for details) | 
  [production] | 
            
  | 12:22 | 
  <moritzm> | 
  installing xorg-server security updates | 
  [production] | 
            
  | 12:22 | 
  <marostegui@tin> | 
  scap failed: average error rate on 4/11 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/3888cca979647b9381a7739b0bdbc88e for details) | 
  [production] | 
            
  | 12:12 | 
  <marostegui> | 
  Deploy alter table on db1091 - T168661 | 
  [production] | 
            
  | 12:11 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Depool db1091 - T168661 (duration: 00m 42s) | 
  [production] | 
            
  | 12:02 | 
  <marostegui> | 
  Upgrade db1102 to 10.1 and enable rbr triggers - T153743 | 
  [production] | 
            
  | 12:02 | 
  <moritzm> | 
  installing bind security updates (we only have client libs/tools installed) | 
  [production] | 
            
  | 11:39 | 
  <marostegui@tin> | 
  Synchronized wmf-config/db-eqiad.php: Restore original weight for db1097 - T168661 (duration: 00m 42s) | 
  [production] | 
            
  | 11:36 | 
  <marostegui> | 
  Stop MySQL on db1102 for maintenance - T153743 | 
  [production] | 
            
  | 11:16 | 
  <moritzm> | 
  installing libgcrypt and expat security updates | 
  [production] | 
            
  | 11:16 | 
  <kartik@tin> | 
  Finished deploy [cxserver/deploy@c209bec]: Update cxserver to 3375da5 (duration: 02m 49s) | 
  [production] | 
            
  | 11:13 | 
  <kartik@tin> | 
  Started deploy [cxserver/deploy@c209bec]: Update cxserver to 3375da5 | 
  [production] | 
            
  | 10:28 | 
  <addshore> | 
  WMDE Summer campaign deploy slot DONE | 
  [production] | 
            
  | 10:28 | 
  <addshore@tin> | 
  Synchronized wmf-config/InitialiseSettings.php: [[gerrit:364184|WMDE Summer campaign - Add logging]] (fix spacing) NOOP (duration: 00m 43s) | 
  [production] | 
            
  | 10:22 | 
  <addshore@tin> | 
  Synchronized php-1.30.0-wmf.7/extensions/WikimediaEvents: [[gerrit:364172|WMDE Summer campaign - Add hook]] (duration: 00m 42s) | 
  [production] | 
            
  | 10:21 | 
  <addshore@tin> | 
  Synchronized php-1.30.0-wmf.7/extensions/CentralAuth: CentralAuth (undeployed patches) [[gerrit:363892]], [[gerrit:363893]], [[gerrit:363891]] & revert [[gerrit:364182]] T169261 (duration: 00m 47s) | 
  [production] | 
            
  | 10:13 | 
  <addshore> | 
  reverting https://gerrit.wikimedia.org/r/#/c/363891 as it is sitting on tin undeployed T169261 | 
  [production] |