| 2016-02-26
      
      § | 
    
  | 15:51 | <jynus> | shutting down mariadb on db2030 to clone from db1009 | [production] | 
            
  | 15:24 | <urandom> | forcing puppet run on restbase1009.eqiad.wmnet | [production] | 
            
  | 15:24 | <urandom> | re-enabling puppet on restbase1009.eqiad.wmnet | [production] | 
            
  | 15:20 | <jynus> | performing backup of m5-master mysql data | [production] | 
            
  | 15:17 | <urandom> | blocking CQL native port on restbase1009.eqiad.wmnet : https://phabricator.wikimedia.org/P2677 | [production] | 
            
  | 15:14 | <urandom> | disabling puppet on restbase1009.eqiad to preserve local changes during a quick experiment | [production] | 
            
  | 15:03 | <hashar> | Switched MediaWiki core npm test to Nodepool instance T119143 | [production] | 
            
  | 13:59 | <krinkle@tin> | Synchronized wmf-config/InitialiseSettings.php: T99096: Enable wmgUseWmfstatic on remaining wikis (duration: 00m 50s) | [production] | 
            
  | 13:54 | <moritzm> | rebooting lithium for kernel update | [production] | 
            
  | 13:26 | <godog> | launch swiftrepl continuous replication for unsharded containers on ms-fe1003 T128096 | [production] | 
            
  | 12:31 | <elukey> | added mc1017/mc1018 back to the redis/memcached pools after maintenance | [production] | 
            
  | 11:42 | <godog> | run swiftrepl eqiad -> codfw for unsharded containers | [production] | 
            
  | 11:01 | <elukey> | removed mc1018/1017 from the redis memcached pools for maintenance | [production] | 
            
  | 09:46 | <elukey> | mc1016.eqiad re-added to the memcached/redis pools after maintenance | [production] | 
            
  | 08:12 | <elukey> | removed mc1016.eqiad from the redis/memcached pools for maintenance | [production] | 
            
  | 08:01 | <moritzm> | blacklisting aufs kernel module | [production] | 
            
  | 02:32 | <l10nupdate@tin> | ResourceLoader cache refresh completed at Fri Feb 26 02:32:19 UTC 2016 (duration 7m 42s) | [production] | 
            
  | 02:24 | <mwdeploy@tin> | sync-l10n completed (1.27.0-wmf.14) (duration: 10m 34s) | [production] | 
            
  | 01:06 | <catrope@tin> | Synchronized wmf-config/InitialiseSettings.php: Lower survey rate again (duration: 01m 05s) | [production] | 
            
  | 00:33 | <catrope@tin> | Synchronized php-1.27.0-wmf.14/extensions/MobileFrontend/: SWAT (duration: 01m 05s) | [production] | 
            
  | 00:31 | <catrope@tin> | Synchronized wmf-config/InitialiseSettings.php: Raise file upload limit to 2047MB (duration: 01m 02s) | [production] | 
            
  | 00:22 | <catrope@tin> | Synchronized wmf-config/CommonSettings.php: Add plumbing for wmgUseGraphWithJsonNamespace (duration: 01m 03s) | [production] | 
            
  | 00:21 | <catrope@tin> | Synchronized wmf-config/InitialiseSettings.php: Add wmgUseGraphWithJsonNamespace (duration: 01m 04s) | [production] | 
            
  | 00:15 | <catrope@tin> | Synchronized wmf-config/InitialiseSettings.php: Run reader segmentation survey at 1:500 to test DNT (duration: 01m 21s) | [production] | 
            
  
    | 2016-02-25
      
      § | 
    
  | 23:16 | <bblack> | turning puppet back on for cp*, pushing changes through https://gerrit.wikimedia.org/r/#/c/273385/ | [production] | 
            
  | 22:33 | <bblack> | disabling puppet on caches for more scary VCL merges | [production] | 
            
  | 22:33 | <ori@tin> | Synchronized php-1.27.0-wmf.14/extensions/CentralAuth: I2cfcbf98f3: Reduce memcache traffic for central session storage (duration: 01m 21s) | [production] | 
            
  | 22:06 | <bblack> | turning puppet back on for cp*, pushing changes through https://gerrit.wikimedia.org/r/273217 to all | [production] | 
            
  | 21:59 | <thcipriani@tin> | Synchronized php-1.27.0-wmf.14/extensions/CirrusSearch/includes/InterwikiSearcher.php: Fix undefined variable $term in InterwikiSearcher [[gerrit:273369]] (duration: 01m 08s) | [production] | 
            
  | 21:49 | <ori> | Upgraded Grafana to v3.0.0-pre1. | [production] | 
            
  | 21:28 | <thcipriani@tin> | rebuilt wikiversions.php and synchronized wikiversions files: all wikis to 1.27.0-wmf.14 | [production] | 
            
  | 21:21 | <ebernhardson@tin> | Synchronized php-1.27.0-wmf.14/extensions/CirrusSearch/includes/Searcher.php: Update file that wasnt synced properly (duration: 01m 50s) | [production] | 
            
  | 20:57 | <bblack> | disabling puppet on caches for scarier VCL merges | [production] | 
            
  | 20:46 | <urandom> | starting bootstrap of restbase1008-a T119935 | [production] | 
            
  | 20:31 | <thcipriani@tin> | rebuilt wikiversions.php and synchronized wikiversions files: rollback wmf.14 | [production] | 
            
  | 20:04 | <thcipriani@tin> | rebuilt wikiversions.php and synchronized wikiversions files: all wikis to 1.27.0-wmf.14 | [production] | 
            
  | 19:13 | <elukey> | added mc1015.eqiad.wmnet back to the redis/memcached pools | [production] | 
            
  | 18:58 | <jynus> | testing schema changes on db2057:testwiki (s3) | [production] | 
            
  | 18:07 | <jynus> | testing schema change on db2070 (enwiki) | [production] | 
            
  | 17:59 | <ori@tin> | Synchronized php-1.27.0-wmf.13/includes/session/SessionBackend.php: I43cde3a48: Revert Revert SessionBackend: skip isUserSessionPrevented check for anons (duration: 01m 38s) | [production] | 
            
  | 17:57 | <ori@tin> | Synchronized php-1.27.0-wmf.14/includes/session/SessionBackend.php: I43cde3a48: Revert Revert SessionBackend: skip isUserSessionPrevented check for anons (duration: 01m 39s) | [production] | 
            
  | 17:57 | <godog> | bounce cassandra instances on restbase2001, cql not listening | [production] | 
            
  | 17:37 | <elukey> | removed mc1015 from the redis/memcached pool for maintenance | [production] | 
            
  | 17:37 | <thcipriani@tin> | Synchronized README: test sync-file (duration: 01m 42s) | [production] | 
            
  | 17:32 | <thcipriani@tin> | Synchronized README: test sync-file (duration: 01m 46s) | [production] | 
            
  | 17:11 | <moritzm> | installing xerces-c security updates | [production] | 
            
  | 17:08 | <thcipriani@tin> | Synchronized php-1.27.0-wmf.13/extensions/QuickSurveys/resources/ext.quicksurveys.init/init.js: SWAT (after touch): Do not show a survey if DNT is enabled [[gerrit:273268]] (duration: 01m 32s) | [production] | 
            
  | 17:04 | <jynus> | applying schema change to flowdb (x1) | [production] | 
            
  | 16:59 | <thcipriani@tin> | Synchronized php-1.27.0-wmf.13/extensions/QuickSurveys/resources/ext.quicksurveys.init/init.js: SWAT: Do not show a survey if DNT is enabled [[gerrit:273268]] (duration: 01m 31s) | [production] | 
            
  | 16:56 | <jynus> | applying schema change to officewiki:flow (s3) | [production] |