| 2016-02-25
      
      § | 
    
  | 23:16 | <bblack> | turning puppet back on for cp*, pushing changes through https://gerrit.wikimedia.org/r/#/c/273385/ | [production] | 
            
  | 22:33 | <bblack> | disabling puppet on caches for more scary VCL merges | [production] | 
            
  | 22:33 | <ori@tin> | Synchronized php-1.27.0-wmf.14/extensions/CentralAuth: I2cfcbf98f3: Reduce memcache traffic for central session storage (duration: 01m 21s) | [production] | 
            
  | 22:06 | <bblack> | turning puppet back on for cp*, pushing changes through https://gerrit.wikimedia.org/r/273217 to all | [production] | 
            
  | 21:59 | <thcipriani@tin> | Synchronized php-1.27.0-wmf.14/extensions/CirrusSearch/includes/InterwikiSearcher.php: Fix undefined variable $term in InterwikiSearcher [[gerrit:273369]] (duration: 01m 08s) | [production] | 
            
  | 21:49 | <ori> | Upgraded Grafana to v3.0.0-pre1. | [production] | 
            
  | 21:28 | <thcipriani@tin> | rebuilt wikiversions.php and synchronized wikiversions files: all wikis to 1.27.0-wmf.14 | [production] | 
            
  | 21:21 | <ebernhardson@tin> | Synchronized php-1.27.0-wmf.14/extensions/CirrusSearch/includes/Searcher.php: Update file that wasnt synced properly (duration: 01m 50s) | [production] | 
            
  | 20:57 | <bblack> | disabling puppet on caches for scarier VCL merges | [production] | 
            
  | 20:46 | <urandom> | starting bootstrap of restbase1008-a T119935 | [production] | 
            
  | 20:31 | <thcipriani@tin> | rebuilt wikiversions.php and synchronized wikiversions files: rollback wmf.14 | [production] | 
            
  | 20:04 | <thcipriani@tin> | rebuilt wikiversions.php and synchronized wikiversions files: all wikis to 1.27.0-wmf.14 | [production] | 
            
  | 19:13 | <elukey> | added mc1015.eqiad.wmnet back to the redis/memcached pools | [production] | 
            
  | 18:58 | <jynus> | testing schema changes on db2057:testwiki (s3) | [production] | 
            
  | 18:07 | <jynus> | testing schema change on db2070 (enwiki) | [production] | 
            
  | 17:59 | <ori@tin> | Synchronized php-1.27.0-wmf.13/includes/session/SessionBackend.php: I43cde3a48: Revert Revert SessionBackend: skip isUserSessionPrevented check for anons (duration: 01m 38s) | [production] | 
            
  | 17:57 | <ori@tin> | Synchronized php-1.27.0-wmf.14/includes/session/SessionBackend.php: I43cde3a48: Revert Revert SessionBackend: skip isUserSessionPrevented check for anons (duration: 01m 39s) | [production] | 
            
  | 17:57 | <godog> | bounce cassandra instances on restbase2001, cql not listening | [production] | 
            
  | 17:37 | <elukey> | removed mc1015 from the redis/memcached pool for maintenance | [production] | 
            
  | 17:37 | <thcipriani@tin> | Synchronized README: test sync-file (duration: 01m 42s) | [production] | 
            
  | 17:32 | <thcipriani@tin> | Synchronized README: test sync-file (duration: 01m 46s) | [production] | 
            
  | 17:11 | <moritzm> | installing xerces-c security updates | [production] | 
            
  | 17:08 | <thcipriani@tin> | Synchronized php-1.27.0-wmf.13/extensions/QuickSurveys/resources/ext.quicksurveys.init/init.js: SWAT (after touch): Do not show a survey if DNT is enabled [[gerrit:273268]] (duration: 01m 32s) | [production] | 
            
  | 17:04 | <jynus> | applying schema change to flowdb (x1) | [production] | 
            
  | 16:59 | <thcipriani@tin> | Synchronized php-1.27.0-wmf.13/extensions/QuickSurveys/resources/ext.quicksurveys.init/init.js: SWAT: Do not show a survey if DNT is enabled [[gerrit:273268]] (duration: 01m 31s) | [production] | 
            
  | 16:56 | <jynus> | applying schema change to officewiki:flow (s3) | [production] | 
            
  | 16:51 | <thcipriani@tin> | Synchronized php-1.27.0-wmf.14/extensions/QuickSurveys/resources/ext.quicksurveys.init/init.js: SWAT: Do not show a survey if DNT is enabled [[gerrit:273267]] (duration: 01m 35s) | [production] | 
            
  | 16:23 | <thcipriani@tin> | rebuilt wikiversions.php and synchronized wikiversions files: SWAT: wikidatawiki to wmf.14 | [production] | 
            
  | 16:16 | <elukey> | added mc1014 back to the redis/memcached pool after maintenance | [production] | 
            
  | 16:16 | <thcipriani@tin> | Synchronized php-1.27.0-wmf.14/resources/lib/oojs-ui/oojs-ui-core.js: SWAT: OOjs UI: Fix #gatherPreInfuseState called incorrectly, causing TypeErrors [[gerrit:273250]] (duration: 01m 42s) | [production] | 
            
  | 15:32 | <godog> | stop cassandra/restbase on restbase2001 to finish raid0 grow | [production] | 
            
  | 15:21 | <demon@tin> | rebuilt wikiversions.php and synchronized wikiversions files: wikidata back to wmf.13 for now | [production] | 
            
  | 11:47 | <hashar> | Reverting session manager cherry picks from wmf branches ( https://gerrit.wikimedia.org/r/#/c/273201/ and https://gerrit.wikimedia.org/r/#/c/273202/ ) they have not been deployed after they got merged | [production] | 
            
  | 11:33 | <godog> | depool ms-fe1004 for trusty dist-upgrade | [production] | 
            
  | 11:23 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Repool db1021 and db1024 (duration: 01m 45s) | [production] | 
            
  | 10:58 | <moritzm> | powercycling cp2010 | [production] | 
            
  | 10:41 | <godog> | set xff to 0.01 for graphite metrics swift.*.containers (was 0.5) | [production] | 
            
  | 10:03 | <hashar> | starting Jenkins | [production] | 
            
  | 09:57 | <hashar> | Stopping Jenkins | [production] | 
            
  | 09:29 | <elukey> | removed mc1014.eqiad from the redis/memcached pool for maintenance | [production] | 
            
  | 04:44 | <urandom> | decommissioning Cassandra on restbase1008-a.eqiad.wmnet T119935 | [production] | 
            
  | 04:35 | <urandom> | restarting restbase1008-a to cancel rebuild T108611 T119935 | [production] | 
            
  | 03:19 | <l10nupdate@tin> | ResourceLoader cache refresh completed at Thu Feb 25 03:19:30 UTC 2016 (duration 9m 6s) | [production] | 
            
  | 03:10 | <mwdeploy@tin> | sync-l10n completed (1.27.0-wmf.14) (duration: 18m 00s) | [production] | 
            
  | 02:35 | <mwdeploy@tin> | sync-l10n completed (1.27.0-wmf.13) (duration: 19m 03s) | [production] | 
            
  | 00:57 | <bd808> | Started crashed Logstash process on logstash1002 (systemd doesn't restart authomatically due to T127677) | [production] | 
            
  | 00:03 | <ori@tin> | Synchronized wmf-config/CommonSettings.php: I4cc836f3ca: Fully-qualify EventLoggingBaseUri (duration: 01m 40s) | [production] |