| 
      
        2016-01-20
      
      §
     | 
  
    
  | 14:52 | 
  <reedy@tin> | 
  reedy@tin Synchronized php-1.27.0-wmf.11/vendor/: Fix ?PHP properly from commit (duration: 00m 36s) | 
  [production] | 
            
  | 14:50 | 
  <godog> | 
  powercycle mw1123, hhvm oom | 
  [production] | 
            
  | 14:47 | 
  <ema> | 
  Finished reverting migration of mobile traffic to text cluster in codfw https://phabricator.wikimedia.org/T109286 | 
  [production] | 
            
  | 14:24 | 
  <hoo@tin> | 
  hoo@tin Synchronized wmf-config/db-eqiad.php: Set db1045 load to 0 (duration: 00m 32s) | 
  [production] | 
            
  | 14:23 | 
  <reedy@tin> | 
  reedy@tin Synchronized php-1.27.0-wmf.11/: consistency (duration: 02m 38s) | 
  [production] | 
            
  | 14:15 | 
  <hoo@tin> | 
  hoo@tin Synchronized wmf-config/db-eqiad.php: Re-Pool lagged db1045 (duration: 00m 35s) | 
  [production] | 
            
  | 14:14 | 
  <_joe_> | 
  syncronizing /srv/deployment manually between the two deployment servers for the first time | 
  [production] | 
            
  | 14:11 | 
  <hoo@tin> | 
  hoo@tin Synchronized wmf-config/db-eqiad.php: Has not been synced before (duration: 00m 32s) | 
  [production] | 
            
  | 14:07 | 
  <reedy@tin> | 
  reedy@tin Synchronized php-1.27.0-wmf.10/: consistency (duration: 02m 38s) | 
  [production] | 
            
  | 13:58 | 
  <reedy@tin> | 
  reedy@tin Synchronized php-1.27.0-wmf.11/extensions/Validator/: noop for wikitech deploy (duration: 00m 32s) | 
  [production] | 
            
  | 13:58 | 
  <reedy@tin> | 
  reedy@tin Synchronized php-1.27.0-wmf.11/extensions/SemanticMediaWiki/: noop for wikitech deploy (duration: 00m 34s) | 
  [production] | 
            
  | 13:57 | 
  <reedy@tin> | 
  reedy@tin Synchronized php-1.27.0-wmf.11/extensions/SemanticResultFormats/: noop for wikitech deploy (duration: 00m 33s) | 
  [production] | 
            
  | 13:57 | 
  <reedy@tin> | 
  reedy@tin Synchronized php-1.27.0-wmf.10/extensions/SemanticResultFormats/: noop for wikitech deploy (duration: 00m 33s) | 
  [production] | 
            
  | 13:55 | 
  <reedy@tin> | 
  reedy@tin Synchronized php-1.27.0-wmf.10/extensions/SemanticMediaWiki/: noop for wikitech deploy (duration: 00m 33s) | 
  [production] | 
            
  | 13:54 | 
  <reedy@tin> | 
  reedy@tin Synchronized php-1.27.0-wmf.10/extensions/Validator/: noop for wikitech deploy (duration: 00m 33s) | 
  [production] | 
            
  | 13:52 | 
  <jynus@tin> | 
  jynus@tin Synchronized wmf-config/db-eqiad.php: Setting db1049 as API node as db1045 is lagged (duration: 00m 33s) | 
  [production] | 
            
  | 13:41 | 
  <ema> | 
  Revert migration of mobile traffic to text cluster in codfw https://phabricator.wikimedia.org/T109286 | 
  [production] | 
            
  | 12:55 | 
  <akosiaris> | 
  restart hhvm on mw1130 | 
  [production] | 
            
  | 12:43 | 
  <jynus> | 
  performing alter table on db1026 (ETA: 5 hours) | 
  [production] | 
            
  | 12:20 | 
  <jynus@tin> | 
  jynus@tin Synchronized wmf-config/db-eqiad.php: Setting s5 master as recentchanges role (duration: 00m 32s) | 
  [production] | 
            
  | 12:04 | 
  <jynus> | 
  trying schema change on wikidata (wb_terms) | 
  [production] | 
            
  | 09:36 | 
  <akosiaris> | 
  gnt-instance modify -H disk_aio=native cygnus.codfw.wmnet | 
  [production] | 
            
  | 09:18 | 
  <akosiaris> | 
  offline fr_archive volume on nas1001-a | 
  [production] | 
            
  | 09:15 | 
  <akosiaris> | 
  unexport /vol/fr_archive on nas1001-a | 
  [production] | 
            
  | 07:56 | 
  <_joe_> | 
  powercycling mw1162, unable to login from console, memory exhaustion | 
  [production] | 
            
  | 07:24 | 
  <ebernhardson@tin> | 
  ebernhardson@tin Synchronized php-1.27.0-wmf.10/extensions/CirrusSearch/includes/DataSender.php: stop checking for frozen indices while codfw elasticsearch recovers (duration: 01m 42s) | 
  [production] | 
            
  | 06:24 | 
  <ebernhardson> | 
  codfw elasticsearch cluster stopped responding during load test, idling test to see if it recovers | 
  [production] | 
            
  | 03:44 | 
  <l10nupdate@tin> | 
  l10nupdate@tin ResourceLoader cache refresh completed at Wed Jan 20 03:44:48 UTC 2016 (duration 7m 29s) | 
  [production] | 
            
  | 03:37 | 
  <mwdeploy@tin> | 
  mwdeploy@tin sync-l10n completed (1.27.0-wmf.11) (duration: 16m 21s) | 
  [production] | 
            
  | 03:02 | 
  <mwdeploy@tin> | 
  mwdeploy@tin sync-l10n completed (1.27.0-wmf.10) (duration: 10m 06s) | 
  [production] | 
            
  | 02:35 | 
  <mwdeploy@tin> | 
  mwdeploy@tin sync-l10n completed (1.27.0-wmf.9) (duration: 11m 20s) | 
  [production] | 
            
  | 01:27 | 
  <aaron@tin> | 
  aaron@tin Synchronized wmf-config: Configure $wgCdnReboundPurgeDelay (duration: 00m 32s) | 
  [production] | 
            
  | 01:01 | 
  <mobrovac> | 
  restbase deploy end of d621b76 | 
  [production] | 
            
  | 00:57 | 
  <krenair@tin> | 
  krenair@tin Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/264917/ (duration: 00m 32s) | 
  [production] | 
            
  | 00:56 | 
  <legoktm> | 
  delete from localuser where lu_name ="Αντώνης Μανιός" and lu_wiki ="mediawikiwiki" limit 1 on centralauth db for T119736 | 
  [production] | 
            
  | 00:53 | 
  <krenair@tin> | 
  krenair@tin Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/264920/ (duration: 00m 33s) | 
  [production] | 
            
  | 00:49 | 
  <krenair@tin> | 
  krenair@tin Synchronized php-1.27.0-wmf.10/extensions/MobileFrontend/includes/api/ApiMobileView.php: https://gerrit.wikimedia.org/r/#/c/264973/ (duration: 00m 32s) | 
  [production] | 
            
  | 00:49 | 
  <mobrovac> | 
  restbase deploy start of d621b76 | 
  [production] | 
            
  | 00:38 | 
  <krenair@tin> | 
  krenair@tin Synchronized wmf-config/CommonSettings.php: https://gerrit.wikimedia.org/r/#/c/264961/ (duration: 00m 31s) | 
  [production] | 
            
  | 00:37 | 
  <krenair@tin> | 
  krenair@tin Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/264961/ (duration: 00m 33s) | 
  [production] | 
            
  | 00:22 | 
  <krenair@tin> | 
  krenair@tin Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/264260/ (duration: 00m 32s) | 
  [production] | 
            
  | 00:21 | 
  <krenair@tin> | 
  krenair@tin Synchronized wmf-config/InitialiseSettings-labs.php: https://gerrit.wikimedia.org/r/#/c/264260/ (duration: 00m 32s) | 
  [production] | 
            
  | 00:17 | 
  <krenair@tin> | 
  krenair@tin Synchronized php-1.27.0-wmf.10/extensions/CirrusSearch: https://gerrit.wikimedia.org/r/#/c/265146/ (duration: 00m 33s) | 
  [production] | 
            
  | 00:10 | 
  <krenair@tin> | 
  krenair@tin Synchronized php-1.27.0-wmf.10/extensions/CirrusSearch/includes/ElasticsearchIntermediary.php: https://gerrit.wikimedia.org/r/#/c/264989/ (duration: 00m 32s) | 
  [production] |