| 2016-01-20
      
      § | 
    
  | 16:56 | <thcipriani@tin> | thcipriani@tin Synchronized wmf-config/InitialiseSettings.php: SWAT: Set default graph vega version back to 1 [[gerrit:265289]] (duration: 00m 32s) | [production] | 
            
  | 16:46 | <thcipriani@tin> | thcipriani@tin Synchronized wmf-config/InitialiseSettings.php: SWAT: Add davidabian.com to wgCopyUploadsDomains [[gerrit:265286]] (duration: 00m 32s) | [production] | 
            
  | 16:42 | <thcipriani@tin> | thcipriani@tin Synchronized wmf-config/CommonSettings.php: SWAT: Change default graph version param. Part II [[gerrit:265282]] (duration: 00m 32s) | [production] | 
            
  | 16:42 | <thcipriani@tin> | thcipriani@tin Synchronized wmf-config/InitialiseSettings.php: SWAT: Change default graph version param. Part I [[gerrit:265282]] (duration: 00m 36s) | [production] | 
            
  | 16:33 | <thcipriani@tin> | thcipriani@tin Synchronized wmf-config/InitialiseSettings.php: SWAT: Add davidabian.com to wgCopyUploadsDomains [[gerrit:259003]] (duration: 00m 32s) | [production] | 
            
  | 16:21 | <thcipriani@tin> | thcipriani@tin Synchronized wmf-config/InitialiseSettings.php: SWAT: Add *.bodleian.ox.ac.uk to wgCopyUploadsDomains [[gerrit:265165]] (duration: 00m 33s) | [production] | 
            
  | 16:19 | <thcipriani@tin> | thcipriani@tin Synchronized wmf-config/InitialiseSettings.php: SWAT: Add *.archives.gov to wgCopyUploadsDomains [[gerrit:265163]] (duration: 00m 32s) | [production] | 
            
  | 16:13 | <godog> | bounce hhvm on mw1191 and syntaxlight runaway processes | [production] | 
            
  | 16:05 | <thcipriani@tin> | thcipriani@tin Synchronized wmf-config/InitialiseSettings.php: SWAT: Disable active gadget user stats on enwiki since it takes too long [[gerrit:265185]] (duration: 00m 32s) | [production] | 
            
  | 14:52 | <reedy@tin> | reedy@tin Synchronized php-1.27.0-wmf.11/vendor/: Fix ?PHP properly from commit (duration: 00m 36s) | [production] | 
            
  | 14:50 | <godog> | powercycle mw1123, hhvm oom | [production] | 
            
  | 14:47 | <ema> | Finished reverting migration of mobile traffic to text cluster in codfw https://phabricator.wikimedia.org/T109286 | [production] | 
            
  | 14:24 | <hoo@tin> | hoo@tin Synchronized wmf-config/db-eqiad.php: Set db1045 load to 0 (duration: 00m 32s) | [production] | 
            
  | 14:23 | <reedy@tin> | reedy@tin Synchronized php-1.27.0-wmf.11/: consistency (duration: 02m 38s) | [production] | 
            
  | 14:15 | <hoo@tin> | hoo@tin Synchronized wmf-config/db-eqiad.php: Re-Pool lagged db1045 (duration: 00m 35s) | [production] | 
            
  | 14:14 | <_joe_> | syncronizing /srv/deployment manually between the two deployment servers for the first time | [production] | 
            
  | 14:11 | <hoo@tin> | hoo@tin Synchronized wmf-config/db-eqiad.php: Has not been synced before (duration: 00m 32s) | [production] | 
            
  | 14:07 | <reedy@tin> | reedy@tin Synchronized php-1.27.0-wmf.10/: consistency (duration: 02m 38s) | [production] | 
            
  | 13:58 | <reedy@tin> | reedy@tin Synchronized php-1.27.0-wmf.11/extensions/Validator/: noop for wikitech deploy (duration: 00m 32s) | [production] | 
            
  | 13:58 | <reedy@tin> | reedy@tin Synchronized php-1.27.0-wmf.11/extensions/SemanticMediaWiki/: noop for wikitech deploy (duration: 00m 34s) | [production] | 
            
  | 13:57 | <reedy@tin> | reedy@tin Synchronized php-1.27.0-wmf.11/extensions/SemanticResultFormats/: noop for wikitech deploy (duration: 00m 33s) | [production] | 
            
  | 13:57 | <reedy@tin> | reedy@tin Synchronized php-1.27.0-wmf.10/extensions/SemanticResultFormats/: noop for wikitech deploy (duration: 00m 33s) | [production] | 
            
  | 13:55 | <reedy@tin> | reedy@tin Synchronized php-1.27.0-wmf.10/extensions/SemanticMediaWiki/: noop for wikitech deploy (duration: 00m 33s) | [production] | 
            
  | 13:54 | <reedy@tin> | reedy@tin Synchronized php-1.27.0-wmf.10/extensions/Validator/: noop for wikitech deploy (duration: 00m 33s) | [production] | 
            
  | 13:52 | <jynus@tin> | jynus@tin Synchronized wmf-config/db-eqiad.php: Setting db1049 as API node as db1045 is lagged (duration: 00m 33s) | [production] | 
            
  | 13:41 | <ema> | Revert migration of mobile traffic to text cluster in codfw https://phabricator.wikimedia.org/T109286 | [production] | 
            
  | 12:55 | <akosiaris> | restart hhvm on mw1130 | [production] | 
            
  | 12:43 | <jynus> | performing alter table on db1026 (ETA: 5 hours) | [production] | 
            
  | 12:20 | <jynus@tin> | jynus@tin Synchronized wmf-config/db-eqiad.php: Setting s5 master as recentchanges role (duration: 00m 32s) | [production] | 
            
  | 12:04 | <jynus> | trying schema change on wikidata (wb_terms) | [production] | 
            
  | 09:36 | <akosiaris> | gnt-instance modify -H disk_aio=native cygnus.codfw.wmnet | [production] | 
            
  | 09:18 | <akosiaris> | offline fr_archive volume on nas1001-a | [production] | 
            
  | 09:15 | <akosiaris> | unexport /vol/fr_archive on nas1001-a | [production] | 
            
  | 07:56 | <_joe_> | powercycling mw1162, unable to login from console, memory exhaustion | [production] | 
            
  | 07:24 | <ebernhardson@tin> | ebernhardson@tin Synchronized php-1.27.0-wmf.10/extensions/CirrusSearch/includes/DataSender.php: stop checking for frozen indices while codfw elasticsearch recovers (duration: 01m 42s) | [production] | 
            
  | 06:24 | <ebernhardson> | codfw elasticsearch cluster stopped responding during load test, idling test to see if it recovers | [production] | 
            
  | 03:44 | <l10nupdate@tin> | l10nupdate@tin ResourceLoader cache refresh completed at Wed Jan 20 03:44:48 UTC 2016 (duration 7m 29s) | [production] | 
            
  | 03:37 | <mwdeploy@tin> | mwdeploy@tin sync-l10n completed (1.27.0-wmf.11) (duration: 16m 21s) | [production] | 
            
  | 03:02 | <mwdeploy@tin> | mwdeploy@tin sync-l10n completed (1.27.0-wmf.10) (duration: 10m 06s) | [production] | 
            
  | 02:35 | <mwdeploy@tin> | mwdeploy@tin sync-l10n completed (1.27.0-wmf.9) (duration: 11m 20s) | [production] | 
            
  | 01:27 | <aaron@tin> | aaron@tin Synchronized wmf-config: Configure $wgCdnReboundPurgeDelay (duration: 00m 32s) | [production] | 
            
  | 01:01 | <mobrovac> | restbase deploy end of d621b76 | [production] | 
            
  | 00:57 | <krenair@tin> | krenair@tin Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/264917/ (duration: 00m 32s) | [production] | 
            
  | 00:56 | <legoktm> | delete from localuser where lu_name ="Αντώνης Μανιός" and lu_wiki ="mediawikiwiki" limit 1 on centralauth db for T119736 | [production] | 
            
  | 00:53 | <krenair@tin> | krenair@tin Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/264920/ (duration: 00m 33s) | [production] | 
            
  | 00:49 | <krenair@tin> | krenair@tin Synchronized php-1.27.0-wmf.10/extensions/MobileFrontend/includes/api/ApiMobileView.php: https://gerrit.wikimedia.org/r/#/c/264973/ (duration: 00m 32s) | [production] | 
            
  | 00:49 | <mobrovac> | restbase deploy start of d621b76 | [production] | 
            
  | 00:38 | <krenair@tin> | krenair@tin Synchronized wmf-config/CommonSettings.php: https://gerrit.wikimedia.org/r/#/c/264961/ (duration: 00m 31s) | [production] | 
            
  | 00:37 | <krenair@tin> | krenair@tin Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/264961/ (duration: 00m 33s) | [production] | 
            
  | 00:22 | <krenair@tin> | krenair@tin Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/#/c/264260/ (duration: 00m 32s) | [production] |