| 2015-02-04
      
      § | 
    
  | 20:33 | <ori> | rebuilt wikiversions.cdb and synchronized wikiversions files: I4fb67945b: Revert "[Regression] Revert "Non wikipedias to 1.25wmf15" | [production] | 
            
  | 20:18 | <aude> | Synchronized wmf-config/Wikibase.php: set useLegacyChangesSubscription to true for Wikidata (duration: 00m 07s) | [production] | 
            
  | 18:30 | <godog> | bounce txstatsd on cache hosts in eqiad | [production] | 
            
  | 18:17 | <godog> | bounce txstatsd on cache hosts in ulsfo | [production] | 
            
  | 18:08 | <godog> | bounce txstatsd on cache hosts in esams | [production] | 
            
  | 17:30 | <marktraceur> | Synchronized php-1.25wmf14/extensions/UploadWizard/: Touching pretty much everything in UploadWizard, maybe it will help (duration: 00m 07s) | [production] | 
            
  | 17:22 | <marktraceur> | Synchronized php-1.25wmf14/extensions/UploadWizard/resources/mw.UploadWizard.js: Touch an UploadWizard file to try and fix caching (duration: 00m 07s) | [production] | 
            
  | 16:58 | <robh> | replacing the intermediary cert on dumps.w.o (so nginx will flap on it shortly) | [production] | 
            
  | 16:56 | <godog> | restart ES on elastic1001 | [production] | 
            
  | 15:43 | <marktraceur> | Synchronized php-1.25wmf14/extensions/UploadWizard/resources/controller/uw.controller.Upload.js: Touch an UploadWizard file to try and fix caching (duration: 00m 07s) | [production] | 
            
  | 15:25 | <marktraceur> | Synchronized php-1.25wmf14/extensions/UploadWizard/resources/controller/uw.controller.Upload.js: Touch an UploadWizard file to try and fix caching (duration: 00m 05s) | [production] | 
            
  | 15:22 | <godog> | graphite move close to completion, updating dashboards | [production] | 
            
  | 15:16 | <godog> | bounce diamond in batches in eqiad | [production] | 
            
  | 14:50 | <marktraceur> | Synchronized php-1.25wmf15/extensions/UploadWizard/resources/controller/uw.controller.Upload.js: Touch an UploadWizard file to try and fix caching (duration: 00m 05s) | [production] | 
            
  | 14:14 | <godog> | bounce webperf-related services on hafnium too: ve, statsd-mw-js-deprecate, statsv, asset-check | [production] | 
            
  | 14:10 | <godog> | bounce navtiming on hafnium to pick up dns changes | [production] | 
            
  | 12:42 | <godog> | stop bacula-fd on tungsten, backups running during migration | [production] | 
            
  | 12:41 | <_joe_> | installing the new HHVM package on jobrunners | [production] | 
            
  | 12:28 | <godog> | bounce txstatsd on ms-fe* | [production] | 
            
  | 12:28 | <godog> | bounce txstatsd on ms-be* | [production] | 
            
  | 12:00 | <godog> | bounce diamond in batches in ulsfo | [production] | 
            
  | 11:57 | <godog> | bounce diamond in batches in esams | [production] | 
            
  | 11:51 | <godog> | bounce mwprof on tungsten to force picking up dns changes | [production] | 
            
  | 11:35 | <_joe_> | installing the new hhvm package on api, one at a time | [production] | 
            
  | 11:23 | <godog> | start migrating graphite from tungsten to graphite1001  https://gerrit.wikimedia.org/r/#/c/188036/1 https://gerrit.wikimedia.org/r/#/c/188035/1  https://phabricator.wikimedia.org/T85909 | [production] | 
            
  | 10:14 | <ori> | Finished scap: I78446aacb: [Regression] Revert "Non wikipedias to 1.25wmf15" (duration: 31m 34s) | [production] | 
            
  | 10:06 | <godog> | start migrating graphite from tungsten to graphite1001 https://gerrit.wikimedia.org/r/#/c/188036/1 https://gerrit.wikimedia.org/r/#/c/188035/1 https://phabricator.wikimedia.org/T85909 | [production] | 
            
  | 10:06 | <ori> | restarted hung HHVM on mw1039 | [production] | 
            
  | 09:42 | <ori> | Started scap: I78446aacb: [Regression] Revert "Non wikipedias to 1.25wmf15" | [production] | 
            
  | 09:42 | <ori> | scap aborted: I78446aacb: [Regression] Revert "Non wikipedias to 1.25wmf15" (duration: 00m 02s) | [production] | 
            
  | 09:42 | <ori> | Started scap: I78446aacb: [Regression] Revert "Non wikipedias to 1.25wmf15" | [production] | 
            
  | 08:57 | <_joe_> | installing the new hhvm package on all appservers, one at a time | [production] | 
            
  | 07:49 | <qchris> | Manual failover of Hadoop namenode from analytics1002 to analytics1001, as analytics1002 had Heap space errors | [production] | 
            
  | 05:16 | <springle> | Synchronized wmf-config/db-eqiad.php: depool db1057 (duration: 00m 06s) | [production] | 
            
  | 04:49 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Wed Feb  4 04:48:42 UTC 2015 (duration 48m 41s) | [production] | 
            
  | 02:48 | <tstarling> | Synchronized php-1.25wmf15/includes/specials/SpecialUserrights.php: Unbreak interwiki user rights granting (duration: 00m 05s) | [production] | 
            
  | 02:30 | <LocalisationUpdate> | completed (1.25wmf15) at 2015-02-04 02:29:17+00:00 | [production] | 
            
  | 02:29 | <l10nupdate> | Synchronized php-1.25wmf15/cache/l10n: (no message) (duration: 00m 02s) | [production] | 
            
  | 02:16 | <LocalisationUpdate> | completed (1.25wmf14) at 2015-02-04 02:15:35+00:00 | [production] | 
            
  | 02:15 | <l10nupdate> | Synchronized php-1.25wmf14/cache/l10n: (no message) (duration: 00m 03s) | [production] | 
            
  | 01:47 | <bd808> | restarted elasticsearch on logstash1001; rolling restart part 3 of 3 | [production] | 
            
  | 01:39 | <bd808> | restarted elasticsearch on logstash1002; rolling restart of cluster part 2 of 3 | [production] | 
            
  | 00:38 | <robh> | replacing dumps.w.o sha1 cert with sha256 | [production] |