| 
      
        2015-01-12
      
      §
     | 
  
    
  | 18:03 | 
  <demon> | 
  Synchronized wmf-config/InitialiseSettings.php: (no message) (duration: 00m 08s) | 
  [production] | 
            
  | 18:01 | 
  <bd808> | 
  Applied 2015 schema changes to scholarships database on m2-master | 
  [production] | 
            
  | 17:33 | 
  <hoo> | 
  mw1010: rsync: failed to set times on "/srv/mediawiki/.": Read-only file system (30) | 
  [production] | 
            
  | 17:31 | 
  <hoo> | 
  Synchronized php-1.25wmf13/extensions/CentralAuth/: Only test passwords once in CentralAuthUser::prepareMigration - 2nd try (duration: 00m 07s) | 
  [production] | 
            
  | 17:31 | 
  <hoo> | 
  Synchronized php-1.25wmf14/extensions/CentralAuth/: Only test passwords once in CentralAuthUser::prepareMigration (duration: 00m 06s) | 
  [production] | 
            
  | 17:31 | 
  <hoo> | 
  Synchronized php-1.25wmf13/extensions/CentralAuth/: Only test passwords once in CentralAuthUser::prepareMigration (duration: 00m 06s) | 
  [production] | 
            
  | 17:22 | 
  <mutante> | 
  restarted icinga-wm to join -releng | 
  [production] | 
            
  | 17:02 | 
  <mutante> | 
  labmon1001 - purging mlocate package that was status 'rc' | 
  [production] | 
            
  | 16:30 | 
  <godog> | 
  stop/start graphite-web on tungsten to clear logs | 
  [production] | 
            
  | 16:28 | 
  <bd808> | 
  deleted 2014-01-* and 2015-12-* indices from logstash elasticsearch cluster | 
  [production] | 
            
  | 16:13 | 
  <bd808> | 
  logs on logstash1001 reporting elasticserch connection errors; restarted logstash service | 
  [production] | 
            
  | 16:10 | 
  <anomie> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: Enable "Other projects sidebar" by default on frwiki [[gerrit:183288]] (duration: 00m 05s) | 
  [production] | 
            
  | 16:09 | 
  <bd808> | 
  logstash elasticsearch cluster has strange indices dated 2014-01-* and 2015-12-* again | 
  [production] | 
            
  | 16:05 | 
  <bd808> | 
  restarted elasticsearch on logstash1001 | 
  [production] | 
            
  | 16:05 | 
  <anomie> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: Disable thumbnail prerendering in production [[gerrit:183885]] (duration: 00m 06s) | 
  [production] | 
            
  | 16:03 | 
  <_joe_> | 
  depooling mw1062, disk errors | 
  [production] | 
            
  | 16:03 | 
  <bd808> | 
  elasticsearch on logstash1001 not responding to http requests | 
  [production] | 
            
  | 16:01 | 
  <aude> | 
  Synchronized wmf-config/Wikibase.php: Enable usage tracking on test.wikidata and testwiki (duration: 00m 05s) | 
  [production] | 
            
  | 16:00 | 
  <aude> | 
  Synchronized wmf-config/InitialiseSettings.php: (no message) (duration: 00m 05s) | 
  [production] | 
            
  | 15:59 | 
  <bd808> | 
  logstash not showing any events at all since 2015-01-12T13:58:59.728Z | 
  [production] | 
            
  | 15:58 | 
  <aude> | 
  Finished scap: Update Wikidata and WikimediaMessages (duration: 32m 06s) | 
  [production] | 
            
  | 15:35 | 
  <hashar> | 
  Enabling test/gate of several extensions together. {{gerrit|180494}} ,  RFC extensions continuous integration {{bug|T1350}} | 
  [production] | 
            
  | 15:27 | 
  <aude> | 
  Added and populated wbc_entity_usage table on testwiki and testwikidatawiki | 
  [production] | 
            
  | 15:25 | 
  <aude> | 
  Started scap: Update Wikidata and WikimediaMessages | 
  [production] | 
            
  | 15:10 | 
  <hashar> | 
  Synchronized php-1.25wmf13/extensions/Echo/tests/phpunit/includes/cache/TitleLocalCacheTest.php: php-1.25wmf14/extensions/Echo/tests/phpunit/includes/cache/TitleLocalCacheTest.php (duration: 00m 05s) | 
  [production] | 
            
  | 15:02 | 
  <hashar> | 
  restarting Zuul. Deadlocked due to Gerrit database | 
  [production] | 
            
  | 12:20 | 
  <_joe_> | 
  upgrading HHVM on the API cluster | 
  [production] | 
            
  | 10:45 | 
  <_joe_> | 
  restarting hhvm on mw1126, stuck in HPHP::StatCache::refresh | 
  [production] | 
            
  | 10:22 | 
  <_joe_> | 
  upgrading HHVM on all appservers | 
  [production] | 
            
  | 10:11 | 
  <hashar> | 
  Synchronized wmf-config/throttle.php: Tel-Hai Academic College event - Bug: T85773 (duration: 00m 07s) | 
  [production] | 
            
  | 09:23 | 
  <hashar> | 
  Restarting Zuul | 
  [production] | 
            
  | 09:06 | 
  <hashar> | 
  Tweak Zuul configuration to pin python-daemon <= 2.0  and deploying tag wmf-deploy-20150112-1. {{bug|T86513}} | 
  [production] | 
            
  | 07:24 | 
  <andrewbogott> | 
  on virt1005 and virt1006, ran 'ln -s /usr/bin/qemu-system-x86_64 /usr/bin/kvm'  that allows nova to migrate instances between hosts. | 
  [production] | 
            
  | 03:54 | 
  <LocalisationUpdate> | 
  ResourceLoader cache refresh completed at Mon Jan 12 03:54:34 UTC 2015 (duration 54m 33s) | 
  [production] | 
            
  | 02:17 | 
  <LocalisationUpdate> | 
  completed (1.25wmf14) at 2015-01-12 02:17:48+00:00 | 
  [production] | 
            
  | 02:17 | 
  <l10nupdate> | 
  Synchronized php-1.25wmf14/cache/l10n: (no message) (duration: 00m 01s) | 
  [production] | 
            
  | 02:11 | 
  <LocalisationUpdate> | 
  completed (1.25wmf13) at 2015-01-12 02:10:54+00:00 | 
  [production] | 
            
  | 02:10 | 
  <l10nupdate> | 
  Synchronized php-1.25wmf13/cache/l10n: (no message) (duration: 00m 01s) | 
  [production] | 
            
  
    | 
      
        2015-01-11
      
      §
     | 
  
    
  | 16:44 | 
  <springle> | 
  db1050 dberror log noise was https://phabricator.wikimedia.org/T86482 | 
  [production] | 
            
  | 16:29 | 
  <springle> | 
  Synchronized wmf-config/db-eqiad.php: repool db1050, warm up (duration: 00m 05s) | 
  [production] | 
            
  | 16:08 | 
  <springle> | 
  Synchronized wmf-config/db-eqiad.php: depool db1050, mysqld got TERM somehow (duration: 00m 05s) | 
  [production] | 
            
  | 03:59 | 
  <LocalisationUpdate> | 
  ResourceLoader cache refresh completed at Sun Jan 11 03:59:26 UTC 2015 (duration 59m 25s) | 
  [production] | 
            
  | 02:18 | 
  <LocalisationUpdate> | 
  completed (1.25wmf14) at 2015-01-11 02:18:10+00:00 | 
  [production] | 
            
  | 02:18 | 
  <l10nupdate> | 
  Synchronized php-1.25wmf14/cache/l10n: (no message) (duration: 00m 04s) | 
  [production] | 
            
  | 02:11 | 
  <LocalisationUpdate> | 
  completed (1.25wmf13) at 2015-01-11 02:11:13+00:00 | 
  [production] | 
            
  | 02:11 | 
  <l10nupdate> | 
  Synchronized php-1.25wmf13/cache/l10n: (no message) (duration: 00m 02s) | 
  [production] |