| 
      
        2014-11-13
      
      §
     | 
  
    
  | 15:06 | 
  <yurik> | 
  Synchronized php-1.25wmf8/extensions/ZeroPortal: updatidng ZeroPortal to master (duration: 01m 13s) | 
  [production] | 
            
  | 15:04 | 
  <chasemp> | 
  phabricator upgrades T1203 | 
  [production] | 
            
  | 14:43 | 
  <hashar:> | 
  restarted zuul-merger on gallium | 
  [production] | 
            
  | 14:42 | 
  <hashar:> | 
  restarting Jenkins and Zuul | 
  [production] | 
            
  | 12:45 | 
  <godog> | 
  investigating high iops on swift eqiad with paravoid, stopped object-auditor on ms-be1005 and ms-be1015 | 
  [production] | 
            
  | 11:09 | 
  <hashar> | 
  resurrected morebots in #wikimedia-operations (see [[Morebots]]). | 
  [production] | 
            
  | 11:08 | 
  <hashar> | 
  Killed Jenkins due to a deadlock | 
  [production] | 
            
  | 11:08 | 
  <hashar> | 
  Killing Jenkins due to a deadlock | 
  [production] | 
            
  | 02:52 | 
  <mutante> | 
  beta puppet freshness - UNKNOWN: No valid datapoints found .. since 13d  | 
  [production] | 
            
  | 02:30 | 
  <LocalisationUpdate> | 
  completed (1.25wmf8) at 2014-11-13 02:30:00+00:00 | 
  [production] | 
            
  | 02:18 | 
  <LocalisationUpdate> | 
  completed (1.25wmf7) at 2014-11-13 02:18:44+00:00 | 
  [production] | 
            
  | 00:46 | 
  <mutante> | 
  thulium -  Could not intern from pson: expected value in object at '"[PHP]\\n\\n; puppet:t'! | 
  [production] | 
            
  
    | 
      
        2014-11-12
      
      §
     | 
  
    
  | 21:59 | 
  <reedy> | 
  Synchronized wmf-config/: Set useLegacyUsageIndex = true for Wikibase client (duration: 00m 17s) | 
  [production] | 
            
  | 20:57 | 
  <reedy> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: group0 to 1.25wmf8 | 
  [production] | 
            
  | 20:55 | 
  <hashar> | 
  Restarting Jenkins, deadlock on deployment-bastion | 
  [production] | 
            
  | 20:28 | 
  <reedy> | 
  Synchronized wmf-config/: (no message) (duration: 00m 15s) | 
  [production] | 
            
  | 20:28 | 
  <reedy> | 
  Synchronized database lists: (no message) (duration: 00m 15s) | 
  [production] | 
            
  | 20:25 | 
  <manybubbles> | 
  restarting elastic1021 to pick up new plugins | 
  [production] | 
            
  | 20:21 | 
  <reedy> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: wikipedias to 1.25wmf7 | 
  [production] | 
            
  | 20:13 | 
  <reedy> | 
  Finished scap: testwiki to 1.25wmf8 and build l10n cache (duration: 53m 57s) | 
  [production] | 
            
  | 19:37 | 
  <hoo> | 
  Made myself oauthadmin on mediawikiwiki | 
  [production] | 
            
  | 19:19 | 
  <reedy> | 
  Started scap: testwiki to 1.25wmf8 and build l10n cache | 
  [production] | 
            
  | 19:05 | 
  <mutante> | 
  installing package upgrades on bast1001 (incl. PHP version) | 
  [production] | 
            
  | 19:04 | 
  <mutante> | 
  installing package upgrades on iron | 
  [production] | 
            
  | 18:38 | 
  <YuviPanda> | 
  turned off yurik's zerosms cronjob on stat1002 (already discussed with him, he was ok with it being stopped until he could find time to fix it) | 
  [production] | 
            
  | 17:58 | 
  <_joe_> | 
  gracefulling apache on problematic API hosts | 
  [production] | 
            
  | 17:05 | 
  <demon> | 
  Synchronized wmf-config/InitialiseSettings.php: (no message) (duration: 00m 04s) | 
  [production] | 
            
  | 16:51 | 
  <anomie> | 
  Synchronized php-1.25wmf7/extensions/SecurePoll/: SWAT: SecurePoll fix for jump-text and title on create/edit [[gerrit:172718]] (for real this time) (duration: 00m 09s) | 
  [production] | 
            
  | 16:48 | 
  <anomie> | 
  Finished scap: SWAT: SecurePoll fix for jump-text and title on create/edit [[gerrit:172718]] (duration: 22m 13s) | 
  [production] | 
            
  | 16:26 | 
  <anomie> | 
  Started scap: SWAT: SecurePoll fix for jump-text and title on create/edit [[gerrit:172718]] | 
  [production] | 
            
  | 16:25 | 
  <anomie> | 
  Synchronized php-1.25wmf7/extensions/MultimediaViewer/: SWAT: Backport MediaViewer options menu layout fix [[gerrit:172737]] (duration: 00m 09s) | 
  [production] | 
            
  | 16:04 | 
  <anomie> | 
  Synchronized wmf-config: SWAT: Set different ImageMetrics sampling factor for logged-in users [[gerrit:172720]] (duration: 00m 12s) | 
  [production] | 
            
  | 16:01 | 
  <anomie> | 
  Synchronized wmf-config/Wikibase.php: SWAT: Add "featured portal" badge (Q17580674) [[gerrit:172729]] (duration: 00m 10s) | 
  [production] | 
            
  | 14:55 | 
  <oblivian> | 
  Synchronized wmf-config/CommonSettings.php: Open HHVM to 20% of anons (duration: 00m 06s) | 
  [production] | 
            
  | 14:27 | 
  <manybubbles> | 
  restarting elastic1016 to pick up new plugins....  half way done | 
  [production] | 
            
  | 14:24 | 
  <_joe_> | 
  load test on hhvm done | 
  [production] | 
            
  | 13:55 | 
  <godog> | 
  rolling reload of swift on ms-be1* to pick up statsd changes | 
  [production] | 
            
  | 13:32 | 
  <godog> | 
  rolling reload of swift on ms-fe1* to pick up statsd changes | 
  [production] | 
            
  | 10:28 | 
  <_joe_> | 
  repooling mw1189 with a reduced hhvm thread count for testing (puppet disabled, as well) | 
  [production] | 
            
  | 10:16 | 
  <_joe_> | 
  depooling mw1189 from the api pool for reimaging | 
  [production] | 
            
  | 08:17 | 
  <_joe_> | 
  stress testing a group of HHVM servers in anticipation for the move to 20% of traffic | 
  [production] | 
            
  | 03:35 | 
  <LocalisationUpdate> | 
  ResourceLoader cache refresh completed at Wed Nov 12 03:35:17 UTC 2014 (duration 35m 16s) | 
  [production] | 
            
  | 02:15 | 
  <LocalisationUpdate> | 
  completed (1.25wmf7) at 2014-11-12 02:15:49+00:00 | 
  [production] | 
            
  | 02:09 | 
  <LocalisationUpdate> | 
  completed (1.25wmf6) at 2014-11-12 02:09:14+00:00 | 
  [production] |