| 
      
        2014-01-04
      
      §
     | 
  
    
  | 21:06 | 
  <hashar> | 
  jenkins: purged php5-mysqld package from lanthanum and gallium. It was no more installed but had some conf file left in /etc/php5/conf.d/ | 
  [production] | 
            
  | 14:26 | 
  <springle> | 
  synchronized wmf-config/db-eqiad.php  'db1040 back to full steam' | 
  [production] | 
            
  | 08:54 | 
  <springle> | 
  synchronized wmf-config/db-eqiad.php  'reduce db1026 load while reindexing wikidatawiki.wb_terms' | 
  [production] | 
            
  | 07:42 | 
  <springle> | 
  synchronized wmf-config/db-eqiad.php  'reduce db1040 load while adding indexes' | 
  [production] | 
            
  | 04:13 | 
  <springle> | 
  schema change, ad-hoc, additional indexes on recentchanges & wb_terms for recent slow queries | 
  [production] | 
            
  | 03:45 | 
  <LocalisationUpdate> | 
  ResourceLoader cache refresh completed at Sat Jan  4 03:45:39 UTC 2014 | 
  [production] | 
            
  | 03:13 | 
  <LocalisationUpdate> | 
  completed (1.23wmf9) at Sat Jan  4 03:13:45 UTC 2014 | 
  [production] | 
            
  | 02:38 | 
  <LocalisationUpdate> | 
  completed (1.23wmf8) at Sat Jan  4 02:38:29 UTC 2014 | 
  [production] | 
            
  | 01:33 | 
  <hashar> | 
  jenkins fixed up mwext-VisualEditor-qunit job, its configuration got reverted to some old/incorrect version when I downgraded the git plugin.  Retriggered all changes | 
  [production] | 
            
  | 00:02 | 
  <ori> | 
  synchronized php-1.23wmf9/skins/common  'Fix SVG MIME-type detection by reverting a9b855eea52ba3 (part 2/2)' | 
  [production] | 
            
  | 00:01 | 
  <ori> | 
  synchronized php-1.23wmf9/skins/vector/images  'Fix SVG MIME-type detection by reverting a9b855eea52ba3' | 
  [production] | 
            
  
    | 
      
        2014-01-03
      
      §
     | 
  
    
  | 22:07 | 
  <jeremyb> | 
  del | 
  [production] | 
            
  | 22:03 | 
  <awight> | 
  tools updated from b54854fcdd7a272ca2af8475affa033517456d07 to 0df36fd9bc55b24ffad49c0dbd0db67d67a2b1f7 | 
  [production] | 
            
  | 21:53 | 
  <mwalker> | 
  updating fundraising-tools on al from b54854fcdd7a272ca2af8475affa033517456d07 (for statistics generation in json format) | 
  [production] | 
            
  | 19:33 | 
  <ori> | 
  Some migration pains while moving graphite from professor to tungsten; expect graphite & gdash flakiness | 
  [production] | 
            
  | 19:01 | 
  <hashar> | 
  apparently jenkins is back up and happy. Had to revert the git plugin to a previous version ... | 
  [production] | 
            
  | 18:42 | 
  <aaron> | 
  started scap | 
  [production] | 
            
  | 18:36 | 
  <aaron> | 
  started scap: active Testing timing | 
  [production] | 
            
  | 18:11 | 
  <hashar> | 
  Jenkins downgrading git plugin client to 1.4.6 and restarting jenkins | 
  [production] | 
            
  | 17:52 | 
  <hashar> | 
  Jenkins downgrading git plugin from 2.0 to 1.5  , we might be hit by https://issues.jenkins-ci.org/browse/JENKINS-21057 | 
  [production] | 
            
  | 16:15 | 
  <reedy> | 
  synchronized php-1.23wmf8/extensions/ | 
  [production] | 
            
  | 16:14 | 
  <reedy> | 
  synchronized php-1.23wmf9/extensions/ | 
  [production] | 
            
  | 14:53 | 
  <hashar> | 
  jenkins restarted | 
  [production] | 
            
  | 14:43 | 
  <akosiaris> | 
  zapped /vol/root export from nas-1001-a | 
  [production] | 
            
  | 14:23 | 
  <hashar> | 
  restarting Jenkins , some git plugin are misbehaving | 
  [production] | 
            
  | 13:35 | 
  <springle> | 
  killed msnbot spike on s2 | 
  [production] | 
            
  | 12:43 | 
  <Krinkle> | 
  Jenkins still unable to use integration-slave01 (restarted the node in labs, and disconnected/re-launched slave agent afterwards, too; no effect) | 
  [production] | 
            
  | 12:14 | 
  <Krinkle> | 
  Jenkins is showing failures for tests executed on integration-slave01 (remote file system failing) | 
  [production] | 
            
  | 09:50 | 
  <apergos> | 
  restarted parsoid on wtp1010 and 1023, several gigs of logs full of "Error: Can't set headers after they are sent." from ServerResponse.OutgoingMessage.setHeader  | 
  [production] | 
            
  | 07:37 | 
  <apergos> | 
  streber from mgmt console reports eth0 link down (hence it appears down to icinga and ganglia) | 
  [production] | 
            
  | 07:06 | 
  <mark> | 
  Disabled OSPF3 on csw2-knams:xe-1/1/0.0 | 
  [production] | 
            
  | 04:27 | 
  <Ryan_Lane> | 
  added python-keystone-redis to apt repo | 
  [production] | 
            
  | 03:24 | 
  <springle> | 
  start schema changes for bug 59236, indexing only, ipblocks ipb_parent_block_id | 
  [production] | 
            
  | 03:21 | 
  <LocalisationUpdate> | 
  ResourceLoader cache refresh completed at Fri Jan  3 03:20:58 UTC 2014 | 
  [production] | 
            
  | 02:47 | 
  <LocalisationUpdate> | 
  completed (1.23wmf9) at Fri Jan  3 02:47:34 UTC 2014 | 
  [production] | 
            
  | 02:18 | 
  <LocalisationUpdate> | 
  completed (1.23wmf8) at Fri Jan  3 02:17:55 UTC 2014 | 
  [production] | 
            
  | 01:57 | 
  <reedy> | 
  finished scap: Fix Disambiguator hewiki magicwords | 
  [production] | 
            
  | 01:30 | 
  <reedy> | 
  started scap: Fix Disambiguator hewiki magicwords | 
  [production] | 
            
  | 00:23 | 
  <tstarling> | 
  synchronized wmf-config/CommonSettings.php | 
  [production] | 
            
  | 00:02 | 
  <maxsem> | 
  synchronized php-1.23wmf9/extensions/MobileFrontend  'https://gerrit.wikimedia.org/r/105108' | 
  [production] |