| 
      
        2013-04-26
      
      §
     | 
  
    
  | 13:15 | 
  <^demon> | 
  gerrit replication freaked out, restarting | 
  [production] | 
            
  | 06:37 | 
  <hashar> | 
  Restarted Jenkins. It was stuck in some while loop doing tons of stat() on unexisting file :(   Would be up in roughly 40 minutes :( | 
  [production] | 
            
  | 03:45 | 
  <LocalisationUpdate> | 
  ResourceLoader cache refresh completed at Fri Apr 26 03:45:44 UTC 2013 | 
  [production] | 
            
  | 02:12 | 
  <LocalisationUpdate> | 
  completed (1.22wmf2) at Fri Apr 26 02:12:38 UTC 2013 | 
  [production] | 
            
  | 00:57 | 
  <kaldari> | 
  synchronized wmf-config/InitialiseSettings.php  'temporarily disabling WikiLove per bug 47457' | 
  [production] | 
            
  | 00:53 | 
  <catrope> | 
  synchronized php-1.22wmf2/extensions/VisualEditor/  'Update VisualEditor to master' | 
  [production] | 
            
  | 00:33 | 
  <catrope> | 
  synchronized wmf-config/InitialiseSettings.php  'Clean up VisualEditor config' | 
  [production] | 
            
  | 00:32 | 
  <Tim> | 
  on fenari: fixed permissions on /etc/ssh/ssh_known_hosts | 
  [production] | 
            
  | 00:32 | 
  <catrope> | 
  synchronized wmf-config/CommonSettings.php  'Remove wmgVisualEditorHide' | 
  [production] | 
            
  
    | 
      
        2013-04-25
      
      §
     | 
  
    
  | 23:09 | 
  <mwalker> | 
  synchronized docroot/bits/favicon/  'Pushing new favicons for incubator, wiktionary, wiktionary.en, wikibooks, wikinews, and wikispecies' | 
  [production] | 
            
  | 22:31 | 
  <mflaschen> | 
  Finished syncing Wikimedia installation... : | 
  [production] | 
            
  | 22:22 | 
  <^demon> | 
  new ssl cert (gerrit.wikimedia.org) live on gerrit. | 
  [production] | 
            
  | 20:55 | 
  <notpeter> | 
  reimaging db55 | 
  [production] | 
            
  | 20:44 | 
  <kaldari> | 
  Finished syncing Wikimedia installation... : | 
  [production] | 
            
  | 20:16 | 
  <kaldari> | 
  Started syncing Wikimedia installation... : | 
  [production] | 
            
  | 20:06 | 
  <awight> | 
  payments cluster updated from 7f70ccd580d to fdf308c8f | 
  [production] | 
            
  | 19:11 | 
  <kaldari> | 
  synchronized wmf-config/CommonSettings.php  'updating for new schema number' | 
  [production] | 
            
  | 18:52 | 
  <reedy> | 
  synchronized docroot | 
  [production] | 
            
  | 18:19 | 
  <LeslieCarr> | 
  cleaning up old home directories on fenari | 
  [production] | 
            
  | 17:00 | 
  <demon> | 
  synchronized wmf-config/CommonSettings.php  'Removing oldskin notice' | 
  [production] | 
            
  | 16:49 | 
  <catrope> | 
  synchronized wmf-config/InitialiseSettings.php  'Enable VisualEditor on 14 more Wikipedias' | 
  [production] | 
            
  | 16:37 | 
  <catrope> | 
  synchronized php-1.22wmf2/extensions/VisualEditor/  'Update VisualEditor to master' | 
  [production] | 
            
  | 16:19 | 
  <hashar> | 
  Jenkins is busy reparsing all the build history for some reason. That slow down the continuous integration box by an order of magnitude :/  Should take roughly 40 minutes (no idea when it started) | 
  [production] | 
            
  | 16:08 | 
  <hashar> | 
  Zuul git operations are slow again :/ | 
  [production] | 
            
  | 14:56 | 
  <demon> | 
  synchronized php-1.22wmf2/extensions/MWSearch/MWSearch_body.php  'rm debug hack' | 
  [production] | 
            
  | 14:55 | 
  <demon> | 
  synchronized wmf-config/InitialiseSettings.php  'Turn mwsearch debug log back off, its not phps fault' | 
  [production] | 
            
  | 14:53 | 
  <demon> | 
  synchronized wmf-config/InitialiseSettings.php  'Turn mwsearch debug log on' | 
  [production] | 
            
  | 14:52 | 
  <demon> | 
  synchronized php-1.22wmf2/extensions/MWSearch/MWSearch_body.php  'Debugging' | 
  [production] | 
            
  | 11:56 | 
  <krinkle> | 
  synchronized wmf-config/InitialiseSettings.php  'I2f0990fa087a' | 
  [production] | 
            
  | 11:55 | 
  <krinkle> | 
  synchronized docroot | 
  [production] | 
            
  | 03:39 | 
  <LocalisationUpdate> | 
  ResourceLoader cache refresh completed at Thu Apr 25 03:39:44 UTC 2013 | 
  [production] | 
            
  | 02:14 | 
  <LocalisationUpdate> | 
  completed (1.22wmf2) at Thu Apr 25 02:14:20 UTC 2013 | 
  [production] | 
            
  | 00:44 | 
  <mutante> | 
  upgrading packages on manutius | 
  [production] | 
            
  | 00:29 | 
  <mutante> | 
  DNS update - removing internproxy, removing tesla (minus mgmt) | 
  [production] | 
            
  
    | 
      
        2013-04-24
      
      §
     | 
  
    
  | 23:03 | 
  <dzahn> | 
  gracefulled all apaches | 
  [production] | 
            
  | 22:39 | 
  <dzahn> | 
  gracefulled all apaches | 
  [production] | 
            
  | 22:38 | 
  <LeslieCarr> | 
  upgrading mw's | 
  [production] | 
            
  | 22:37 | 
  <LeslieCarr> | 
  replaced smokeping targets for sdtpa c3 and sdtpa d3 | 
  [production] | 
            
  | 22:26 | 
  <krinkle> | 
  synchronized wmf-config/InitialiseSettings.php  'New location for icons - Ib5554c54729' | 
  [production] | 
            
  | 22:25 | 
  <mutante> | 
  sync-apache, add coffee | 
  [production] | 
            
  | 22:24 | 
  <krinkle> | 
  synchronized docroot/bits/apple-touch/  'New location for icons - Ib5554c54729' | 
  [production] | 
            
  | 21:47 | 
  <mutante> | 
  DNS update - remove survey, remove wikitech-old, remove argon | 
  [production] | 
            
  | 21:47 | 
  <demon> | 
  synchronized php-1.22wmf2/extensions/Wikibase  'b4b5fb4046175ca29834a4669c253c97ac5ccd07' | 
  [production] | 
            
  | 21:37 | 
  <demon> | 
  synchronized wmf-config/InitialiseSettings.php | 
  [production] | 
            
  | 21:33 | 
  <demon> | 
  synchronized wmf-config/InitialiseSettings.php  'Turn mwsearch log back on' | 
  [production] | 
            
  | 21:32 | 
  <demon> | 
  synchronized php-1.22wmf2/extensions/MWSearch/MWSearch_body.php | 
  [production] | 
            
  | 21:26 | 
  <mutante> | 
  DNS update - readding voip | 
  [production] | 
            
  | 21:17 | 
  <cmjohnson1> | 
  removing sq33 from dsh node groups and pybal cfg | 
  [production] | 
            
  | 20:32 | 
  <RobH> | 
  racktables on hooper upgraded to recent stable | 
  [production] | 
            
  | 20:25 | 
  <RobH> | 
  racktables flap again, working on it | 
  [production] |