| 2013-09-04
      
      § | 
    
  | 19:41 | <Ryan_Lane> | upgraded wiktech-static to 1.22wmf15 | [production] | 
            
  | 19:16 | <andrewbogott> | turning on pluginsync, which will (temporarily) make puppet even slower | [production] | 
            
  | 19:10 | <andrewbogott> | Moved morebots from wikitech-static to toollabs | [production] | 
            
  | 06:26 | <apergos> | truncated fact_values puppet table and reset auto increment to start at 1, puppet was broken on all hosts, see http://projects.puppetlabs.com/issues/9225 | [production] | 
            
  | 02:22 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Wed Sep  4 02:21:58 UTC 2013 | [production] | 
            
  | 02:11 | <LocalisationUpdate> | completed (1.22wmf14) at Wed Sep  4 02:11:48 UTC 2013 | [production] | 
            
  | 02:06 | <LocalisationUpdate> | completed (1.22wmf15) at Wed Sep  4 02:06:47 UTC 2013 | [production] | 
            
  | 00:49 | <olivneh> | synchronized php-1.22wmf15/includes/RecentChange.php  'Fix for bug 53720' | [production] | 
            
  
    | 2013-09-03
      
      § | 
    
  | 22:45 | <Ryan_Lane> | upgrading wikitech to 1.22wmf15 | [production] | 
            
  | 19:45 | <reedy> | synchronized wmf-config/  'touch' | [production] | 
            
  | 19:37 | <reedy> | synchronized php-1.22wmf15/extensions/UploadWizard/resources  'touch' | [production] | 
            
  | 19:04 | <reedy> | rebuilt wikiversions.cdb and synchronized wikiversions files: Everything non 'pedia to 1.22wmf15 | [production] | 
            
  | 18:53 | <reedy> | rebuilt wikiversions.cdb and synchronized wikiversions files: special, wikimedia, closed and private wikis to 1.22wmf15 | [production] | 
            
  | 18:46 | <awight> | civicrm updated 5c28984d7e4 to 75d31d3443e45c | [production] | 
            
  | 18:37 | <mwalker> | optimizing civicrm.civicrm_email on db1025 (db1008) | [production] | 
            
  | 18:17 | <mwalker> | altering db1025 (db1008) civicrm.civicrm_email --> "ALTER TABLE civicrm_email MODIFY COLUMN email varchar(255);" | [production] | 
            
  | 18:00 | <gwicke> | updated Parsoid to 859a701 | [production] | 
            
  | 15:21 | <hashar> | gallium running mw phpunit code coverage under gdb in a tmux. | [production] | 
            
  | 14:26 | <hashar> | upgrading PHP5 on gallium (contint server) | [production] | 
            
  | 02:25 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Tue Sep  3 02:25:20 UTC 2013 | [production] | 
            
  | 02:14 | <LocalisationUpdate> | completed (1.22wmf15) at Tue Sep  3 02:14:15 UTC 2013 | [production] | 
            
  | 02:08 | <LocalisationUpdate> | completed (1.22wmf14) at Tue Sep  3 02:08:03 UTC 2013 | [production] | 
            
  
    | 2013-09-02
      
      § | 
    
  | 16:19 | <hashar> | restarted Zuul (paranoia) | [production] | 
            
  | 16:13 | <Reedy> | Running cleanupTitles.php in screen on terbium | [production] | 
            
  | 16:00 | <reedy> | synchronized wmf-config/InitialiseSettings.php | [production] | 
            
  | 14:35 | <hashar> | Jenkins: restoring mediawiki-core jobs to previous state | [production] | 
            
  | 14:31 | <hashar> | broken Jenkins whenever a huge repository is being used (i.e. mediawiki/core or operations/puppet ) | [production] | 
            
  | 13:41 | <akosiaris> | aptitude hold puppet puppet-common puppetmaster puppetmaster-common puppetmaster-passenger on stafford till we solve the problem with CPU spiking present in puppet 2.7.11-1ubuntu2.4 | [production] | 
            
  | 13:35 | <springle> | synchronized wmf-config/db-eqiad.php  'reutrn db1027 to the pool' | [production] | 
            
  | 06:20 | <springle> | granted process/show/usage to watchdog on db masters | [production] | 
            
  | 04:33 | <springle> | granted process/show/usage to watchdog on db masters | [production] | 
            
  | 02:25 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Mon Sep  2 02:25:02 UTC 2013 | [production] | 
            
  | 02:13 | <LocalisationUpdate> | completed (1.22wmf15) at Mon Sep  2 02:13:50 UTC 2013 | [production] | 
            
  | 02:07 | <LocalisationUpdate> | completed (1.22wmf14) at Mon Sep  2 02:07:46 UTC 2013 | [production] | 
            
  | 00:49 | <faidon> | synchronized wmf-config/InitialiseSettings.php  'new wikivoyage logo' | [production] | 
            
  | 00:48 | <faidon> | synchronized docroot/bits/favicon/wikivoyage.ico  'new wikivoyage favicon' | [production] | 
            
  
    | 2013-09-01
      
      § | 
    
  | 23:57 | <springle> | running pt-table-sync over db1027 wikis | [production] | 
            
  | 21:29 | <springle> | restarted db1027 mysqld. leaving port 3306 blocked while replication recovers | [production] | 
            
  | 20:51 | <paravoid> | powercycled db1027, BUG: soft lockup - CPU#14 stuck for 22s! [xfsaild/dm-0:933] | [production] | 
            
  | 20:29 | <paravoid> | restarting gerrit | [production] | 
            
  | 17:56 | <paravoid> | killed snapshot1004 workers, flooding dberror with wikiadmin auth failures + WikiExporter::dumpFrom slow queries (10' ago) | [production] | 
            
  | 17:52 | <faidon> | cleared profiling data | [production] | 
            
  | 17:04 | <maxsem> | synchronized /php-1.22wmf14/includes/api/ApiQueryImageInfo.php  'Not the cause, restoring back' | [production] | 
            
  | 16:53 | <maxsem> | synchronized /php-1.22wmf14/includes/api/ApiQueryImageInfo.php  'Trying to disable this...' | [production] | 
            
  | 02:07 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Sun Sep  1 02:07:18 UTC 2013 | [production] | 
            
  | 02:02 | <LocalisationUpdate> | completed (1.22wmf15) at Sun Sep  1 02:02:30 UTC 2013 | [production] | 
            
  | 02:01 | <LocalisationUpdate> | completed (1.22wmf14) at Sun Sep  1 02:01:46 UTC 2013 | [production] |