| 2013-09-05
      
      § | 
    
  | 15:28 | <cmjohnson1> | replacing disk slot 0 ms-be1011 | [production] | 
            
  | 15:18 | <reedy> | synchronized docroot and w | [production] | 
            
  | 15:17 | <reedy> | synchronized php-1.22wmf16  'Initial file sync' | [production] | 
            
  | 14:12 | <paravoid> | recreating mediawiki cgroup lost after cgroup-bin upgrade; thumbnail issues fixed | [production] | 
            
  | 13:14 | <paravoid> | adjusting depool-threshold, restarting pybal on lvs1003 | [production] | 
            
  | 12:13 | <paravoid> | pushing new swift object/account/container rings: setting ms-be9's sde1 to 33, swapping ms-be5's sda/b <-> sdm/n, also 33 | [production] | 
            
  | 10:21 | <apergos> | lanthanum bios processor settings set to 'custom' and c states disabled (attempt to fix power limit notification issue) | [production] | 
            
  | 09:57 | <springle> | leaving sync_binlog=0 on db36 and db38 so replication keeps up. to be reviewed | [production] | 
            
  | 09:05 | <hashar_> | jenkins: killed two requesthandler threads that were eating CPU power. | [production] | 
            
  | 02:47 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Thu Sep  5 02:47:15 UTC 2013 | [production] | 
            
  | 02:30 | <LocalisationUpdate> | completed (1.22wmf14) at Thu Sep  5 02:30:03 UTC 2013 | [production] | 
            
  | 02:16 | <LocalisationUpdate> | completed (1.22wmf15) at Thu Sep  5 02:15:59 UTC 2013 | [production] | 
            
  | 01:42 | <^d> | gallium: updated git remotes in /srv/ssd/zuul/git/* to not point to old manganese box and point to gerrit.wm.o instead | [production] | 
            
  
    | 2013-09-04
      
      § | 
    
  | 23:17 | <^d> | restarting gerrit one last time | [production] | 
            
  | 23:04 | <^d> | apache stopped on manganese | [production] | 
            
  | 23:01 | <^d> | stopped puppet on manganese | [production] | 
            
  | 21:23 | <maxsem> | Finished syncing Wikimedia installation... : Weekly mobile deployment | [production] | 
            
  | 21:11 | <maxsem> | Started syncing Wikimedia installation... : Weekly mobile deployment | [production] | 
            
  | 19:41 | <Ryan_Lane> | upgraded wiktech-static to 1.22wmf15 | [production] | 
            
  | 19:16 | <andrewbogott> | turning on pluginsync, which will (temporarily) make puppet even slower | [production] | 
            
  | 19:10 | <andrewbogott> | Moved morebots from wikitech-static to toollabs | [production] | 
            
  | 06:26 | <apergos> | truncated fact_values puppet table and reset auto increment to start at 1, puppet was broken on all hosts, see http://projects.puppetlabs.com/issues/9225 | [production] | 
            
  | 02:22 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Wed Sep  4 02:21:58 UTC 2013 | [production] | 
            
  | 02:11 | <LocalisationUpdate> | completed (1.22wmf14) at Wed Sep  4 02:11:48 UTC 2013 | [production] | 
            
  | 02:06 | <LocalisationUpdate> | completed (1.22wmf15) at Wed Sep  4 02:06:47 UTC 2013 | [production] | 
            
  | 00:49 | <olivneh> | synchronized php-1.22wmf15/includes/RecentChange.php  'Fix for bug 53720' | [production] | 
            
  
    | 2013-09-03
      
      § | 
    
  | 22:45 | <Ryan_Lane> | upgrading wikitech to 1.22wmf15 | [production] | 
            
  | 19:45 | <reedy> | synchronized wmf-config/  'touch' | [production] | 
            
  | 19:37 | <reedy> | synchronized php-1.22wmf15/extensions/UploadWizard/resources  'touch' | [production] | 
            
  | 19:04 | <reedy> | rebuilt wikiversions.cdb and synchronized wikiversions files: Everything non 'pedia to 1.22wmf15 | [production] | 
            
  | 18:53 | <reedy> | rebuilt wikiversions.cdb and synchronized wikiversions files: special, wikimedia, closed and private wikis to 1.22wmf15 | [production] | 
            
  | 18:46 | <awight> | civicrm updated 5c28984d7e4 to 75d31d3443e45c | [production] | 
            
  | 18:37 | <mwalker> | optimizing civicrm.civicrm_email on db1025 (db1008) | [production] | 
            
  | 18:17 | <mwalker> | altering db1025 (db1008) civicrm.civicrm_email --> "ALTER TABLE civicrm_email MODIFY COLUMN email varchar(255);" | [production] | 
            
  | 18:00 | <gwicke> | updated Parsoid to 859a701 | [production] | 
            
  | 15:21 | <hashar> | gallium running mw phpunit code coverage under gdb in a tmux. | [production] | 
            
  | 14:26 | <hashar> | upgrading PHP5 on gallium (contint server) | [production] | 
            
  | 02:25 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Tue Sep  3 02:25:20 UTC 2013 | [production] | 
            
  | 02:14 | <LocalisationUpdate> | completed (1.22wmf15) at Tue Sep  3 02:14:15 UTC 2013 | [production] | 
            
  | 02:08 | <LocalisationUpdate> | completed (1.22wmf14) at Tue Sep  3 02:08:03 UTC 2013 | [production] | 
            
  
    | 2013-09-02
      
      § | 
    
  | 16:19 | <hashar> | restarted Zuul (paranoia) | [production] | 
            
  | 16:13 | <Reedy> | Running cleanupTitles.php in screen on terbium | [production] | 
            
  | 16:00 | <reedy> | synchronized wmf-config/InitialiseSettings.php | [production] | 
            
  | 14:35 | <hashar> | Jenkins: restoring mediawiki-core jobs to previous state | [production] | 
            
  | 14:31 | <hashar> | broken Jenkins whenever a huge repository is being used (i.e. mediawiki/core or operations/puppet ) | [production] | 
            
  | 13:41 | <akosiaris> | aptitude hold puppet puppet-common puppetmaster puppetmaster-common puppetmaster-passenger on stafford till we solve the problem with CPU spiking present in puppet 2.7.11-1ubuntu2.4 | [production] | 
            
  | 13:35 | <springle> | synchronized wmf-config/db-eqiad.php  'reutrn db1027 to the pool' | [production] | 
            
  | 06:20 | <springle> | granted process/show/usage to watchdog on db masters | [production] | 
            
  | 04:33 | <springle> | granted process/show/usage to watchdog on db masters | [production] | 
            
  | 02:25 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Mon Sep  2 02:25:02 UTC 2013 | [production] |