| 2012-11-13
      
      § | 
    
  | 22:56 | <logmsgbot_> | kaldari synchronized wmf-config/PrivateSettings.php  'syncing PrivateSettings back to previous version' | [production] | 
            
  | 22:41 | <binasher> | converted puppet.updates_update table to innodb - received the most number of of any puppet db table and was myisam, yikes. this might help with concurrency. | [production] | 
            
  | 22:31 | <LeslieCarr> | restarting gmetad on nickel | [production] | 
            
  | 22:22 | <Reedy> | Publishing schema for lucene/oai on mw.o.... | [production] | 
            
  | 22:07 | <mutante> | mailman lists: renaming wikimedia-th to wikimedia-th-private | [production] | 
            
  | 22:06 | <logmsgbot_> | kaldari synchronized wmf-config/PrivateSettings.php  'syncing PrivateSettings for Flickr API key' | [production] | 
            
  | 18:43 | <RobH> | analytics1007 now pxe boots | [production] | 
            
  | 18:41 | <RobH> | analytics1007.mgmt locked up, had to remove all power to fix, now working | [production] | 
            
  | 18:33 | <RobH> | testing analytics1002, pxe boots no problem | [production] | 
            
  | 17:53 | <paravoid> | setting up media1:thumbs on nas1001-a and semi-sync snapmirror from nas1-a | [production] | 
            
  | 17:24 | <Jeff_Green> | wikimedia.org zone update, new DKIM keys for silverpop | [production] | 
            
  | 16:54 | <paravoid> | snapmirror between nas1-a/nas1001-a for all volumes in-sync again | [production] | 
            
  | 16:50 | <paravoid> | running "cf giveback" on nas1001-b to restore nas1001-a; had to "partner snapmirror off" first. | [production] | 
            
  | 16:46 | <apergos> | had to restart varnish frontend by hand after reboot, also the varnishncsa for emery, locke, multicast | [production] | 
            
  | 16:30 | <apergos> | rebooting cp1042 (XFS: possible memory allocation deadlock in kmem_alloc), lots of stuck /bin/ps form nagios, load average ridiculous | [production] | 
            
  | 16:10 | <paravoid> | creating media1:thumbs volume at nas1-a and setting up /etc/export rules for it | [production] | 
            
  | 16:08 | <cmjohnson1> | provision labsdb3 | [production] | 
            
  | 15:32 | <cmjohnson1> | provisioning labsdb2 | [production] | 
            
  | 15:15 | <hashar> | restarting jenkins on gallium | [production] | 
            
  | 08:57 | <logmsgbot_> | reedy synchronized php-1.21wmf4/extensions/TimedMediaHandler | [production] | 
            
  | 08:56 | <logmsgbot_> | reedy synchronized php-1.21wmf3/extensions/TimedMediaHandler | [production] | 
            
  | 02:39 | <logmsgbot_> | LocalisationUpdate completed (1.21wmf4) at Tue Nov 13 02:39:21 UTC 2012 | [production] | 
            
  | 02:22 | <logmsgbot_> | LocalisationUpdate completed (1.21wmf3) at Tue Nov 13 02:22:06 UTC 2012 | [production] | 
            
  
    | 2012-11-12
      
      § | 
    
  | 22:46 | <Tim> | deleted refreshLinks jobs corresponding to my recent edit to enwiki's Template:Citation/core | [production] | 
            
  | 21:47 | <logmsgbot_> | reedy synchronized wmf-config/CommonSettings.php | [production] | 
            
  | 21:45 | <logmsgbot_> | reedy synchronized wmf-config/CommonSettings.php | [production] | 
            
  | 21:43 | <logmsgbot_> | reedy synchronized wmf-config/CommonSettings.php | [production] | 
            
  | 21:32 | <logmsgbot_> | reedy synchronized wmf-config/InitialiseSettings.php | [production] | 
            
  | 21:17 | <logmsgbot_> | reedy synchronized wmf-config/InitialiseSettings.php | [production] | 
            
  | 21:15 | <logmsgbot_> | reedy synchronized wmf-config/InitialiseSettings.php | [production] | 
            
  | 21:11 | <logmsgbot_> | reedy synchronized wmf-config/InitialiseSettings.php | [production] | 
            
  | 19:22 | <RoanKattouw> | Updated Parsoid on wtp1 | [production] | 
            
  | 19:15 | <logmsgbot_> | reedy synchronized wmf-config/ | [production] | 
            
  | 19:13 | <logmsgbot_> | reedy synchronized php-1.21wmf4/extensions/EducationProgram | [production] | 
            
  | 19:02 | <logmsgbot_> | reedy synchronized php-1.21wmf4/extensions/Wikibase | [production] | 
            
  | 18:53 | <logmsgbot_> | reedy rebuilt wikiversions.cdb and synchronized wikiversions files: mediawikiwiki, testwiki and wikidatawiki to 1.21wmf4 | [production] | 
            
  | 18:53 | <paravoid> | upgrading tmh1/tmh2 and srv{190,219,220,221,222,223,224} for libav vulnerability (USN-1630-1) | [production] | 
            
  | 18:52 | <logmsgbot_> | reedy synchronized php-1.21wmf4/cache/l10n/ | [production] | 
            
  | 18:36 | <logmsgbot_> | reedy gracefulled all apaches | [production] | 
            
  | 18:33 | <logmsgbot_> | reedy gracefulled all apaches | [production] | 
            
  | 18:29 | <logmsgbot_> | reedy synchronized live-1.5/ | [production] | 
            
  | 18:19 | <sbernardin> | changing ssd's on labsdb3 | [production] | 
            
  | 18:05 | <logmsgbot_> | reedy synchronized live-1.5/ | [production] | 
            
  | 18:04 | <logmsgbot_> | reedy synchronized php-1.21wmf3/extensions/EducationProgram/ | [production] | 
            
  | 17:59 | <logmsgbot_> | reedy synchronized live-1.5/ | [production] | 
            
  | 17:35 | <logmsgbot_> | reedy Started syncing Wikimedia installation... : Build 1.21wmf4 messages | [production] | 
            
  | 17:14 | <logmsgbot_> | reedy rebuilt wikiversions.cdb and synchronized wikiversions files: test2wiki to 1.21wmf4 | [production] | 
            
  | 17:07 | <logmsgbot_> | reedy synchronized live-1.5/ | [production] | 
            
  | 17:05 | <logmsgbot_> | reedy synchronized wmf-config/ExtensionMessages-1.21wmf4.php | [production] | 
            
  | 17:03 | <logmsgbot_> | reedy synchronized php-1.21wmf4/cache | [production] |