| 2013-03-01
      
      § | 
    
  | 19:04 | <mutante> | create missing backup dir on streber to enable RT_Shredder plugin | [production] | 
            
  | 19:04 | <Ryan_Lane> | maxsem synchronized wmf-config/InitialiseSettings.php  'make upload endpoint for Commons local https://gerrit.wikimedia.org/r/#/c/51612/' | [production] | 
            
  | 19:03 | <Ryan_Lane> | LocalisationUpdate completed (1.21wmf10) at Fri Mar  1 02:31:13 UTC 2013 | [production] | 
            
  | 19:03 | <Ryan_Lane> | csteipp synchronized php-1.21wmf10/includes | [production] | 
            
  | 19:03 | <mutante> | db9 - zipping old sql dumps of drupal and civirm to prevent full disk | [production] | 
            
  | 19:03 | <Ryan_Lane> | dzahn synchronized ./docroot/wikivoyage.org/mywotf83102384bfcd1e52152.html | [production] | 
            
  | 19:03 | <Ryan_Lane> | restarted glusterd service on labstore1-4 | [production] | 
            
  | 19:03 | <Ryan_Lane> | pgehres Finished syncing Wikimedia installation... : Updating i18n for DonationInterface-langonly | [production] | 
            
  | 19:02 | <Ryan_Lane> | pgehres Started syncing Wikimedia installation... : Updating i18n for DonationInterface-langonly | [production] | 
            
  | 19:02 | <Ryan_Lane> | binasher: shutting down mysql on db1043, converting to mariadb | [production] | 
            
  | 19:02 | <LeslieCarr> | powering back on spence | [production] | 
            
  | 19:02 | <Ryan_Lane> | asher synchronized wmf-config/db-eqiad.php  'pulling db1043 and shifting watchlist q's to db1050' | [production] | 
            
  | 19:02 | <Ryan_Lane> | pgehres synchronized php-1.21wmf10/extensions/DonationInterface/  'Updating DonatonInterface-langonly' | [production] | 
            
  | 19:02 | <Ryan_Lane> | LeslieCarr: powering off spence | [production] | 
            
  | 19:01 | <Ryan_Lane> | asher synchronized wmf-config/lucene.php  'sending search-pool4 traffic to pmtpa, where the index/host distribution has been rebalanced' | [production] | 
            
  | 19:01 | <Ryan_Lane> | notpeter: upgrading mariadb boxes to 5.5.29 | [production] | 
            
  | 19:01 | <Ryan_Lane> | tstarling synchronized wmf-config/InitialiseSettings.php | [production] | 
            
  | 19:01 | <Ryan_Lane> | Jeff_Green: stopped gmetad on neon for testing | [production] | 
            
  | 19:01 | <Ryan_Lane> | olivneh synchronized php-1.21wmf10/extensions/GettingStarted  'Update to split test (3/3)' | [production] | 
            
  | 19:00 | <Ryan_Lane> | olivneh synchronized php-1.21wmf10/extensions/E3Experiments  'Update to split test (2/3)' | [production] | 
            
  | 19:00 | <Ryan_Lane> | olivneh synchronized php-1.21wmf10/extensions/GuidedTour  'Update to split test (1/3)' | [production] | 
            
  | 19:00 | <Ryan_Lane> | mutante: create missing dump directory on streber, enables "RT-shredder" plugin which comes with RT since 3.8 | [production] | 
            
  | 19:00 | <Ryan_Lane> | olivneh synchronized php-1.21wmf10/extensions/EventLogging  'Updates to JavaScript API' | [production] | 
            
  | 19:00 | <Ryan_Lane> | changing A record for wikitech.wm.o and changing labsconsole.wm.o into a cname to wikitech | [production] | 
            
  | 18:59 | <Ryan_Lane> | disabled editing on wikitech.wikimedia.org | [production] | 
            
  | 18:59 | <Ryan_Lane> | binasher: resharding search-pool4 in pmtpa - restarted lsearchd on all local nodes and indexers on searchidx2 | [production] | 
            
  | 18:59 | <mwalker> | ... therefore the payments cluster is now running dced5328fd6ae4b395622abc41d2afa0ccad21c8 | [production] | 
            
  | 18:59 | <mutante> | restart nagios-nrpe,gmond,apache,salt-minion on bastion hosts | [production] | 
            
  | 18:59 | <Ryan_Lane> | binasher: rebuilding db35 from a hotbackup of db55 (s5); testing a new build of mariadb 5.5.29 | [production] | 
            
  | 18:59 | <Ryan_Lane> | reedy synchronized php-1.21wmf10/cache/interwiki.cdb  'Updating 1.21wmf10 interwiki cache' | [production] | 
            
  | 18:59 | <Ryan_Lane> | reedy synchronized php-1.21wmf10/cache/interwiki.cdb  'Updating 1.21wmf10 interwiki cache' | [production] | 
            
  | 18:58 | <Ryan_Lane> | reedy synchronized php-1.21wmf10/extensions/WikimediaMaintenance | [production] | 
            
  | 18:58 | <Ryan_Lane> | apergos: restarted icinga (not sure why it stopped, log file simply said 'caught TERM signal', in any case it complained that Error: Could not create external command file '/var/lib/nagios/rw/nagios.cmd' as named pipe ) | [production] | 
            
  | 18:58 | <mwalker> | enabled sofort forms for CH and DE on the payments cluster | [production] | 
            
  | 18:39 | <mutante> | installing package upgrades on bast1001 | [production] | 
            
  | 18:37 | <mutante> | installing package upgrades on fenari | [production] | 
            
  | 17:40 | <^demon> | rescanned all trackingids for gerrit on manganese | [production] | 
            
  | 17:31 | <^demon> | upgrading gerrit to 2.5.2-1506-g278aa9a on manganese/formey | [production] | 
            
  | 17:07 | <Ryan_Lane> | test | [production] |