| 
      
        2012-04-02
      
      §
     | 
  
    
  | 18:17 | 
  <RobH> | 
  removed old bin files on db1004 and prolly borked it by removing the wrong files | 
  [production] | 
            
  | 17:54 | 
  <nikerabbit> | 
  synchronized wmf-config/InitialiseSettings.php  '35436 - Enable Narayam at Hindi Wikipedia' | 
  [production] | 
            
  | 17:47 | 
  <preilly> | 
  synchronized wmf-config/InitialiseSettings.php  'add disable images option for default on zero domain' | 
  [production] | 
            
  | 17:45 | 
  <nikerabbit> | 
  synchronized wmf-config/InitialiseSettings.php  'Bug 35328 - Enable WebFonts for fr.wikisource.org' | 
  [production] | 
            
  | 17:40 | 
  <nikerabbit> | 
  synchronized php-1.19/languages/Names.php  'I18ndeploy [[rev:114656|r114656]]' | 
  [production] | 
            
  | 17:15 | 
  <preilly> | 
  carrier testing push for DIGI | 
  [production] | 
            
  | 17:15 | 
  <preilly> | 
  synchronized php-1.19/extensions/ZeroRatedMobileAccess/ZeroRatedMobileAccess.body.php  'changes for zero needed for carrier testing header of landing page only for mswiki' | 
  [production] | 
            
  | 16:46 | 
  <preilly> | 
  synchronized php-1.19/extensions/ZeroRatedMobileAccess/ZeroRatedMobileAccess.body.php  'changes for zero needed for carrier testing header of landing page only for mswiki' | 
  [production] | 
            
  | 02:16 | 
  <LocalisationUpdate> | 
  completed (1.19) at Mon Apr  2 02:16:47 UTC 2012 | 
  [production] | 
            
  
    | 
      
        2012-03-30
      
      §
     | 
  
    
  | 19:37 | 
  <hashar> | 
  configured jenkins on gallium to use smtp.pmtpa.wmnet as outgoing SMTP server | 
  [production] | 
            
  | 19:28 | 
  <RobH> | 
  puppet daemon restarted on brewster | 
  [production] | 
            
  | 18:13 | 
  <RobH> | 
  killing puppet daemon on brewster, i need to hack at local configuration for cisco server stuff | 
  [production] | 
            
  | 12:56 | 
  <mutante> | 
  db1047 - added system startup for /etc/init.d/mysql | 
  [production] | 
            
  | 12:47 | 
  <mutante> | 
  powercycling db1047 | 
  [production] | 
            
  | 12:28 | 
  <mutante> | 
  deleted old kernel sources on upgraded srvs for that little extra space during peaks, suggesting to nuke /usr/share/doc if there should be more disk space warnings | 
  [production] | 
            
  | 10:41 | 
  <mutante> | 
  same for srv223 | 
  [production] | 
            
  | 09:18 | 
  <mutante> | 
  srv224,srv219,srv220, upgrade apache, dist-upgrading w/ kernel, disabling ureadahead, rebooting one by one | 
  [production] | 
            
  | 08:06 | 
  <mutante> | 
  storage3 - gmond unable to find the metric information for any mysql_* .."module has not been loaded", starting mysql, running puppet ... | 
  [production] | 
            
  | 07:57 | 
  <mutante> | 
  powercycling storage3 | 
  [production] | 
            
  | 07:03 | 
  <Tim> | 
  running [[bugzilla:35578|bug 35578]] cleanup script in screen on fenari | 
  [production] | 
            
  | 06:41 | 
  <tstarling> | 
  synchronized php-1.19/includes/specials/SpecialBlock.php  | 
  [production] | 
            
  | 06:40 | 
  <tstarling> | 
  synchronized php-1.19/includes/specials/SpecialBlock.php  | 
  [production] | 
            
  | 06:39 | 
  <tstarling> | 
  synchronized php-1.19/includes/specials/SpecialBlock.php  | 
  [production] | 
            
  | 06:15 | 
  <Tim> | 
  killed vi on fenari owned by awjrichards, locking CommonSettings.php for two days | 
  [production] | 
            
  | 02:17 | 
  <LocalisationUpdate> | 
  completed (1.19) at Fri Mar 30 02:17:56 UTC 2012 | 
  [production] | 
            
  | 01:13 | 
  <reedy> | 
  synchronized wmf-config/CommonSettings.php  'Remove more crap' | 
  [production] | 
            
  | 01:02 | 
  <reedy> | 
  synchronized wmf-config/InitialiseSettings.php  'Remove some dupe code' | 
  [production] | 
            
  | 01:00 | 
  <reedy> | 
  synchronized wmf-config/InitialiseSettings.php  'Remove wmgUsabilityPrefSwitch' | 
  [production] | 
            
  | 00:59 | 
  <reedy> | 
  synchronized wmf-config/CommonSettings.php  'Remove wmgUsabilityPrefSwitch' | 
  [production] | 
            
  | 00:58 | 
  <reedy> | 
  synchronized wmf-config/InitialiseSettings.php  'Remove unused wmgUseUsabilityInitiativeAlpha' | 
  [production] | 
            
  
    | 
      
        2012-03-29
      
      §
     | 
  
    
  | 23:49 | 
  <aaron> | 
  synchronized php-1.19/includes/revisiondelete/RevisionDeleteUser.php  'deployed [[rev:114619|r114619]]' | 
  [production] | 
            
  | 21:20 | 
  <LeslieCarr> | 
  rebooting db47 | 
  [production] | 
            
  | 20:28 | 
  <reedy> | 
  synchronized wmf-config/InitialiseSettings.php  'Swap wgUseCommaCount to wgArticleCountMethod' | 
  [production] | 
            
  | 20:07 | 
  <notpeter> | 
  restarting lsearchd on search2 to del the logfile to end all logfiles | 
  [production] | 
            
  | 20:05 | 
  <RoanKattouw> | 
  Stopping and starting Gerrit on manganese to apply Chad's change of the -1 text in the DB | 
  [production] | 
            
  | 20:02 | 
  <notpeter> | 
  restarting lsearchd on search7 to del the logfile to end all logfiles | 
  [production] | 
            
  | 18:11 | 
  <catrope> | 
  synchronized php-1.19/extensions/ClickTracking/ClickTracking.hooks.php  | 
  [production] | 
            
  | 17:59 | 
  <RobH> | 
  search1021 coming back up, done with tests | 
  [production] | 
            
  | 17:53 | 
  <RobH> | 
  search1021 coming down for ssd fit test | 
  [production] | 
            
  | 17:07 | 
  <notpeter> | 
  disabling notifications for search lvs nagios checks for 24 hours to test fix | 
  [production] | 
            
  | 15:42 | 
  <notpeter> | 
  finished clearning up all pmtpa search hosts. hey look! they all have lots of space now! | 
  [production] | 
            
  | 15:15 | 
  <notpeter> | 
  restarting lsearchd on search3 | 
  [production] | 
            
  | 15:02 | 
  <RobH> | 
  brewster puppet re-enabled | 
  [production] | 
            
  | 15:02 | 
  <RobH> | 
  virt1001 pxe boots via dhcp and fails tftp download, i have to hold off on further troubleshooting until i have a network admin | 
  [production] | 
            
  | 14:47 | 
  <RobH> | 
  did virt1001 wrong, reupdating dns | 
  [production] | 
            
  | 14:39 | 
  <RobH> | 
  all nameservers still online after udpate | 
  [production] |