| 2013-05-07
      
      § | 
    
  | 16:28 | <reedy> | synchronized multiversion/MWMultiVersion.php | [production] | 
            
  | 16:19 | <gwicke> | updated Parsoid to current master | [production] | 
            
  | 16:03 | <reedy> | synchronized multiversion/MWMultiVersion.php | [production] | 
            
  | 15:56 | <reedy> | synchronized wmf-config/InitialiseSettings.php | [production] | 
            
  | 15:45 | <reedy> | synchronized docroot | [production] | 
            
  | 14:47 | <cmjohnson1> | authdns update | [production] | 
            
  | 14:32 | <cmjohnson1> | authdns update | [production] | 
            
  | 13:09 | <mark> | Power cycled line card 2 in csw1-esams (which has the SSL servers) to fix esams SSL reachability issues | [production] | 
            
  | 09:15 | <nikerabbit> | synchronized wmf-config/CommonSettings.php  'New ULS config part 2' | [production] | 
            
  | 09:14 | <nikerabbit> | synchronized wmf-config/InitialiseSettings.php  'New ULS config part 1' | [production] | 
            
  | 09:09 | <nikerabbit> | synchronized php-1.22wmf3/extensions/UniversalLanguageSelector/  'ULS to master' | [production] | 
            
  | 08:20 | <paravoid> | ceph: all radosgw & mon + osd.24 upgraded to 0.56.6 | [production] | 
            
  | 07:59 | <paravoid> | partially upgrading ceph to 0.56.6 | [production] | 
            
  | 07:27 | <Tim> | on holmium: reverted firewall, too hard | [production] | 
            
  | 07:22 | <Tim> | on holmium: added some ad hoc iptables rules intended to keep the rest of the site mostly safe from it overnight | [production] | 
            
  | 06:59 | <Tim> | on holmium: chown'd to root all script files writable by www-data and all directories which are configured to allow PHP execution | [production] | 
            
  | 06:27 | <Tim> | on marmontel: disabled apache logrotate, stopped apache, disabled apache init script | [production] | 
            
  | 04:19 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Tue May  7 04:19:39 UTC 2013 | [production] | 
            
  | 02:25 | <LocalisationUpdate> | completed (1.22wmf2) at Tue May  7 02:25:11 UTC 2013 | [production] | 
            
  | 02:16 | <LocalisationUpdate> | completed (1.22wmf3) at Tue May  7 02:16:06 UTC 2013 | [production] | 
            
  | 01:29 | <bsitu> | synchronized php-1.22wmf3/includes/User.php  'Two fixes to User::getNewMessageRevisionId() from kaldari' | [production] | 
            
  | 01:28 | <reedy> | synchronized php-1.22wmf3/extensions/WikimediaMaintenance | [production] | 
            
  | 00:38 | <reedy> | synchronized php-1.22wmf3/includes/User.php  'Live hack to stop fatals' | [production] | 
            
  | 00:06 | <bsitu> | synchronized php-1.22wmf3/extensions/Echo  'Update Echo to master' | [production] | 
            
  | 00:06 | <bsitu> | synchronized php-1.22wmf3/docs/hooks.txt  'Adding GetNewMessagesAlert hook and wgUserNewMsgRevisionId JS global' | [production] | 
            
  | 00:06 | <bsitu> | synchronized php-1.22wmf3/RELEASE-NOTES-1.22  'Adding GetNewMessagesAlert hook and wgUserNewMsgRevisionId JS global' | [production] | 
            
  | 00:05 | <bsitu> | synchronized php-1.22wmf3/includes/WikiPage.php  'Adding GetNewMessagesAlert hook and wgUserNewMsgRevisionId JS global' | [production] | 
            
  | 00:03 | <bsitu> | synchronized php-1.22wmf3/includes/User.php  'Adding GetNewMessagesAlert hook and wgUserNewMsgRevisionId JS global' | [production] | 
            
  | 00:02 | <bsitu> | synchronized php-1.22wmf3/includes/Skin.php  'Adding GetNewMessagesAlert hook and wgUserNewMsgRevisionId JS global' | [production] | 
            
  | 00:01 | <bsitu> | synchronized php-1.22wmf3/includes/OutputPage.php  'Adding GetNewMessagesAlert hook and wgUserNewMsgRevisionId JS global' | [production] | 
            
  
    | 2013-05-06
      
      § | 
    
  | 23:25 | <mutante> | restarting prelabsdbdb on db1054 | [production] | 
            
  | 22:47 | <mutante> | apache-graceful-all, add iegcom virtual host | [production] | 
            
  | 21:00 | <mutante> | blog is back up | [production] | 
            
  | 20:59 | <mutante> | holmium: bring up interface lo, comment "search" line in resolv.conf, restart varnish, ... | [production] | 
            
  | 20:46 | <paravoid> | swift replication scripts running on ms-fe1002 under screen -- kill if they have any effect on production | [production] | 
            
  | 20:14 | <mutante> | DNS update - sync with ns2 | [production] | 
            
  | 20:04 | <mutante> | restarting parsoid on wtp1001 | [production] | 
            
  | 20:02 | <mutante> | restarting pdns on ns0 | [production] | 
            
  | 20:00 | <mutante> | killing parsoid server.js on wtp1001 | [production] | 
            
  | 19:59 | <aaron> | synchronized multiversion/MWMultiVersion.php  'f257ddac39194f8bfcfd145104999fdba83d33f5' | [production] | 
            
  | 19:58 | <mutante> | DNS update - switch all blog cnames over to holmium | [production] | 
            
  | 19:29 | <reedy> | synchronized php-1.22wmf3/extensions/ZeroRatedMobileAccess/ | [production] | 
            
  | 19:24 | <LeslieCarr> | moved gateway from csw1-esams to cr1-esams | [production] | 
            
  | 19:18 | <Krinkle> | Reloading Zuul to deploy I259585e78ee | [production] | 
            
  | 19:17 | <LeslieCarr> | rebooting ns2 | [production] | 
            
  | 19:15 | <reedy> | synchronized wmf-config/ | [production] | 
            
  | 19:13 | <binasher> | rebuilt db1016 from hotbackup of db1001; pmtpa side of m1 needs to be rebuilt as well | [production] | 
            
  | 18:41 | <RobH> | ns2/nescio appears offline, investigating | [production] | 
            
  | 18:37 | <RobH> | ns2 appears offline | [production] | 
            
  | 18:36 | <RobH> | authdns-update crashed on ns2, rerunning | [production] |