| 
      
        2013-06-19
      
      §
     | 
  
    
  | 21:05 | 
  <RobH> | 
  rebooting tridge due to updates | 
  [production] | 
            
  | 20:56 | 
  <Ryan_Lane> | 
  updating OpenStackManager on virt0 to 3e92211313ed8b73ee44fb395230bafb14dc9bdc | 
  [production] | 
            
  | 20:34 | 
  <ottomata> | 
  analytics1019 and analytics1020 are fully puppetized.  rebalancing them back into the hadoop cluster | 
  [production] | 
            
  | 18:41 | 
  <nikerabbit> | 
  synchronized php-1.22wmf6/extensions/UniversalLanguageSelector/  'ULS to master again' | 
  [production] | 
            
  | 18:34 | 
  <gwicke> | 
  updated Parsoid to ddc47d8 | 
  [production] | 
            
  | 18:34 | 
  <nikerabbit> | 
  synchronized php-1.22wmf7/extensions/UniversalLanguageSelector/  'ULS to master' | 
  [production] | 
            
  | 18:31 | 
  <asher> | 
  synchronized wmf-config/db-pmtpa.php  'pull db64' | 
  [production] | 
            
  | 18:28 | 
  <mutante> | 
  sync-apache and graceful for votewiki rewrites (change 67203) | 
  [production] | 
            
  | 18:14 | 
  <nikerabbit> | 
  synchronized php-1.22wmf6/extensions/UniversalLanguageSelector/  'ULS to master' | 
  [production] | 
            
  | 15:01 | 
  <cmjohnson1> | 
  authdns update | 
  [production] | 
            
  | 14:57 | 
  <cmjohnson1> | 
  powering down and relocating searchidx1001 | 
  [production] | 
            
  | 14:54 | 
  <Jeff_Green> | 
  update & reboot boron | 
  [production] | 
            
  | 14:35 | 
  <cmjohnson1> | 
  powering down mw1171 to replace hard drive and reinstalling | 
  [production] | 
            
  | 13:51 | 
  <cmjohnson1> | 
  powering down carbon to replace bad disk | 
  [production] | 
            
  | 10:00 | 
  <hashar> | 
  synchronized wmf-config  '{{gerrit|69466}} beta: $wmg for AbuseFilterCentralDB' | 
  [production] | 
            
  | 09:59 | 
  <hashar> | 
  synchronized wmf-config/CommonSettings.php  '{{gerrit|69466}} beta: $wmg for AbuseFilterCentralDB' | 
  [production] | 
            
  | 02:18 | 
  <LocalisationUpdate> | 
  ResourceLoader cache refresh completed at Wed Jun 19 02:18:52 UTC 2013 | 
  [production] | 
            
  | 02:15 | 
  <catrope> | 
  synchronized php-1.22wmf7/resources/startup.js  'touch' | 
  [production] | 
            
  | 02:15 | 
  <catrope> | 
  synchronized php-1.22wmf6/resources/startup.js  'touch' | 
  [production] | 
            
  | 02:14 | 
  <LocalisationUpdate> | 
  completed (1.22wmf6) at Wed Jun 19 02:14:07 UTC 2013 | 
  [production] | 
            
  | 02:07 | 
  <LocalisationUpdate> | 
  completed (1.22wmf7) at Wed Jun 19 02:07:42 UTC 2013 | 
  [production] | 
            
  | 02:04 | 
  <catrope> | 
  Finished syncing Wikimedia installation... : Updating VisualEditor to master | 
  [production] | 
            
  | 01:50 | 
  <catrope> | 
  Started syncing Wikimedia installation... : Updating VisualEditor to master | 
  [production] | 
            
  | 00:56 | 
  <gwicke> | 
  updated Parsoid to 0445108 | 
  [production] | 
            
  
    | 
      
        2013-06-18
      
      §
     | 
  
    
  | 22:51 | 
  <olivneh> | 
  synchronized wmf-config/CommonSettings-labs.php  'Iaade2591b: Configure $wgEventLoggingBaseUri & $wgEventLoggingFile on beta cluster' | 
  [production] | 
            
  | 22:30 | 
  <olivneh> | 
  synchronized wmf-config/InitialiseSettings-labs.php  'Ibeb07a4b9: Re-enable EventLogging on beta cluster' | 
  [production] | 
            
  | 22:29 | 
  <olivneh> | 
  synchronized wmf-config/CommonSettings.php  'I7fc1a5ad4: Ensure forward-compatibility by setting ' | 
  [production] | 
            
  | 22:15 | 
  <mutante> | 
  scheduled 24 hour downtime for eqiad search-pools to stop paging | 
  [production] | 
            
  | 22:04 | 
  <mutante> | 
  rebooting iron to finish upgrade | 
  [production] | 
            
  | 22:03 | 
  <maxsem> | 
  synchronized php-1.22wmf6/extensions/MobileFrontend/ | 
  [production] | 
            
  | 21:54 | 
  <maxsem> | 
  synchronized php-1.22wmf6/extensions/MobileFrontend/ | 
  [production] | 
            
  | 21:49 | 
  <maxsem> | 
  Finished syncing Wikimedia installation... : Weekly mobile deployment | 
  [production] | 
            
  | 21:38 | 
  <mutante> | 
  apache-graceful-all for redirects | 
  [production] | 
            
  | 21:36 | 
  <maxsem> | 
  Started syncing Wikimedia installation... : Weekly mobile deployment | 
  [production] | 
            
  | 20:40 | 
  <LeslieCarr> | 
  added temp logging filter to cr2-eqiad analytics4 filter | 
  [production] | 
            
  | 20:34 | 
  <mutante> | 
  upgrading iron to precise | 
  [production] | 
            
  | 19:43 | 
  <kaldari> | 
  synchronized wmf-config/InitialiseSettings.php  'Turn Disambiguator on on en.wiki' | 
  [production] | 
            
  | 19:16 | 
  <kaldari> | 
  Finished syncing Wikimedia installation... : | 
  [production] | 
            
  | 18:57 | 
  <kaldari> | 
  Started syncing Wikimedia installation... : | 
  [production] | 
            
  | 18:49 | 
  <mutante> | 
  rebooting iron | 
  [production] | 
            
  | 18:44 | 
  <mutante> | 
  installing kernel and package upgrades on iron | 
  [production] | 
            
  | 18:32 | 
  <mwalker> | 
  updating payments cluster (listeners) from aa0039104e71417845ec14a5ad666c2de464cfab to 8ca174296d570b6fda365cab37b2222d81c47c3d for adyen USD capture bug | 
  [production] | 
            
  | 18:11 | 
  <greg-g> | 
  it's better than bad, it's good! | 
  [production] | 
            
  | 18:08 | 
  <RobH> | 
  log log log log | 
  [production] | 
            
  | 18:08 | 
  <mutante> | 
  LVS monitoring pages caused by neon recovering | 
  [production] | 
            
  | 18:07 | 
  <RobH> | 
  ignore eqiad based search errors for now, as tampa is hosting all search traffic while hardware is repaired | 
  [production] | 
            
  | 18:07 | 
  <RobH> | 
  search is based in tampa | 
  [production] |