| 
      
        2013-01-30
      
      §
     | 
  
    
  | 19:06 | 
  <reedy> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: Everything else to 1.21wmf8 | 
  [production] | 
            
  | 18:56 | 
  <reedy> | 
  synchronized wmf-config/InitialiseSettings.php | 
  [production] | 
            
  | 18:48 | 
  <Ryan_Lane> | 
  downgrading memcached to ubuntu version on virt0 | 
  [production] | 
            
  | 17:18 | 
  <Reedy> | 
  Created wb_changes_dispatch on wikidatawiki | 
  [production] | 
            
  | 16:55 | 
  <Reedy> | 
  Created sites/site_identifier tables on hewiki and itwiki | 
  [production] | 
            
  | 16:34 | 
  <cmjohnson1> | 
  mw1072 going to down to check h/w state currently failed rt4381 | 
  [production] | 
            
  | 15:38 | 
  <hashar> | 
  synchronized wmf-config/InitialiseSettings.php  '([[bugzilla:44395|bug 44395]]) Allow bureaucrats to remove the translateadmin group on wikidatawiki {{gerrit|46011}}' | 
  [production] | 
            
  | 15:37 | 
  <hashar> | 
  synchronized wmf-config/throttle.php  '([[bugzilla:44506|bug 44506]]) raise throttle for an Israel editthon {{gerrit|46747}}' | 
  [production] | 
            
  | 12:54 | 
  <maxsem> | 
  synchronized php-1.21wmf7/extensions/GeoData | 
  [production] | 
            
  | 12:52 | 
  <maxsem> | 
  synchronized php-1.21wmf8/extensions/GeoData | 
  [production] | 
            
  | 05:55 | 
  <Tim> | 
  rebooted kaulen, was not responding on network or serial console | 
  [production] | 
            
  | 03:11 | 
  <mutante> | 
  wikimania.asia redirect now works, it did not before because of a bug in apache-graceful-all not actually restarting Apaches in eqiad | 
  [production] | 
            
  | 03:07 | 
  <mutante> | 
  reloading Apaches in eqiad using manual dsh (because apache-graceful-all sanity check has a bug) | 
  [production] | 
            
  | 02:55 | 
  <catrope> | 
  gracefulled all apaches | 
  [production] | 
            
  | 02:52 | 
  <LocalisationUpdate> | 
  completed (1.21wmf7) at Wed Jan 30 02:52:03 UTC 2013 | 
  [production] | 
            
  | 02:39 | 
  <dzahn> | 
  gracefulled all apaches | 
  [production] | 
            
  | 02:28 | 
  <LocalisationUpdate> | 
  completed (1.21wmf8) at Wed Jan 30 02:28:22 UTC 2013 | 
  [production] | 
            
  | 00:44 | 
  <dzahn> | 
  synchronized ./wmf-config/InitialiseSettings.php | 
  [production] | 
            
  | 00:43 | 
  <mutante> | 
  sync InitialiseSettings.php for [[bugzilla:44473|bug 44473]] | 
  [production] | 
            
  | 00:19 | 
  <binasher> | 
  restarted pdns-recursor on dobson | 
  [production] | 
            
  | 00:13 | 
  <reedy> | 
  synchronized wmf-config/ | 
  [production] | 
            
  
    | 
      
        2013-01-29
      
      §
     | 
  
    
  | 23:54 | 
  <reedy> | 
  synchronized wmf-config/ | 
  [production] | 
            
  | 23:21 | 
  <pgehres> | 
  synchronized php-1.21wmf8/extensions/CentralNotice/  'Updating CentralNotice to master, resolving namespace issues' | 
  [production] | 
            
  | 23:17 | 
  <pgehres> | 
  synchronized php-1.21wmf7/extensions/CentralNotice/  'Updating CentralNotice to master, resolving namespace issues' | 
  [production] | 
            
  | 23:00 | 
  <maxsem> | 
  synchronized wmf-config/InitialiseSettings.php  'Enable Special:Nearby everywhere' | 
  [production] | 
            
  | 22:51 | 
  <maxsem> | 
  synchronized php-1.21wmf7/extensions/MobileFrontend  'https://gerrit.wikimedia.org/r/46630' | 
  [production] | 
            
  | 22:48 | 
  <maxsem> | 
  synchronized php-1.21wmf8/extensions/MobileFrontend  'https://gerrit.wikimedia.org/r/46630' | 
  [production] | 
            
  | 22:47 | 
  <Tim> | 
  rotated SAL wikitech page - test logging | 
  [production] | 
            
  | 22:27 | 
  <RobH> | 
  torrus is all happy again, huzzah | 
  [production] | 
            
  | 22:25 | 
  <maxsem> | 
  Finished syncing Wikimedia installation... : https://www.mediawiki.org/wiki/Extension:MobileFrontend/Deployments/2013-01-29 | 
  [production] | 
            
  | 22:11 | 
  <maxsem> | 
  Started syncing Wikimedia installation... : https://www.mediawiki.org/wiki/Extension:MobileFrontend/Deployments/2013-01-29 | 
  [production] | 
            
  | 22:02 | 
  <Ryan_Lane> | 
  rebooting labstore2 | 
  [production] | 
            
  | 22:01 | 
  <RobH> | 
  updating torrus files via puppet, sorry if torrus crashes out, its bitchy | 
  [production] | 
            
  | 22:00 | 
  <notpeter> | 
  temp stopping puppet on brewster | 
  [production] | 
            
  | 21:58 | 
  <Ryan_Lane> | 
  rebooting labstore1 | 
  [production] | 
            
  | 21:57 | 
  <Ryan_Lane> | 
  stopping all gluster services | 
  [production] | 
            
  | 21:38 | 
  <cmjohnson1> | 
  authdns update adding mgmt ip's to zone files for hpm misc servers in eqiad | 
  [production] | 
            
  | 21:33 | 
  <Ryan_Lane> | 
  stopping gluster services on labstore1, this will affect nfs mounts (like ssh keys and public data exports) | 
  [production] | 
            
  | 20:43 | 
  <RobH> | 
  updated rt config, pmtpa comments now come from pmtpa-comment, not rt-comment to match the rest of the queues configuration | 
  [production] | 
            
  | 19:51 | 
  <cmjohnson1> | 
  ms-be1004 going down for h/w replacement | 
  [production] | 
            
  | 19:47 | 
  <hashar> | 
  jenkins: updating all jobs | 
  [production] | 
            
  | 19:31 | 
  <cmjohnson1> | 
  ms-be1003 going down to replace h/w | 
  [production] | 
            
  | 19:29 | 
  <Ryan_Lane> | 
  upgrading glusterfs to 3.3.1 from 3.3.0 and replacing old glusterfs package with new packages | 
  [production] | 
            
  | 19:09 | 
  <cmjohnson1> | 
  ms-be1002 going down to replace h/w | 
  [production] | 
            
  | 18:37 | 
  <cmjohnson1> | 
  ms-be1001 going down to replace h/w | 
  [production] | 
            
  | 17:57 | 
  <andrewbogott> | 
  restarted apache and memcached on virt0 (apparently fixing main page errors) | 
  [production] | 
            
  | 17:10 | 
  <demon> | 
  synchronized wmf-config/InitialiseSettings.php | 
  [production] | 
            
  | 14:14 | 
  <mark> | 
  powercycled sodium | 
  [production] | 
            
  | 08:20 | 
  <pgehres> | 
  synchronized php-1.21wmf8/extensions/CentralNotice/  'Updating CentralNotice to master' | 
  [production] | 
            
  | 04:58 | 
  <tstarling> | 
  synchronized php-1.21wmf6/cache/trusted-xff.cdb | 
  [production] |