| 
      
        2012-02-14
      
      §
     | 
  
    
  | 20:52 | 
  <asher> | 
  synchronized wmf-config/db.php  'setting s7 to read-only for master swap, db37 to be new master, db16 still out' | 
  [production] | 
            
  | 20:50 | 
  <asher> | 
  synchronized wmf-config/db.php  'setting s7 to read-only for master swap, db37 to be new master' | 
  [production] | 
            
  | 20:45 | 
  <reedy> | 
  synchronized wmf-config/InitialiseSettings.php  'Fix hewiki namespace talk typo' | 
  [production] | 
            
  | 20:41 | 
  <reedy> | 
  synchronized wmf-config/InitialiseSettings.php  'fix some mrwikisource aliases' | 
  [production] | 
            
  | 20:38 | 
  <reedy> | 
  synchronized wmf-config/InitialiseSettings.php  'fix some mrwikisource aliases' | 
  [production] | 
            
  | 20:33 | 
  <hashar> | 
  synchronized php-1.19/maintenance/purgeList.php  '[[rev:111480|r111480]] : enable purge of HTTPS URLs' | 
  [production] | 
            
  | 20:29 | 
  <mutante> | 
  purged https://office link, using modified purgeList.php that accepts https urls, thanks hashar | 
  [production] | 
            
  | 20:27 | 
  <hashar> | 
  synchronized php-1.18/maintenance/purgeList.php  '[[rev:111480|r111480]] : enable purge of HTTPS URLs' | 
  [production] | 
            
  | 20:13 | 
  <Ryan_Lane> | 
  built two raid6 arrays per labstore host. raid sets are initializing. | 
  [production] | 
            
  | 19:53 | 
  <reedy> | 
  synchronized wmf-config/db.php  'Bring db46 back in' | 
  [production] | 
            
  | 19:21 | 
  <RoanKattouw> | 
  Ran Varnish purge for 'office | 
  [production] | 
            
  | 19:19 | 
  <mutante> | 
  used purgeList.php on office.wm URLs, but it appears to be in varnish cache (broken redirect) | 
  [production] | 
            
  | 19:06 | 
  <AaronSchulz> | 
  gracefulled apaches to deal with APC corruption | 
  [production] | 
            
  | 18:57 | 
  <mutante> | 
  reverting the (circular) office redirect, syncing.. | 
  [production] | 
            
  | 18:49 | 
  <mutante> | 
  running sync-apache to fix office redirect | 
  [production] | 
            
  | 18:39 | 
  <reedy> | 
  synchronized wmf-config/db.php  'Comment out db46 from s6 due to really high lag from db schema updates' | 
  [production] | 
            
  | 16:02 | 
  <mutante> | 
  spence lost /home, mount was "Stale NFS file handle", causing outage of stats.wikimedia.org, fixed by remounting | 
  [production] | 
            
  | 15:48 | 
  <reedy> | 
  synchronized wmf-config/InitialiseSettings.php  'Bug 34378 - Rename namespaces on mr.wikisource.org' | 
  [production] | 
            
  | 15:39 | 
  <reedy> | 
  synchronized wmf-config/InitialiseSettings.php  'Bug 34342 - Create a new books namespace on he.wiki' | 
  [production] | 
            
  | 14:05 | 
  <reedy> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: testwiki to 1.19 | 
  [production] | 
            
  | 09:29 | 
  <tstarling> | 
  synchronized php-1.18/extensions/cldr/LanguageNames.body.php  '[[rev:111453|r111453]]' | 
  [production] | 
            
  | 07:19 | 
  <apergos> | 
  symlinkd wikidiff2.so to php_wikidiff2.so on searchidx2 | 
  [production] | 
            
  | 03:17 | 
  <reedy> | 
  synchronized wmf-config/ExtensionMessages-1.19.php  'Fix fr message file locations' | 
  [production] | 
            
  | 02:48 | 
  <reedy> | 
  synchronizing Wikimedia installation... : Rebuild messages | 
  [production] | 
            
  | 02:48 | 
  <tstarling> | 
  synchronized wmf-config/AdminSettings.php  'remove $wgUseRootUser and $wgUseNormalUser, broken since 1.17 and 1.16 respectively' | 
  [production] | 
            
  | 02:41 | 
  <reedy> | 
  synchronized php-1.19/extensions/Contest/Contest.php  'Comment out stupid die for the moment' | 
  [production] | 
            
  | 02:38 | 
  <reedy> | 
  synchronizing Wikimedia installation... : Rebuild messages | 
  [production] | 
            
  | 02:38 | 
  <tstarling> | 
  synchronized wmf-config/InitialiseSettings.php  're-enabling the collection extension' | 
  [production] | 
            
  | 02:35 | 
  <reedy> | 
  synchronizing Wikimedia installation... : Rebuild messages | 
  [production] | 
            
  | 02:26 | 
  <reedy> | 
  synchronized php-1.19/extensions/VisualEditor  | 
  [production] | 
            
  | 02:25 | 
  <reedy> | 
  synchronized php-1.19/extensions/FundraiserLandingPage/  | 
  [production] | 
            
  | 02:25 | 
  <LocalisationUpdate> | 
  failed (1.19) at Tue Feb 14 02:25:33 UTC 2012 | 
  [production] | 
            
  | 02:25 | 
  <LocalisationUpdate> | 
  completed (1.18) at Tue Feb 14 02:25:32 UTC 2012 | 
  [production] | 
            
  | 02:19 | 
  <reedy> | 
  synchronized wmf-config/ExtensionMessages-1.19.php  'Remove variablepage' | 
  [production] | 
            
  | 01:54 | 
  <Reedy> | 
  Make that ddsh -F5 | 
  [production] | 
            
  | 01:53 | 
  <Ryan_Lane> | 
  when rebooting hume I also applied security updates | 
  [production] | 
            
  | 01:52 | 
  <Tim> | 
  started indexer on searchidx2 with  /home/rainman/scripts/search-restart-indexer per docs | 
  [production] | 
            
  | 01:52 | 
  <Reedy> | 
  running ddsh -F30 -cM -g mediawiki-installation /usr/bin/sync-common | 
  [production] | 
            
  | 01:47 | 
  <Tim> | 
  rebooting srv193 | 
  [production] | 
            
  | 01:45 | 
  <Tim> | 
  on searchidx2: doing apt-get upgrade and rebooting | 
  [production] | 
            
  | 01:44 | 
  <Ryan_Lane> | 
  rebooting hume | 
  [production] | 
            
  | 01:28 | 
  <binasher> | 
  resuming 1.19 schema migrations after fenari reboot (on first s4 commons slave, db22) | 
  [production] | 
            
  | 01:19 | 
  <Tim> | 
  rebooting fenari for kernel upgrades | 
  [production] | 
            
  | 01:14 | 
  <Tim> | 
  doing apt-get upgrade on fenari | 
  [production] | 
            
  | 01:12 | 
  <Tim> | 
  rebooted fenari to fix stale NFS file handle | 
  [production] | 
            
  | 00:57 | 
  <LeslieCarr> | 
  rebooted nfs1 as it was unresponsive on console and via IP | 
  [production] | 
            
  | 00:37 | 
  <Reedy> | 
  killed /usr/local/apache/common/php-1.19 from apaches | 
  [production] |