| 
      
        2013-01-15
      
      §
     | 
  
    
  | 23:21 | 
  <reedy> | 
  synchronized wmf-config/InitialiseSettings.php  'Bug 44015 - Add en.wikivoyage autopatrolled group' | 
  [production] | 
            
  | 22:20 | 
  <mutante> | 
  powercycling msfe1002 | 
  [production] | 
            
  | 22:19 | 
  <RobH> | 
  authdns-update | 
  [production] | 
            
  | 22:14 | 
  <reedy> | 
  synchronized wmf-config/InitialiseSettings.php  'wgUseRCPatrol on for wikivoyages' | 
  [production] | 
            
  | 21:32 | 
  <mutante> | 
  setting knsq17 to enabled: False in pybal esams/uploads-squids as it is broken per RT-4321 | 
  [production] | 
            
  | 21:17 | 
  <RobH> | 
  cerium & titanium being installed as parsoid caching servers, ignore any alerts for now. | 
  [production] | 
            
  | 20:20 | 
  <paravoid> | 
  stopping squid3 on sq48 | 
  [production] | 
            
  | 20:00 | 
  <Nemo_bis> | 
  nikerabbit rebuilt message index on all Translate-enabled wikis (half a hour ago) | 
  [production] | 
            
  | 19:54 | 
  <RobH> | 
  authdns-update | 
  [production] | 
            
  | 19:15 | 
  <RobH> | 
  authdns-update | 
  [production] | 
            
  | 18:36 | 
  <mutante> | 
  stopping Apache on sanger | 
  [production] | 
            
  | 16:01 | 
  <hashar> | 
  Made a mistake in Zuul configuration. A slight outage while correcting it :/ | 
  [production] | 
            
  | 12:55 | 
  <nikerabbit> | 
  synchronized php-1.21wmf7/extensions/Translate/  'Translate to master again' | 
  [production] | 
            
  | 12:09 | 
  <nikerabbit> | 
  synchronized wmf-config/CommonSettings.php  'Translate config fix' | 
  [production] | 
            
  | 10:26 | 
  <nikerabbit> | 
  synchronized php-1.21wmf7/extensions/Translate  'Translate to master' | 
  [production] | 
            
  | 10:18 | 
  <nikerabbit> | 
  synchronized php-1.21wmf7/includes/HTMLForm.php | 
  [production] | 
            
  | 10:18 | 
  <nikerabbit> | 
  synchronized php-1.21wmf7/includes/Preferences.php | 
  [production] | 
            
  | 10:17 | 
  <nikerabbit> | 
  synchronized php-1.21wmf7/includes/AutoLoader.php | 
  [production] | 
            
  | 10:04 | 
  <nikerabbit> | 
  synchronized wmf-config/InitialiseSettings.php  'Enable ULS and disable WebFonts/Narayam on Translate wikis' | 
  [production] | 
            
  | 10:03 | 
  <nikerabbit> | 
  synchronized wmf-config/CommonSettings.php  'Updated ULS configuration' | 
  [production] | 
            
  | 09:42 | 
  <nikerabbit> | 
  synchronized php-1.21wmf7/extensions/UniversalLanguageSelector/  'ULS to master' | 
  [production] | 
            
  | 08:33 | 
  <hashar> | 
  gallium : installing liblua5.1-0-dev package. | 
  [production] | 
            
  | 04:50 | 
  <binasher> | 
  set pc[1-3] to replicate from pc100[1-3], full master/master all the way across the sky | 
  [production] | 
            
  | 02:26 | 
  <LocalisationUpdate> | 
  completed (1.21wmf7) at Tue Jan 15 02:26:37 UTC 2013 | 
  [production] | 
            
  | 02:16 | 
  <LeslieCarr> | 
  rebooting neon | 
  [production] | 
            
  | 02:02 | 
  <LeslieCarr> | 
  created a new lvm for /var/log on neon, copied files over, and restarted processes.  some file permission issues and incorrect file owners may remain | 
  [production] | 
            
  | 01:39 | 
  <LeslieCarr> | 
  restarting neon | 
  [production] | 
            
  | 01:23 | 
  <aaron> | 
  synchronized wmf-config/InitialiseSettings.php  'Set sqlbagostuff log' | 
  [production] | 
            
  | 01:12 | 
  <mutante> | 
  neon - changing logrotate config for atop to rotate 7 instead of 14 | 
  [production] | 
            
  | 01:09 | 
  <mutante> | 
  deleting atop.log.1* in /var/log on neon to free disk space | 
  [production] | 
            
  | 00:47 | 
  <binasher> | 
  the production persistent parsercache is now on pc[1-3], replicated to pc100[1-3] | 
  [production] | 
            
  | 00:43 | 
  <asher> | 
  synchronized wmf-config/CommonSettings.php  'deploying db parsercache, sharded and replicated' | 
  [production] | 
            
  
    | 
      
        2013-01-14
      
      §
     | 
  
    
  | 23:18 | 
  <Tim> | 
  changed CNAME for search.wikimedia.org to point to wikimedia-lb instead of ekrem | 
  [production] | 
            
  | 22:46 | 
  <mutante> | 
  installing apache,mysql-common,perl,sudo package updates on sanger | 
  [production] | 
            
  | 22:45 | 
  <mutante> | 
  killed java process on sanger, restarted opendj | 
  [production] | 
            
  | 22:10 | 
  <hashar> | 
  pullen {{gerrit|43849}} on fenari (simply changes some comments) | 
  [production] | 
            
  | 22:04 | 
  <Reedy> | 
  Modified rc_params on huwiki to make it a blob | 
  [production] | 
            
  | 21:30 | 
  <Jeff_Green> | 
  tearing down community-analytics vhost and cname | 
  [production] | 
            
  | 21:26 | 
  <binasher> | 
  rebooting pc1 | 
  [production] | 
            
  | 20:36 | 
  <reedy> | 
  cleared profiling data | 
  [production] | 
            
  | 20:33 | 
  <RobH> | 
  xenon & caesium added to parsoid pybal config | 
  [production] | 
            
  | 20:27 | 
  <reedy> | 
  synchronized wmf-config/InitialiseSettings.php  'Enable WikibaseClient on huwiki' | 
  [production] | 
            
  | 20:19 | 
  <reedy> | 
  synchronized wmf-config/InitialiseSettings.php | 
  [production] | 
            
  | 20:07 | 
  <reedy> | 
  synchronized wmf-config/ | 
  [production] | 
            
  | 19:24 | 
  <reedy> | 
  synchronized php-1.21wmf7/extensions/Wikibase | 
  [production] | 
            
  | 19:09 | 
  <cmjohnson1> | 
  provision sq48 | 
  [production] | 
            
  | 19:05 | 
  <reedy> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: Rest of the 'pedias to 1.21wmf7 | 
  [production] | 
            
  | 18:29 | 
  <Ryan_Lane> | 
  adding glusterfs-client glusterfs-common glusterfs-server and glusterfs-dbg for lucid and precise to our repo | 
  [production] | 
            
  | 17:20 | 
  <reedy> | 
  synchronized php-1.21wmf7/includes/ChangesList.php | 
  [production] | 
            
  | 16:55 | 
  <ottomata> | 
  packet loss on oxygen after adding new wikipedia zero filters.  Stopping puppet to temporarily comment them out and make sure they are the problem. | 
  [production] |