| 
      
        2013-08-19
      
      §
     | 
  
    
  | 19:09 | 
  <Reedy> | 
  mw1046 reporting readonly filesystem | 
  [production] | 
            
  | 19:03 | 
  <reedy> | 
  rebuilt wikiversions.cdb and synchronized wikiversions files: special, closed, wikimedia, wikinews, wikivoyage to 1.22wmf13 | 
  [production] | 
            
  | 18:05 | 
  <csteipp> | 
  Started syncing Wikimedia installation... : | 
  [production] | 
            
  | 17:56 | 
  <paravoid> | 
  rebooting rubidium, mexia, eeden -- kernel upgrade | 
  [production] | 
            
  | 17:51 | 
  <paravoid> | 
  apt: new geoip-database package version, from 2 years later | 
  [production] | 
            
  | 17:42 | 
  <csteipp> | 
  synchronized php-1.22wmf12/extensions/CentralAuth  'revert ca update' | 
  [production] | 
            
  | 17:35 | 
  <csteipp> | 
  synchronized php-1.22wmf12/extensions/CentralAuth  'Updating wmf12 CentralAuth to master for OAuth' | 
  [production] | 
            
  | 17:12 | 
  <cmjohnson1> | 
  removing sq41 from dsh node groups and pybal  | 
  [production] | 
            
  | 16:33 | 
  <csteipp> | 
  synchronized php-1.22wmf13/extensions/CentralAuth  'Update CentralAuth to master for OAuth hooks' | 
  [production] | 
            
  | 14:44 | 
  <paravoid> | 
  authdns-update: remove trailing dot from ulsfo reverse zone names | 
  [production] | 
            
  | 13:16 | 
  <springle> | 
  temporarily running pt-kill on s5 slaves for slow wikidata queries | 
  [production] | 
            
  | 12:49 | 
  <paravoid> | 
  hopefully final swiftrepl (swift->ceph thumbs) run on ms-fe1002 | 
  [production] | 
            
  | 12:18 | 
  <tstarling> | 
  synchronized php-1.22wmf12/includes/GlobalFunctions.php  'revert temp hack, superseded' | 
  [production] | 
            
  | 12:17 | 
  <faidon> | 
  synchronized wmf-config/filebackend.php  'ceph as a filebackend' | 
  [production] | 
            
  | 11:58 | 
  <tstarling> | 
  synchronized wmf-config/CommonSettings.php  'changing session cookie name due to bug 53032' | 
  [production] | 
            
  | 07:55 | 
  <tstarling> | 
  synchronized php-1.22wmf12/includes/GlobalFunctions.php  'hack for session bug' | 
  [production] | 
            
  | 07:21 | 
  <paravoid> | 
  switching text-varnish pybal group back to squids | 
  [production] | 
            
  | 07:20 | 
  <Tim> | 
  destroyed leaked session with ID 796c2b... | 
  [production] | 
            
  | 05:49 | 
  <paravoid> | 
  authdns-update: add ADSP (dkim=unknown), DKIM policy record for lists (o=~) | 
  [production] | 
            
  | 04:18 | 
  <paravoid> | 
  authdns-update: DKIM & DMARC (both no-op, for now) | 
  [production] | 
            
  | 03:00 | 
  <LocalisationUpdate> | 
  ResourceLoader cache refresh completed at Mon Aug 19 03:00:26 UTC 2013 | 
  [production] | 
            
  | 02:41 | 
  <LocalisationUpdate> | 
  completed (1.22wmf13) at Mon Aug 19 02:41:19 UTC 2013 | 
  [production] | 
            
  | 02:15 | 
  <LocalisationUpdate> | 
  completed (1.22wmf12) at Mon Aug 19 02:15:45 UTC 2013 | 
  [production] | 
            
  
    | 
      
        2013-08-16
      
      §
     | 
  
    
  | 21:57 | 
  <demon> | 
  synchronized php-1.22wmf12/extensions/CirrusSearch  'No actual changes, was just a change + revert' | 
  [production] | 
            
  | 21:46 | 
  <ori-l> | 
  Timeouts on last two sync-dirs: srv281, mw1089, mw1173 | 
  [production] | 
            
  | 21:46 | 
  <olivneh> | 
  synchronized php-1.22wmf13/extensions/CoreEvents  'Updating CoreEvents to master for I08f66861b (2/2)' | 
  [production] | 
            
  | 21:45 | 
  <olivneh> | 
  synchronized php-1.22wmf12/extensions/CoreEvents  'Updating CoreEvents to master for I08f66861b (1/2)' | 
  [production] | 
            
  | 21:05 | 
  <reedy> | 
  synchronized docroot and w | 
  [production] | 
            
  | 20:45 | 
  <reedy> | 
  synchronized docroot and w | 
  [production] | 
            
  | 19:36 | 
  <cmjohnson1> | 
  shutting down brewster yet again to replace the disk  | 
  [production] | 
            
  | 17:30 | 
  <demon> | 
  synchronized php-1.22wmf13/extensions/CirrusSearch/CirrusSearch.body.php | 
  [production] | 
            
  | 16:51 | 
  <cmjohnson1> | 
  brewster going down again  | 
  [production] | 
            
  | 16:21 | 
  <cmjohnson1> | 
  shutting down brewster to replace bad disk | 
  [production] | 
            
  | 16:21 | 
  <bblack> | 
  merged vhtcpd ganglia stuff and the VCL for PURGE hash_ignore_busy into puppet | 
  [production] | 
            
  | 15:09 | 
  <reedy> | 
  synchronized wmf-config/  'touch' | 
  [production] | 
            
  | 15:05 | 
  <reedy> | 
  synchronized database lists files: Ad tyvwiki to visualeditor.dblist | 
  [production] | 
            
  | 09:36 | 
  <paravoid> | 
  rolling reboot of ceph ms-be nodes | 
  [production] | 
            
  | 08:09 | 
  <apergos> | 
  sq41 hardware issues, see rt 5615 | 
  [production] |