| 2012-11-09
      
      § | 
    
  | 20:26 | <sbernardin> | installing ssd's in labsdb1 and labsdb2 | [production] | 
            
  | 19:53 | <mutante> | git reset in /h/w/common/ to revert local disabling of "voy" wikis and synced dblist again | [production] | 
            
  | 19:40 | <mutante> | doing a git pull in wmf-config | [production] | 
            
  | 19:17 | <mutante> | running sync-dblist | [production] | 
            
  | 19:15 | <notpeter> | rebooting mw60 for upgarde to precise | [production] | 
            
  | 19:05 | <mutante> | enabling wikivoyage subdomains in apache for QA check | [production] | 
            
  | 19:05 | <logmsgbot_> | dzahn gracefulled all apaches | [production] | 
            
  | 18:54 | <cmjohnson1> | authdns update "Adding .pmtpa.wmnet entries for labsdb1-3" | [production] | 
            
  | 18:45 | <mutante> | wikivoyage imports: "de" (and all for now) done | [production] | 
            
  | 18:38 | <mutante> | wikivoyage imports: "en" done, "it" done | [production] | 
            
  | 16:54 | <sbernardin> | shutting down ms-be7 to row C sdtpa | [production] | 
            
  | 16:53 | <sbernardin> | racking new ms-be7 to row C sdtpa | [production] | 
            
  | 16:51 | <cmjohnson1> | ms-be7 going offline to make room for new 720 servers | [production] | 
            
  | 14:36 | <cmjohnson1> | authodns committing mgmt ip for labsdb3 | [production] | 
            
  | 12:03 | <apergos> | ms-be6 deployed and back in swift rings on shiny new 720xd | [production] | 
            
  | 05:00 | <logmsgbot_> | tstarling synchronized wmf-config/InitialiseSettings.php  'enabling patrol on wikidatawiki' | [production] | 
            
  | 02:47 | <logmsgbot_> | olivneh synchronized php-1.21wmf3/extensions/E3Experiments/Experiments.hooks.php  'Fixing OpenTask event log bug' | [production] | 
            
  | 02:23 | <logmsgbot_> | LocalisationUpdate completed (1.21wmf3) at Fri Nov  9 02:23:01 UTC 2012 | [production] | 
            
  | 01:58 | <binasher> | converted all s3 searchindex tables to innodb | [production] | 
            
  | 01:58 | <notpeter> | starting innobackupex from es3 to es4 | [production] | 
            
  | 01:49 | <logmsgbot_> | olivneh synchronized php-1.21wmf3/extensions/E3Experiments/experiments/openTask.js | [production] | 
            
  | 01:36 | <logmsgbot_> | py synchronized wmf-config/db.php  'pulling es4' | [production] | 
            
  | 01:18 | <mutante> | installing package upgrades on kaulen, half-configured apache2 brought down bugzilla ... | [production] | 
            
  | 00:56 | <Tim> | on fenari: killed stuck git processes run by ExtensionDistributor and LocalisationUpdate | [production] | 
            
  | 00:52 | <logmsgbot_> | LocalisationUpdate failed: git pull of extensions failed | [production] | 
            
  | 00:31 | <logmsgbot_> | pgehres synchronized wmf-config | [production] | 
            
  | 00:25 | <logmsgbot_> | reedy synchronized php-1.21wmf3/extensions/SiteMatrix/ | [production] | 
            
  
    | 2012-11-08
      
      § | 
    
  | 23:56 | <logmsgbot_> | pgehres synchronized wmf-config | [production] | 
            
  | 23:51 | <K4-713> | updated payments cluster to 7d92a1a35f0 | [production] | 
            
  | 23:51 | <logmsgbot_> | pgehres synchronized php-1.21wmf3/extensions/CentralNotice/  'Updating CentralNotice to master' | [production] | 
            
  | 23:29 | <logmsgbot_> | olivneh synchronized php-1.21wmf3/extensions/E3Experiments | [production] | 
            
  | 21:18 | <cmjohnson1> | ms-be6 initial puppet run | [production] | 
            
  | 21:14 | <logmsgbot_> | asher synchronized wmf-config/CommonSettings.php  'multiwrite pcache to new memcached servers, only use redis for sessions' | [production] | 
            
  | 21:08 | <logmsgbot_> | olivneh synchronized php-1.21wmf3/extensions/EventLogging | [production] | 
            
  | 21:01 | <logmsgbot_> | olivneh synchronized php-1.21wmf3/extensions/E3Experiments | [production] | 
            
  | 21:01 | <mutante> | restarted nginx on all SSL proxies without issues now. puppet runs again and does not recreate configs for data/voyage on esams. | [production] | 
            
  | 20:55 | <mutante> | removing wikidata/voyage sites config from esams ssl hosts | [production] | 
            
  | 20:54 | <cmjohnson1> | ms-be6 another OS install | [production] | 
            
  | 20:51 | <mutante> | fixed wikipedia ssl chained cert on ssl1 (all other hosts had it recreated as expected) | [production] | 
            
  | 20:50 | <apergos> | ignore the fail event on ms-be6, this was caused manually and is a bogus warning | [production] | 
            
  | 20:31 | <logmsgbot_> | demon synchronized wmf-config/CommonSettings.php  'Enabling lucene for wikidatawiki' | [production] | 
            
  | 19:29 | <logmsgbot_> | dzahn synchronized ./wmf-config/throttle.php | [production] | 
            
  | 18:16 | <logmsgbot_> | reedy synchronized php-1.21wmf3/extensions/TimedMediaHandler | [production] | 
            
  | 18:15 | <mutante> | removing wm-nl@wikimedia.org as list admin of "press-nl" because it is an OTRS queue (per thehelpfulone) | [production] | 
            
  | 15:55 | <paravoid> | starting ms-be3 {object,container,account}-server | [production] | 
            
  | 14:42 | <logmsgbot_> | LocalisationUpdate completed (1.21wmf3) at Thu Nov  8 14:42:45 UTC 2012 | [production] | 
            
  | 14:13 | <logmsgbot_> | LocalisationUpdate completed (1.21wmf3) at Thu Nov  8 14:13:54 UTC 2012 | [production] | 
            
  | 12:23 | <mark> | Reinstalling sq70 with Precise | [production] | 
            
  | 10:57 | <logmsgbot_> | reedy synchronized wmf-config/InitialiseSettings.php | [production] | 
            
  | 10:57 | <logmsgbot_> | reedy synchronized php-1.21wmf3/extensions/ProofreadPage/ | [production] |