| 2011-12-13
      
      ยง | 
    
  | 23:40 | <binasher> | restarted pdns on dobson | [production] | 
            
  | 20:59 | <nikerabbit> | synchronized php-1.18/extensions/WebFonts/resources/ext.webfonts.js  'Updating font urls' | [production] | 
            
  | 20:55 | <nikerabbit> | synchronized php-1.18/extensions/WebFonts/fonts/  'touching all font files' | [production] | 
            
  | 20:48 | <catrope> | synchronized php-1.18/extensions/WebFonts/fonts/Orya/utkal.ttf  'touch' | [production] | 
            
  | 20:45 | <catrope> | synchronized php-1.18/extensions/WebFonts/fonts/Orya/utkal.ttf  'touch' | [production] | 
            
  | 20:34 | <RoanKattouw> | Actually running Puppet on the bits Apaches now | [production] | 
            
  | 20:28 | <RoanKattouw> | Manually running Puppet on all bits Apaches using dsh | [production] | 
            
  | 19:58 | <RoanKattouw> | Removed spence from the mediawiki-installation node list | [production] | 
            
  | 19:45 | <neilk> | synchronizing Wikimedia installation... : deploying VisualEditor, also deploying some new JS libs and associated tests and messages to core which were taken from UploadWizard | [production] | 
            
  | 19:43 | <neilk> | synchronized wmf-config/extension-list  'added VisualEditor' | [production] | 
            
  | 19:42 | <neilk> | synchronized wmf-config/ExtensionMessages-1.18.php  'added VisualEditor' | [production] | 
            
  | 18:51 | <Reedy> | fixExtLinksProtocolRelative is upto enwiki | [production] | 
            
  | 18:18 | <notpeter> | stopping puppet on brewster again. again, please let me know if you want to restart it... | [production] | 
            
  | 17:15 | <apergos> | thumb cleaner went to bed late... maybe I should set an alarm to remind me :-/ | [production] | 
            
  | 16:58 | <Reedy> | Running foreachwiki fixExtLinksProtocolRelative in screen on fenari as reedy | [production] | 
            
  | 16:56 | <Reedy> | fixExtLinksProtocolRelative run on meta | [production] | 
            
  | 16:49 | <Reedy> | fixExtLinksProtocolRelative run on mw.org and simplewiki | [production] | 
            
  | 15:07 | <mutante> | spence - copied sudoers file from fenari to allow "admins to run commands as apache, mwdeploy and l10nupdate" and wikidev = admins | [production] | 
            
  | 14:47 | <reedy> | synchronized wmf-config/CommonSettings.php  'wgNarayamUseBetaMapping' | [production] | 
            
  | 14:46 | <reedy> | synchronized wmf-config/InitialiseSettings.php  'Add wmgNarayamUseBetaMapping' | [production] | 
            
  | 12:38 | <mutante> | rolling kernel and security upgrades on amssq31-35 | [production] | 
            
  | 11:52 | <mutante> | spence still slow, restarted nagios (as opposed to just reloading) | [production] | 
            
  | 11:03 | <mutante> | spence changed check_interval for check_job_queue from 1 to 15 (min) as it takes ~5 minutes to complete currently - reloaded nagios | [production] | 
            
  | 10:52 | <mutante> | check_job_queue Nagios check finally works again :) - but why do we check for "3rd biggest" (fix me) | [production] | 
            
  | 10:43 | <RoanKattouw> | Added spence to fenari:/etc/dsh/group/mediawiki-installation | [production] | 
            
  | 10:41 | <mutante> | spence - "chown -R mwdeploy:mwdeploy /usr/local/apache/common-local" per Roan, sync-common again, finished fine..after a while. (spence had really outdated copy of MW source tree) | [production] | 
            
  | 10:31 | <mutante> | ran sync-common from spence and got a ton of permission errors | [production] | 
            
  | 10:29 | <mutante> | spence added root to sudoers to fix sync-common | [production] | 
            
  | 10:25 | <mutante> | spence created mwdeploy user to fix sync-common | [production] | 
            
  | 10:17 | <mutante> | spence - fix comment style in apc.ini (# -> //) to get rid of 'PHP Deprecated' messages - install php-apc for check_job_queue | [production] | 
            
  | 10:11 | <mutante> | copied script files provided by wikimedia-task-appserver from fenari to spence | [production] | 
            
  | 06:59 | <apergos> | thumb cleaner back at work for the day on ms5 | [production] | 
            
  | 05:38 | <nikerabbit> | synchronized php-1.18/extensions/WebFonts/resources/ext.webfonts.js  'bugfix for webfonts: [[rev:105989|r105989]]' | [production] | 
            
  | 04:34 | <RobH> | blog.wikimedia.org rolled to wp3.3 successfully | [production] | 
            
  | 02:38 | <notpeter> | belay that. running just fine. just hella slow. | [production] | 
            
  | 02:36 | <notpeter> | puppet currently not completing runs on spence. not going to try to divine root cause this late at night | [production] | 
            
  | 02:28 | <notpeter> | restarting puppet on brewster | [production] | 
            
  | 01:29 | <notpeter> | removing a couple of binlogs from db9 to try to get it through until it's replaced.... | [production] | 
            
  | 01:13 | <tstarling> | synchronized php-1.18/includes/MagicWord.php | [production] | 
            
  | 01:04 | <Ryan_Lane> | switching ssl certificate to digicert certificate for wikipedia | [production] | 
            
  | 00:46 | <Ryan_Lane> | enabling https for *.m.wikipedia.org | [production] | 
            
  | 00:46 | <Ryan_Lane> | repooling ssl1001 | [production] | 
            
  | 00:28 | <Ryan_Lane> | restarted pybal on lvs1004 for mobile ssl and mobile svc services | [production] | 
            
  | 00:27 | <Ryan_Lane> | make that lvs1001 | [production] | 
            
  | 00:27 | <Ryan_Lane> | restarted pybal on ssl1001 for mobile ssl and mobile svc services | [production] | 
            
  | 00:15 | <Ryan_Lane> | depooling ssl1001 for testing | [production] | 
            
  | 00:15 | <LeslieCarr> | depooling and restarting ssl1001 withnew cert | [production] | 
            
  | 00:14 | <LeslieCarr> | pushing out new digicert certificate for *.wikipedia.org | [production] | 
            
  | 00:12 | <reedy> | synchronized php-1.18/extensions/CodeReview/  'Reverting CR back to previous state' | [production] | 
            
  | 00:02 | <reedy> | synchronized php-1.18/extensions/CodeReview/backend/DiffHighlighter.php | [production] |