| 2011-11-10
      
      ยง | 
    
  | 23:29 | <Tim> | on fenari: ran puppetd -tv --debug to fix wikimedia-task-appserver | [production] | 
            
  | 23:11 | <tstarling> | synchronizing Wikimedia installation... : | [production] | 
            
  | 23:10 | <neilk> | synchronized php-1.18/extensions/UploadWizard | [production] | 
            
  | 22:54 | <neilk> | synchronizing Wikimedia installation... : New version of UploadWizard ([[rev:102709|r102709]]), multi-file select, custom licenses (2nd try) | [production] | 
            
  | 22:53 | <neilk> | synchronizing Wikimedia installation... : New version of UploadWizard ([[rev:102709|r102709]]), multi-file select, custom licenses | [production] | 
            
  | 22:22 | <reedy> | synchronized wmf-config/InitialiseSettings.php  'Fix kk logos' | [production] | 
            
  | 22:20 | <awjrichards> | synchronized wmf-config/CommonSettings.php  'Adding better config control for ContributionReporting extension' | [production] | 
            
  | 22:20 | <awjrichards> | synchronized wmf-config/InitialiseSettings.php  'Adding better config control for ContributionReporting extension' | [production] | 
            
  | 22:05 | <Reedy> | Run namespaceDupes on aswiki | [production] | 
            
  | 22:03 | <reedy> | synchronized wmf-config/InitialiseSettings.php  'Bug 32145 - Namespaces/Aliases for Wikipedia, Project and Portal namespace in Assamese wikipedia' | [production] | 
            
  | 21:52 | <reedy> | synchronized wmf-config/InitialiseSettings.php  'Bug 32340 - Translation of project namespace and sitename for tawikibooks' | [production] | 
            
  | 21:42 | <reedy> | synchronized wmf-config/InitialiseSettings.php  'Bug 32176 - Create an author namespace on eo.wikisource' | [production] | 
            
  | 21:39 | <Reedy> | Running namespaceDupes on eowikisource | [production] | 
            
  | 21:38 | <reedy> | synchronized wmf-config/InitialiseSettings.php  'Bug 32176 - Create an author namespace on eo.wikisource' | [production] | 
            
  | 21:21 | <reedy> | synchronized wmf-config/InitialiseSettings.php  'Bug 32048 - Configuration changes for hewiki, add editinterface locally' | [production] | 
            
  | 21:17 | <reedy> | synchronized wmf-config/InitialiseSettings.php  'Bug 32048 - Configuration changes for hewiki, add editinterface locally' | [production] | 
            
  | 21:08 | <reedy> | synchronized wmf-config/InitialiseSettings.php  'Bug 32191 - Update logo for kk.wiktionary.org' | [production] | 
            
  | 20:40 | <Reedy> | Moodbar tables should now be normalised across WMF wikis. Tables had been created everywhere, but updates not applied afterwards | [production] | 
            
  | 20:31 | <Reedy> | running foreachwiki sql.php php-1.18/extensions/MoodBar/sql/mbf_timestamp_id.sql | [production] | 
            
  | 20:27 | <Reedy> | running foreachwiki sql.php php-1.18/extensions/MoodBar/sql/AddIDToIndexes.sql | [production] | 
            
  | 20:16 | <Reedy> | running foreachwiki sql.php php-1.18/extensions/MoodBar/sql/mbf_hidden_state.sql | [production] | 
            
  | 20:08 | <Reedy> | Run other Moodbar related sql patches on nlwiki | [production] | 
            
  | 20:02 | <Reedy> | Running mbf_hidden_state.sql on nlwiki | [production] | 
            
  | 19:57 | <reedy> | synchronized wmf-config/InitialiseSettings.php  'Bug 32202 - Moodbar on nl: wikipedia' | [production] | 
            
  | 19:43 | <notpeter> | restareting nagios | [production] | 
            
  | 19:24 | <ben> | synchronized wmf-config/db.php  'adding es1 and es2 into rotation for all external store clusters' | [production] | 
            
  | 19:07 | <mark> | Repooled knsq24 in pybal, no idea why it was depooled | [production] | 
            
  | 17:26 | <mark> | Increased ms5 nginx worker processes from 4 to 32 (out of puppet) | [production] | 
            
  | 17:25 | <mark> | Increased ms6 worker processes from 4 to 32 (out of puppet) | [production] | 
            
  | 17:06 | <mark> | Moving esams traffic back to esams to seed the upload squid SSD caches | [production] | 
            
  | 14:51 | <hashar> | synchronized wmf-config/InitialiseSettings.php  'resyncing for [[bugzilla:32309|bug 32309]]. Some servers did timeout' | [production] | 
            
  | 14:48 | <hashar> | synchronized wmf-config/InitialiseSettings.php  '[[bugzilla:32309|bug 32309]] - Add WMF domains to $wgNoFollowDomainException' | [production] | 
            
  | 12:20 | <mark> | Switching esams traffic to pmtpa | [production] | 
            
  | 11:09 | <domas> | otrs still broken, williams had old passwords, something else is going on with mail delivery | [production] | 
            
  | 10:46 | <domas> | One needs to run 'FLUSH PRIVILEGES' after modifying mysql.* tables directly. OTRS delivery was broken for a day or so? | [production] | 
            
  | 10:22 | <guillom> | OTRS issue: e-mail isn't getting in. No new messages have come through in the last 15h and 40 min. Outgoing e-mail works fine. | [production] | 
            
  | 02:29 | <LocalisationUpdate> | completed (1.18) at Thu Nov 10 02:32:45 UTC 2011 | [production] | 
            
  | 02:12 | <awjr> | Updating thank you module code for CiviCRM on aluminium/grosley; [[rev:762|r762]] | [production] | 
            
  | 01:57 | <LocalisationUpdate> | failed | [production] | 
            
  | 01:34 | <awjr> | Updating civicrm on aluminium/grosley to [[rev:760|r760]] | [production] | 
            
  | 01:17 | <RobH> | ocg2 installed per rt1593 | [production] | 
            
  | 01:14 | <Tim> | started puppet on fenari | [production] | 
            
  | 01:12 | <RobH> | ocg1 installed, awaiting puppetization (pending labs-ocg1 deployment) | [production] | 
            
  | 00:43 | <RobH> | mobile3 (now ocg1) coming offline for reinstallation, ignore any mobile3 nagios errors | [production] | 
            
  | 00:36 | <RobH> | another dns update, this is me pushing my luck on dns not crashing | [production] | 
            
  | 00:29 | <awjrichards> | synchronized php/extensions/DonationInterface/gateway_common/us-states.i18n.php  '[[rev:102610|r102610]]' | [production] | 
            
  | 00:29 | <awjrichards> | synchronized php/extensions/DonationInterface/gateway_common/interface.i18n.php  '[[rev:102610|r102610]]' | [production] | 
            
  | 00:28 | <awjrichards> | synchronized php/extensions/DonationInterface/gateway_common/countries.i18n.php  '[[rev:102610|r102610]]' | [production] | 
            
  | 00:26 | <awjrichards> | synchronized php/extensions/DonationInterface/globalcollect_gateway/globalcollect_gateway.i18n.php  '[[rev:102610|r102610]]' | [production] | 
            
  | 00:25 | <RobH> | ocg dns update, once more with feeling | [production] |