| 2013-07-23
      
      § | 
    
  | 23:40 | <MaxSem> | Rebuilding Solr index to catch up new schema | [production] | 
            
  | 23:34 | <andrewbogott> | done messing with sockpuppet, stafford, puppet-merge, etc for now. | [production] | 
            
  | 21:27 | <reedy> | synchronized wmf-config/CommonSettings.php | [production] | 
            
  | 20:55 | <mwalker> | updating civicrm on aluminium from e4062d012d0ccfb6220fc3703fd42ad894151ec5 to 180ecd219da68a0b950dafb036078b7c285e2437 | [production] | 
            
  | 20:49 | <mwalker> | kludging payments data -- adding 7 hours to contribution timestamps between IDs 4380557 and 4605567 | [production] | 
            
  | 20:35 | <mwalker> | paused all audit and QC jenkins jobs on the payments cluster in preparation for fixing timestamps | [production] | 
            
  | 19:54 | <reedy> | synchronized php-1.22wmf11/extensions/Wikibase | [production] | 
            
  | 19:46 | <reedy> | synchronized php-1.22wmf11/extensions/Wikibase | [production] | 
            
  | 19:29 | <Jeff_Green> | dist-upgrade & reboot for payments* | [production] | 
            
  | 18:39 | <reedy> | synchronized php-1.22wmf10/extensions/Wikibase | [production] | 
            
  | 18:08 | <reedy> | synchronized wmf-config/InitialiseSettings.php | [production] | 
            
  | 18:06 | <reedy> | synchronized wmf-config/ | [production] | 
            
  | 18:02 | <reedy> | synchronized database lists files: | [production] | 
            
  | 18:00 | <reedy> | synchronized wmf-config/ | [production] | 
            
  | 17:58 | <Reedy> | Populated sites table on all wikivoyage wikis | [production] | 
            
  | 17:40 | <reedy> | synchronized wmf-config/ | [production] | 
            
  | 17:31 | <reedy> | synchronized wmf-config/ | [production] | 
            
  | 17:27 | <reedy> | synchronized php-1.22wmf11/extensions/Wikibase | [production] | 
            
  | 17:25 | <reedy> | synchronized php-1.22wmf10/extensions/Wikibase | [production] | 
            
  | 17:13 | <aaron> | synchronized wmf-config/InitialiseSettings.php  '26162fbc97ebbe25fee84bb333b0b0b2efe4c447' | [production] | 
            
  | 16:40 | <Jeff_Green> | authdns-update to propagate fundraisingdb-* cname changes | [production] | 
            
  | 15:57 | <Jeff_Green> | stop icinga notifications for db1025 mysql | [production] | 
            
  | 15:57 | <Jeff_Green> | stop icinga notifications for db1008 | [production] | 
            
  | 15:21 | <Jeff_Green> | dist-upgrade &reboot db1025 | [production] | 
            
  | 15:14 | <reedy> | synchronized wmf-config/InitialiseSettings.php | [production] | 
            
  | 15:13 | <reedy> | synchronized database lists files: | [production] | 
            
  | 15:13 | <Jeff_Green> | dist-upgrade & reboot payments2-4 | [production] | 
            
  | 14:57 | <Jeff_Green> | dist-upgrade & reboot payments2 | [production] | 
            
  | 14:47 | <ottomata> | stopping puppet on gadolinium, looking into filters causing socat process packet loss | [production] | 
            
  | 14:22 | <hashar> | Jenkins: lanthanum has been added as a Jenkins slave node though NO job should be running there.e | [production] | 
            
  | 10:46 | <mark> | synchronized wmf-config/squid.php  'Add ssl1005/ssl1006' | [production] | 
            
  | 05:44 | <paravoid> | pooling ssl1005/1006 for https/ipv6 | [production] | 
            
  | 05:34 | <paravoid> | authdns-update for cr1-ulsfo <-> cr2-eqiad, dns-auth-lb -> nsN(-new), ssl100x fixed IPv6 | [production] | 
            
  | 02:21 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Tue Jul 23 02:21:20 UTC 2013 | [production] | 
            
  | 02:12 | <LocalisationUpdate> | completed (1.22wmf10) at Tue Jul 23 02:12:36 UTC 2013 | [production] | 
            
  | 02:08 | <LocalisationUpdate> | completed (1.22wmf11) at Tue Jul 23 02:08:03 UTC 2013 | [production] | 
            
  | 01:01 | <Ryan_Lane> | enabled hyperthreading on ssl1005 | [production] | 
            
  | 00:39 | <Ryan_Lane> | depooling ssl1005/6 due to ipv6 issues | [production] | 
            
  | 00:39 | <Ryan_Lane> | pooled new https node ssl1006.wikimedia.org (with hyperthreading enabled) | [production] | 
            
  | 00:38 | <Ryan_Lane> | pooled new https node ssl1005.wikimedia.org | [production] |