| 2010-09-29
      
      § | 
    
  | 22:54 | <tfinc> | synchronized php-1.5/wmf-config/CommonSettings.php  'Turning test wikipedia to non infrastructure mode' | [production] | 
            
  | 22:41 | <tfinc> | synchronized php-1.5/wmf-config/InitialiseSettings.php  'disabling cn' | [production] | 
            
  | 22:39 | <tfinc> | synchronized php-1.5/extensions/CentralNotice/CentralNotice.php  'Picking up fixes from 73993' | [production] | 
            
  | 22:38 | <tfinc> | synchronized php-1.5/extensions/CentralNotice/SpecialBannerController.php  'Picking up fixes from 73993' | [production] | 
            
  | 22:36 | <tfinc> | synchronized php-1.5/includes/Skin.php  'Picking up fixes from 73993' | [production] | 
            
  | 21:31 | <tfinc> | synchronized php-1.5/extensions/CentralNotice/SpecialBannerController.php  'Picking up fix for message translation' | [production] | 
            
  | 21:29 | <tfinc> | synchronized php-1.5/includes/MessageCache.php  'Picking up fix for message translation' | [production] | 
            
  | 21:28 | <tfinc> | synchronized php-1.5/includes/parser/CoreParserFunctions.php  'Picking up fix for message translation' | [production] | 
            
  | 21:27 | <tfinc> | synchronized php-1.5/includes/parser/ParserOptions.php  'Picking up fix for message translation' | [production] | 
            
  | 19:21 | <mark> | Permanently doubled sys.net.ipv4.tcp_max_tw_buckets to 360000, as the Varnish systems are constantly close to the limit (visible in /proc/net/sockstat) | [production] | 
            
  | 19:04 | <mark> | ...on all squid and varnish servers, in puppet sysctl.conf | [production] | 
            
  | 19:03 | <mark> | Increasing net.ipv4.tcp_max_syn_backlog = 262144 | [production] | 
            
  | 18:57 | <mark> | Experimentally doubled sys.net.ipv4.tcp_max_tw_buckets to 360000 on sq70 (manually, so not in sysctl.conf) | [production] | 
            
  | 16:45 | <Ryan_Lane> | make that the *new* tesla cluster subnet | [production] | 
            
  | 16:44 | <Ryan_Lane> | updating dhcpd.conf on brewster to include the tesla subnet (153.192) | [production] | 
            
  | 16:29 | <mark> | Setup IP helper-address to brewster for vlan 103 / ve 7 on csw5-pmtpa | [production] | 
            
  | 02:31 | <Ryan_Lane> | unmounted old data partition and replaced it with the newer, larger data partition and updated the fstab on tridge. | [production] | 
            
  | 00:51 | <Ryan_Lane> | destroyed resized data1 logical volume, and recreated the jfs filesystem on tridge. copying the data from /data to new partition; will remount data as new partition and update fstab | [production] | 
            
  | 00:49 | <awjr> | Upgraded kernel on CC server to 2.6.32-24 | [production] | 
            
  | 00:49 | <tfinc> | synchronized php-1.5/wmf-config/CommonSettings.php  'Re-enabling CC server on variable page post kernel upgrade' | [production] | 
            
  | 00:49 | <Ryan_Lane> | install kvm-pxe on alsted | [production] | 
            
  | 00:49 | <Ryan_Lane> | configured eth1 with vlan tag 103, and added it to a new bridge, br103 on alsted and ersch | [production] | 
            
  | 00:36 | <tfinc> | synchronized php-1.5/wmf-config/CommonSettings.php  'Taking outage on CC server for kernel upgrade' | [production] | 
            
  
    | 2010-09-28
      
      § | 
    
  | 21:57 | <Ryan_Lane> | changing testvm1 to tesla namespace. adding new origin for tesla | [production] | 
            
  | 21:52 | <Ryan_Lane> | adding testvm1.tesla to dns | [production] | 
            
  | 19:58 | <tfinc> | synchronized php-1.5/wmf-config/CommonSettings.php  'Switching from landing page 9 to 1 for donate sidebar link' | [production] | 
            
  | 02:00 | <Ryan_Lane> | added puppet vim syntax highlighting to root's .vim directory on sockpuppet | [production] | 
            
  | 01:17 | <tfinc> | synchronized php-1.5/wmf-config/InitialiseSettings.php  'Enaling centralnotice for all projects' | [production] | 
            
  | 01:12 | <tfinc> | synchronized php-1.5/wmf-config/InitialiseSettings.php  'Enaling dewiki for centralnotice' | [production] | 
            
  | 01:05 | <tfinc> | synchronized php-1.5/wmf-config/InitialiseSettings.php  'Re-enaling cswiki for centralnotice' | [production] | 
            
  | 01:03 | <tfinc> | synchronized php-1.5/extensions/CentralNotice/CentralNotice.php  'Picking up autoload fix' | [production] | 
            
  | 00:57 | <tfinc> | synchronized php-1.5/extensions/CentralNotice/CentralNotice.php  'Picking up db class fix' | [production] | 
            
  | 00:51 | <tfinc> | synchronized php-1.5/extensions/CentralNotice/SpecialBannerListLoader.php  'Picking up db class fix' | [production] | 
            
  | 00:51 | <tfinc> | synchronized php-1.5/extensions/CentralNotice/CentralNotice.php  'Picking up db class fix' | [production] | 
            
  | 00:44 | <tfinc> | synchronized php-1.5/wmf-config/CommonSettings.php  'picking up new cc form variants for thursday fundraiser' | [production] | 
            
  | 00:35 | <tfinc> | synchronized php-1.5/extensions/CentralNotice/SpecialBannerListLoader.php  'Turning on debugging for CN' | [production] | 
            
  | 00:27 | <tfinc> | synchronized php-1.5/wmf-config/InitialiseSettings.php  'Turning cs wiki centralnotice off' | [production] | 
            
  | 00:10 | <tfinc> | synchronized php-1.5/wmf-config/InitialiseSettings.php  'Turning central notice back on for en wikiversity' | [production] | 
            
  
    | 2010-09-27
      
      § | 
    
  | 23:38 | <tfinc> | synchronized php-1.5/wmf-config/InitialiseSettings.php  'Turning central notice back on for cs wiki' | [production] | 
            
  | 23:03 | <tfinc> | synchronizing Wikimedia installation... Revision: 73744 | [production] | 
            
  | 20:23 | <Ryan_Lane> | updated puppet configuration to require php version 5.2.4-2ubuntu5.12wm1 for all apaches | [production] | 
            
  | 20:21 | <Ryan_Lane> | testing updated puppet config for php on srv212 | [production] | 
            
  | 20:18 | <mark> | Changed MIME type for geoiplookup.wikimedia.org in wikimedia.vcl (using puppet) | [production] | 
            
  | 20:17 | <Ryan_Lane> | updated puppet configuration to update php on all apaches | [production] | 
            
  | 20:12 | <mark> | Powercycled sq67 | [production] | 
            
  | 20:06 | <Ryan_Lane> | testing updated php packages deployed from puppet on srv 211 | [production] | 
            
  | 20:05 | <Ryan_Lane> | updating wikis on prototype | [production] | 
            
  | 19:36 | <Ryan_Lane> | reinstalled wikimedia-task-appserver on srv210 | [production] | 
            
  | 19:27 | <Ryan_Lane> | testing updated php from repository on srv210 | [production] | 
            
  | 18:45 | <Ryan_Lane> | pushing updated php to the repository | [production] |