| 
      
        2011-05-26
      
      §
     | 
  
    
  | 18:58 | 
  <catrope> | 
  synchronized php-1.17/wmf-config/CommonSettings.php  'Raise AFT tracking percentage from 2% to 10% (restores pre-ramp-up volume) and put all users in the show-expertise bucket. Also bump tracking version to 7' | 
  [production] | 
            
  | 18:54 | 
  <Ryan_Lane> | 
  adding wikimedialbsecure service to amslvs1, for testing https | 
  [production] | 
            
  | 18:52 | 
  <catrope> | 
  synchronized php-1.17/extensions/ArticleFeedback/modules/jquery.articleFeedback/jquery.articleFeedback.js  'r88909' | 
  [production] | 
            
  | 18:47 | 
  <Ryan_Lane> | 
  adding wikimedia-lb.esams address to ams and kn text squids, also adding lvs configuration for this address | 
  [production] | 
            
  | 18:31 | 
  <Ryan_Lane> | 
  removing mediawiki-lb.esams and foundation-lb.esams reverse addresses, they were assigned to the same ips as text.esams and bits.esams respectively | 
  [production] | 
            
  | 17:48 | 
  <Ryan_Lane> | 
  adding forward entry for wikimedia-lb.pmtpa.wikimedia.org | 
  [production] | 
            
  | 15:34 | 
  <mark> | 
  Moved AS13030 connection configuration from port 2/11 to port 2/11 on br1-knams | 
  [production] | 
            
  
    | 
      
        2011-05-25
      
      §
     | 
  
    
  | 22:39 | 
  <awjr> | 
  svn up'd on civicrm.wikimedia.org to r189 - deploying further optimizations to contribution searching | 
  [production] | 
            
  | 21:22 | 
  <Ryan_Lane> | 
  adding wikimedia-lb svc address to squids | 
  [production] | 
            
  | 21:22 | 
  <Ryan_Lane> | 
  adding wikimedialbsecure service to lvs2, for testing https access to wikimedia.org sites. | 
  [production] | 
            
  | 21:21 | 
  <Ryan_Lane> | 
  modified pybal configuration to add wikimedia-lb ip to text backend | 
  [production] | 
            
  | 19:52 | 
  <demon> | 
  synchronized php-1.17/extensions/CentralAuth/SpecialWikiSets.php  'r88825' | 
  [production] | 
            
  | 18:32 | 
  <Ryan_Lane> | 
  turning off the selenium grid virtual machines on tesla | 
  [production] | 
            
  | 17:04 | 
  <RobH> | 
  updated bugzilla per rt 764 and 768.  whine feature should work now | 
  [production] | 
            
  | 16:44 | 
  <awjr> | 
  deployed patch to civicrm.wikimedia.org to optimize contribution search query (r185, wikimedia repo) | 
  [production] | 
            
  | 14:57 | 
  <demon> | 
  synchronized php-1.17/wmf-config/InitialiseSettings.php  'Don't allow users to create accounts on closed wikis either' | 
  [production] | 
            
  | 14:54 | 
  <demon> | 
  synchronized php-1.17/wmf-config/InitialiseSettings.php  'Cleanup wikimania200[58] entries: bug 29064' | 
  [production] | 
            
  | 14:53 | 
  <demon> | 
  synchronized closed.dblist  'Add wikimania200[58] to closed.dblist: bug 29064' | 
  [production] | 
            
  | 14:45 | 
  <demon> | 
  synchronized php-1.17/wmf-config/InitialiseSettings.php  'bug 29064: close wikimania2009wiki' | 
  [production] | 
            
  | 14:43 | 
  <demon> | 
  synchronized closed.dblist  'bug 29064: close wikimania2009wiki' | 
  [production] | 
            
  | 12:27 | 
  <Tim> | 
  marked the deletion jobs in OTRS as "invalid" to avoid confusing error messages | 
  [production] | 
            
  | 12:25 | 
  <Tim> | 
  re-enabled anti-deletion security patch in OTRS, disabled "temporarily" by Fred on June 4, 2009 | 
  [production] | 
            
  | 11:58 | 
  <domas> | 
  rebooting db9, looks unhealthy | 
  [production] | 
            
  | 11:06 | 
  <apergos> | 
  so that was me on srv225: stopped apache, waited for puppet restart: it now syncs, then does the restart. oh btw srv225 isn't in the pybal apache conf file, weirdly... anyone know why? | 
  [production] | 
            
  | 09:15 | 
  <apergos> | 
  restarted front and backend squids on sq75 | 
  [production] | 
            
  | 08:47 | 
  <apergos> | 
  restarted front and back end squids on sq81, front end was 20gb and back end was out to lunch | 
  [production] | 
            
  | 08:35 | 
  <apergos> | 
  ekrem's root partition was full; cleared out some older logs from /var/log/apache2, compressed the most recently rotated (it had failed from disk full), reloaded puppet | 
  [production] | 
            
  | 07:20 | 
  <Tim> | 
  set read_only=1 on db10, it's a slave so it should be read only. Nagios says it was broken for 6 months. | 
  [production] | 
            
  | 07:07 | 
  <midom> | 
  synchronized php-1.17/wmf-config/db.php  | 
  [production] | 
            
  | 05:09 | 
  <Tim> | 
  restarted nagios IRC bot | 
  [production] | 
            
  | 01:05 | 
  <Tim> | 
  created an account for myself in watchmouse | 
  [production] | 
            
  | 00:48 | 
  <Andrew> | 
  Running populateBv2011EditCount.php in a screen on hume | 
  [production] | 
            
  | 00:32 | 
  <Ryan_Lane> | 
  pruning binlogs on db9 | 
  [production] | 
            
  
    | 
      
        2011-05-24
      
      §
     | 
  
    
  | 23:12 | 
  <pdhanda> | 
  synchronized php-1.17/wmf-config/InitialiseSettings.php  'Bug 29129. Added a namespace alias for metawiki.' | 
  [production] | 
            
  | 20:36 | 
  <notpeter_> | 
  resyncing drbd nfs1 to nfs2 | 
  [production] | 
            
  | 20:28 | 
  <RobH> | 
  mobile4 awaiting final configurations for service, mobile5 installation done, not yet run puppet updates | 
  [production] | 
            
  | 20:20 | 
  <RobH> | 
  mobile4 is running puppet updates, mobile5 is mid-install | 
  [production] | 
            
  | 19:46 | 
  <RobH> | 
  mobile4 & mobile5 racked and accessible for mgmt.  installing OS on mobile4 now | 
  [production] | 
            
  | 18:00 | 
  <Ryan_Lane> | 
  added new reverse zone for svc ips in esams; added entries for bits.svc.esams.wmnet and upload.svc.esams.wmnet | 
  [production] | 
            
  | 17:31 | 
  <Ryan_Lane> | 
  removing bits-lb and upload-lb public ip enties, and adding bits.svc.pmtpa.wmnet and upload.svc.pmtpa.wmnet entries for https testing | 
  [production] | 
            
  | 16:46 | 
  <mark> | 
  Upgraded cr2-eqiad to JUNOS 10.4R4.5 | 
  [production] | 
            
  | 16:44 | 
  <mark> | 
  PIM configuration was missing on some virtual interfaces on csw1-sdtpa, which broke purging to esams. Put back in place | 
  [production] | 
            
  | 16:09 | 
  <mark> | 
  Reloading re1.cr2-eqiad | 
  [production] | 
            
  | 16:01 | 
  <mark> | 
  Fixed tunnel hack between pmtpa and eqiad | 
  [production] | 
            
  | 15:01 | 
  <RobH> | 
  db20 networking restarted, back online | 
  [production] | 
            
  | 12:05 | 
  <mark> | 
  Upgraded bootrom and software on asw-d1-sdtpa and asw-a5-sdtpa | 
  [production] | 
            
  | 10:53 | 
  <mark> | 
  Copied existing 2.7.0.2 image to secondary flash, and installed 2.7.0.3a image on primary flash on csw5-pmtpa | 
  [production] | 
            
  | 10:42 | 
  <mark> | 
  Copied existing 2.6.0 image to secondary flash, and installed 2.7.0.3a image on primary flash on csw1-sdtpa | 
  [production] | 
            
  | 04:20 | 
  <demon> | 
  synchronized php-1.17/extensions/GoogleNewsSitemap/GoogleNewsSitemap_body.php  'r88707' | 
  [production] | 
            
  | 04:03 | 
  <tstarling> | 
  synchronized php-1.17/wmf-config/InitialiseSettings.php  | 
  [production] |