| 2013-11-06
      
      § | 
    
  | 14:33 | <mark> | dist-upgrade && reboot on amssq47..62 | [production] | 
            
  | 12:18 | <paravoid> | started pybal on amslvs1 again; seems to work again, root cause unknown | [production] | 
            
  | 12:08 | <paravoid> | stopping pybal on amslvs1, wikidata-lb:80 issues, pending further investigation | [production] | 
            
  | 10:42 | <akosiaris1> | upgraded etherpad-lite to 1.3.0 on zirconium | [production] | 
            
  | 10:42 | <akosiaris1> | rebooted zirconium for kernel upgrade | [production] | 
            
  | 08:35 | <apergos> | rebooting neon, ext3 errors on root filesystem | [production] | 
            
  | 05:09 | <Tim> | on tin: running bug 53687 cleanup script (WikimediaMaintenance/bug-53687/fixOrphans.php) on 37 wikis | [production] | 
            
  | 03:24 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Wed Nov  6 03:23:59 UTC 2013 | [production] | 
            
  | 02:35 | <LocalisationUpdate> | completed (1.23wmf1) at Wed Nov  6 02:35:33 UTC 2013 | [production] | 
            
  | 02:21 | <LocalisationUpdate> | completed (1.23wmf2) at Wed Nov  6 02:21:52 UTC 2013 | [production] | 
            
  | 00:30 | <aaron> | synchronized php-1.23wmf2/includes/clientpool  'f1df92c9308c3350098dc9ca9e7ac1a221810cd7' | [production] | 
            
  | 00:29 | <aaron> | synchronized php-1.23wmf1/includes/clientpool  '313457c3ed8d58d7193d806e0155daea59adada4' | [production] | 
            
  
    | 2013-11-05
      
      § | 
    
  | 23:27 | <superm401> | Growth team deployment complete | [production] | 
            
  | 23:27 | <mflaschen> | synchronized php-1.23wmf2/extensions/GuidedTour/  'Deploy GuidedTour to 1.23wmf2' | [production] | 
            
  | 23:26 | <mflaschen> | synchronized php-1.23wmf2/extensions/GettingStarted/  'Deploy GettingStarted to 1.23wmf2' | [production] | 
            
  | 23:26 | <mflaschen> | synchronized php-1.23wmf1/extensions/GuidedTour/  'Deploy GuidedTour to 1.23wmf1' | [production] | 
            
  | 23:25 | <mflaschen> | synchronized php-1.23wmf1/extensions/GettingStarted/  'Deploy GettingStarted to 1.23wmf1' | [production] | 
            
  | 22:42 | <hashar> | sleeping. | [production] | 
            
  | 21:15 | <aaron> | synchronized php-1.23wmf1/includes/clientpool/RedisConnectionPool.php  '9e0a5d83fc5350218336a65130e0bb7fd1a5b4a1' | [production] | 
            
  | 21:09 | <hashar> | synchronized wmf-config/logging-labs.php | [production] | 
            
  | 21:08 | <otto> | synchronized wmf-config/InitialiseSettings.php  'Configuring CirrusSearch for 5 new wikis' | [production] | 
            
  | 21:08 | <hashar> | synchronized wmf-config/jobqueue-eqiad.php  'removing ParsoidCacheUpdateJob from default queue. Drained a few weeks ago (gwicke)' | [production] | 
            
  | 20:40 | <ottomata> | re-enabling puppet on analytics1021 | [production] | 
            
  | 20:39 | <RobH> | formey puppet agent disabled | [production] | 
            
  | 20:26 | <cmjohnson1> | dns update | [production] | 
            
  | 20:17 | <cmjohnson1> | shutting down search13-20, decommissoned | [production] | 
            
  | 20:16 | <mutante> | backing up /root on formey to tridge | [production] | 
            
  | 19:55 | <mutante> | DNS update - switching SVN over to antimony (eqiad) | [production] | 
            
  | 19:49 | <RobH> | marmontel decommissioned for shipment to eqiad | [production] | 
            
  | 19:48 | <RobH> | traces of neodyium, promethium, & argon removed and reclaimed to eqiad spare | [production] | 
            
  | 19:28 | <aaron> | synchronized php-1.23wmf2/includes/clientpool/RedisConnectionPool.php  '9e0a5d83fc5350218336a65130e0bb7fd1a5b4a1' | [production] | 
            
  | 19:09 | <ori> | synchronized php-1.23wmf1/extensions/Wikibase  'I9d52070a4: Re-introduce siteGroup setting for performance reasons' | [production] | 
            
  | 19:09 | <ori> | synchronized php-1.23wmf2/extensions/Wikibase  'I9d52070a4: Re-introduce siteGroup setting for performance reasons' | [production] | 
            
  | 19:07 | <ori> | synchronized wmf-config/InitialiseSettings.php  'I4d6862c3a: add wmgWikibaseSiteGroup' | [production] | 
            
  | 19:06 | <ori> | synchronized wmf-config/CommonSettings.php  'I4d6862c3a: add wmgWikibaseSiteGroup' | [production] | 
            
  | 17:56 | <mutante> | more cleanup on kaulen, move old stuff like "1.16wmf4 pilot" :) ,"codereview proxy", old bugzillas etc to backup | [production] | 
            
  | 17:50 | <mutante> | deleting empty /srv/org/mediawiki on kaulen | [production] | 
            
  | 17:47 | <RobH> | enabled spam filtering on ops list (logging in case it holds a ton of them) | [production] | 
            
  | 17:21 | <cmjohnson1> | updating pmtpa pybal searchpool_4 and 5...setting all to false | [production] | 
            
  | 17:21 | <mutante> | DNS update - killing codereview-proxy | [production] | 
            
  | 17:16 | <ori> | synchronized php-1.23wmf2/extensions/WikimediaEvents/modules/ext.wikimediaEvents.ve.js  'I4085ee9cb: Update VisualEditor performance event names' | [production] | 
            
  | 17:15 | <ori> | synchronized php-1.23wmf1/extensions/WikimediaEvents/modules/ext.wikimediaEvents.ve.js  'I4085ee9cb: Update VisualEditor performance event names' | [production] | 
            
  | 17:12 | <cmjohnson1> | disabling icinga checks on search13-20 for decommissioning rt6194 | [production] | 
            
  | 17:11 | <RobH> | reclaimed marmontel for shipment to eqiad, confirmed decommission steps up to powering down. | [production] | 
            
  | 17:10 | <ottomata> | stopping puppet on analytics1021 to test jmxtrans kafka stuff | [production] | 
            
  | 16:40 | <hashar> | beta: extensions update has been blocked since Oct 30th due to some bad registration in mediawiki/extensions.git . Fixed! | [production] | 
            
  | 16:39 | <cmjohnson1> | dns update | [production] | 
            
  | 15:48 | <paravoid> | authdns: upgrading gdnsd to 1.10.1 & switching to the proprietary GeoIP database | [production] | 
            
  | 15:22 | <reedy> | synchronized php-1.23wmf2/includes/media/FormatMetadata.php  'I5f65333cc94b66fd80fcf5abcfad7e10e4669310' | [production] | 
            
  | 13:29 | <mark> | Changed upload-lb.esams IP addresses | [production] |