| 2015-05-28
      
      § | 
    
  | 16:52 | <paravoid> | redirecting ns1 traffic to rubidium (= ns0) in preparation for baham upgrade | [production] | 
            
  | 15:54 | <kartik> | Finished scap: Update ContentTranslation (duration: 03m 19s) | [production] | 
            
  | 15:50 | <kartik> | Started scap: Update ContentTranslation | [production] | 
            
  | 15:47 | <thcipriani> | Synchronized wmf-config/abusefilter.php: SWAT: Modify AbuseFilter block configuration on eswikibooks [[gerrit:206510]] (duration: 00m 15s) | [production] | 
            
  | 15:40 | <thcipriani> | Synchronized wmf-config/InitialiseSettings.php: SWAT: Prevent indexing of User: namespace on ukwiki [[gerrit:210680]] (duration: 00m 14s) | [production] | 
            
  | 15:35 | <thcipriani> | Synchronized wmf-config/InitialiseSettings.php: SWAT: Enable NewUserMessage on sa.wikipedia [[gerrit:212724]] (duration: 00m 13s) | [production] | 
            
  | 15:28 | <godog> | set operations/debs/python-statsd as hidden in gerrit -- deprecated | [production] | 
            
  | 15:24 | <thcipriani> | Synchronized wmf-config/InitialiseSettings.php: SWAT:  Enable Extension:NewUserMessage on ta.wikipedia [[gerrit:213841]] (duration: 00m 12s) | [production] | 
            
  | 15:13 | <thcipriani> | Synchronized wmf-config/InitialiseSettings.php: SWAT: Enable SandboxLink for cswiki [[gerrit:214247]] (duration: 00m 15s) | [production] | 
            
  | 15:11 | <godog> | set operations/debs/txstatsd as hidden in gerrit -- deprecated | [production] | 
            
  | 15:05 | <thcipriani> | Synchronized wmf-config/InitialiseSettings.php: SWAT: CX: Add wikis for CX deployment on 20150528 [[gerrit:213992]] (duration: 00m 15s) | [production] | 
            
  | 15:00 | <bblack> | merged up https://gerrit.wikimedia.org/r/214345 - look here if IPv6 problems! | [production] | 
            
  | 14:37 | <cmjohnson1> | powering down dataset1001 to add disk array | [production] | 
            
  | 14:17 | <bblack> | deploying https://gerrit.wikimedia.org/r/214341 - keep in mind if ipv6-related issues arise! | [production] | 
            
  | 13:50 | <akosiaris> | started ircecho (icinga-wm) on neon | [production] | 
            
  | 13:46 | <hashar> | upgrading Jenkins git plugin from 1.4.6+wmf1 to 1.7.1 {{bug|T100655}}  and restarting Jenkins | [production] | 
            
  | 13:25 | <jynus> | Synchronized wmf-config/db-eqiad.php: repool pc1003 (not to confuse with db1003) after warmup (duration: 00m 15s) | [production] | 
            
  | 13:11 | <akosiaris> | killed ircecho service on neon | [production] | 
            
  | 09:48 | <_joe_> | depooling the HHVM appserver. 503s reduced slightly but still non-irrelevant | [production] | 
            
  | 09:37 | <jynus> | Synchronized wmf-config/db-eqiad.php: Depool pc1003 (duration: 00m 15s) | [production] | 
            
  | 09:35 | <_joe_> | pooling mw1152 into the imagescalers pool after fixes made in Lyon | [production] | 
            
  | 06:11 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Thu May 28 06:09:56 UTC 2015 (duration 9m 55s) | [production] | 
            
  | 04:22 | <springle> | reload dbstore1002 s7 | [production] | 
            
  | 02:41 | <LocalisationUpdate> | completed (1.26wmf6) at 2015-05-28 02:40:00+00:00 | [production] | 
            
  | 02:36 | <l10nupdate> | Synchronized php-1.26wmf6/cache/l10n: (no message) (duration: 06m 46s) | [production] | 
            
  | 02:20 | <springle> | set global read_only=0 on pc1001 pc1002. this config broke in the recent upgrade | [production] | 
            
  | 00:59 | <legoktm> | Synchronized php-1.26wmf8/resources/: Revert "Convert mediawiki.toc and mediawiki.user to using mw.cookie" (duration: 00m 17s) | [production] | 
            
  | 00:58 | <legoktm> | Synchronized php-1.26wmf7/resources/: Revert "Convert mediawiki.toc and mediawiki.user to using mw.cookie" (duration: 00m 13s) | [production] | 
            
  | 00:07 | <twentyafterfour> | Synchronized rpc/RunJobs.php: deploy I98b8a4ddbcdd58d1f2f23e4b1bf154f10b6b279e (duration: 00m 17s) | [production] | 
            
  
    | 2015-05-27
      
      § | 
    
  | 23:46 | <awight> | updated payments from 858b87319daa3d66f62eb32e08cefc6b061748d1 to aa66797553fbcfb63f7cf29abccc44d060b65db0 | [production] | 
            
  | 23:31 | <twentyafterfour> | Finished scap: scap, now with 10% less fail (duration: 22m 07s) | [production] | 
            
  | 23:26 | <awight> | payments rolled back to 858b87319daa3d66f62eb32e08cefc6b061748d1 | [production] | 
            
  | 23:24 | <awight> | updated payments from 858b87319daa3d66f62eb32e08cefc6b061748d1 to aa66797553fbcfb63f7cf29abccc44d060b65db0 | [production] | 
            
  | 23:09 | <twentyafterfour> | Started scap: scap, now with 10% less fail | [production] | 
            
  | 22:57 | <ori> | rebuilt wikiversions.cdb and synchronized wikiversions files: (no message) | [production] | 
            
  | 21:49 | <mutante> | restarted hhvm on mw1250,mw1254,mw1256 | [production] | 
            
  | 21:47 | <mutante> | restarted hhvm on mw1017,mw1243,mw1244 | [production] | 
            
  | 21:42 | <bblack> | restarting hhvm everywhere on 30s intervals between hosts | [production] | 
            
  | 21:10 | <twentyafterfour> | Synchronized php-1.26wmf8: Fix ConfirmEdit fatal Change-Id: I22353669a85391c3d9760a5253cac1263e895cf9 (duration: 01m 08s) | [production] | 
            
  | 20:47 | <twentyafterfour> | Purged l10n cache for 1.26wmf6 | [production] | 
            
  | 20:45 | <twentyafterfour> | rebuilt wikiversions.cdb and synchronized wikiversions files: group0 to 1.26wmf8 | [production] | 
            
  | 20:41 | <twentyafterfour> | rebuilt wikiversions.cdb and synchronized wikiversions files: wikipedias to 1.26wmf7 | [production] | 
            
  | 20:36 | <twentyafterfour> | Finished scap: testwiki to php-1.26wmf8 and rebuild l10n cache (duration: 67m 53s) | [production] | 
            
  | 19:28 | <twentyafterfour> | Started scap: testwiki to php-1.26wmf8 and rebuild l10n cache | [production] | 
            
  | 19:18 | <twentyafterfour> | Started scap: testwiki to php-1.26wmf8 and rebuild l10n cache | [production] | 
            
  | 18:12 | <moritzm> | Uploaded gridengine_6.2u5-4+wmf2 for precise-wikimedia to apt.wikimedia.org | [production] | 
            
  | 17:55 | <jynus> | Synchronized wmf-config/db-eqiad.php: repool pc1002 (duration: 00m 13s) | [production] | 
            
  | 17:42 | <paravoid> | rebooting asw-d2-eqiad | [production] | 
            
  | 17:41 | <ottomata> | initiating controlled shutdown of kafka broker analytics1018 in anticipation of switch reboot | [production] | 
            
  | 15:33 | <jynus> | Synchronized wmf-config/db-eqiad.php: depool pc1002 (duration: 00m 13s) | [production] |