| 2014-08-07
      
      § | 
    
  | 15:38 | <reedy> | Synchronized php-1.24wmf16/maintenance/findMissingFiles.php: (no message) (duration: 00m 20s) | [production] | 
            
  | 15:37 | <reedy> | Synchronized php-1.24wmf15/maintenance/findMissingFiles.php: (no message) (duration: 00m 17s) | [production] | 
            
  | 15:12 | <reedy> | Synchronized wmf-config/: (no message) (duration: 00m 13s) | [production] | 
            
  | 14:43 | <akosiaris> | uploaded varnish_3.0.5plus~x-wm7trusty1 on apt.wikimedia.org (for usage in trusty labs machines, notably cxserver) | [production] | 
            
  | 14:24 | <mutante> | shutting down elastic1018 | [production] | 
            
  | 14:12 | <^d> | elastic1018: blacklisted from shard allocation since it's dead | [production] | 
            
  | 14:05 | <mutante> | depooled elastic1018 - service wasnt running and signs of broken hardware (SSD) | [production] | 
            
  | 13:57 | <mark> | Temporarily set max connections to swift from cp1049 backend varnish from 1000 to 2000 | [production] | 
            
  | 13:56 | <mutante> | starting elasticsearch on elastic1018 | [production] | 
            
  | 12:23 | <hashar> | Zuul upgraded labs branch to match production (i.e. have same version of Zuul cloner) | [production] | 
            
  | 12:20 | <hashar> | restarting Zuul | [production] | 
            
  | 11:25 | <hoo> | Synchronized wmf-config/InitialiseSettings.php: I53f76a35ac - No longer allow voyage 'crats to usermerge (duration: 00m 15s) | [production] | 
            
  | 11:13 | <akosiaris> | removed laner@wikimedia.org entirely. It pointed to rlane@wikimedia.org which no longer exists | [production] | 
            
  | 11:12 | <akosiaris> | removed rlane from root@wikimedia.org and usability@wikimedia.org | [production] | 
            
  | 10:45 | <mutante> | iron, bast1001 - installed package upgrades | [production] | 
            
  | 09:13 | <hashar> | Jenkins: polling a new Jenkins slave using Trusty integration-slave1006-trusty [10.68.17.223] with 4 CPU. Copy pasted from 1004-trusty | [production] | 
            
  | 08:32 | <hashar> | Jenkins: switching [https://integration.wikimedia.org/ci/job/analytics-libcidr/|analytics-libcdr job] from https://github.com/wmf-analytics/libcidr/ to https://gerrit.wikimedia.org/r/analytics/libcidr | [production] | 
            
  | 07:44 | <springle> | Synchronized wmf-config/db-eqiad.php: move s4 api traffic to db1056 (duration: 00m 07s) | [production] | 
            
  | 07:39 | <mark> | Set OSPF metric 1000 on cr2-eqiad:xe-5/2/2 (GTT link) | [production] | 
            
  | 05:39 | <springle> | labsdb1002 restart | [production] | 
            
  | 03:48 | <springle> | labsdb1001 restart | [production] | 
            
  | 03:10 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Thu Aug  7 03:08:49 UTC 2014 (duration 8m 48s) | [production] | 
            
  | 02:29 | <LocalisationUpdate> | completed (1.24wmf16) at 2014-08-07 02:27:52+00:00 | [production] | 
            
  | 02:16 | <LocalisationUpdate> | completed (1.24wmf15) at 2014-08-07 02:15:45+00:00 | [production] | 
            
  
    | 2014-08-06
      
      § | 
    
  | 21:33 | <hashar> | Jenkins: moved mediawiki-core-regression-hhvm-master to run on Trusty instance | [production] | 
            
  | 20:26 | <hashar> | Jenkins: downgraded ansicolor plugin from 0.4 to 0.3.1   Some colors.js function emits ANSI codes to reset the color which are not properly understood | [production] | 
            
  | 20:06 | <hashar> | I have broke Zuul/Jenkins :-] | [production] | 
            
  | 18:53 | <hashar> | Jenkins slow startup is {{bug|69197}} | [production] | 
            
  | 18:50 | <hashar> | restarting jenkins | [production] | 
            
  | 18:49 | <hashar> | Stopping Jenkins.  Reverting upgrade of artifact deployer plugin | [production] | 
            
  | 18:10 | <mutante> | puppet-catalog-compiler says to "wait while Jenkins is getting ready to work" | [production] | 
            
  | 17:20 | <hashar> | Jenkins process jobs again, the UI will take a bunch of hours to load though due to some issue when initializing | [production] | 
            
  | 17:14 | <hashar> | killed Jenkins | [production] | 
            
  | 17:12 | <_joe_> | stopped the jobrunner on mw1053, was running in fcgi mode unpuppetized and with a broken vhost. Fixed it, it started spawning exceptions. DO NOT enable puppet again | [production] | 
            
  | 17:02 | <^d> | jenkins restarted, was stuck | [production] | 
            
  | 15:52 | <hashar> | Restarted Zuul and Zuul-merger on gallium to tweak logging settings {{gerrit|152118}} | [production] | 
            
  | 11:30 | <hoo> | Synchronized wmf-config/CommonSettings.php: Grant 'centralauth-rename' to 'steward' (duration: 00m 24s) | [production] | 
            
  | 11:26 | <demon> | Synchronized wmf-config/abusefilter.php: (no message) (duration: 00m 19s) | [production] | 
            
  | 10:10 | <hashar> | Jenkins web interface is back up | [production] | 
            
  | 09:54 | <demon> | Synchronized wmf-config/abusefilter.php: abuse filter settings for fawiki (duration: 00m 21s) | [production] | 
            
  | 07:33 | <hashar> | restarting Jenkins. It apparently like to parse the whole history on reload, so aborting that. | [production] | 
            
  | 07:13 | <hashar> | Upgrading Jenkins plugin and restarting. | [production] | 
            
  | 07:04 | <hashar> | upgrading Jenkins to latest LTS | [production] | 
            
  | 03:11 | <LocalisationUpdate> | ResourceLoader cache refresh completed at Wed Aug  6 03:10:06 UTC 2014 (duration 10m 5s) | [production] | 
            
  | 02:30 | <LocalisationUpdate> | completed (1.24wmf16) at 2014-08-06 02:29:00+00:00 | [production] | 
            
  | 02:17 | <LocalisationUpdate> | completed (1.24wmf15) at 2014-08-06 02:16:38+00:00 | [production] |