| 2015-09-17
      
      § | 
    
  | 08:05 | <paravoid> | eqiad-codfw -> eqiad-eqord-codfw migration | [production] | 
            
  | 07:49 | <moritzm> | repooled mw1180-mw1188 (T104968) | [production] | 
            
  | 07:42 | <dcausse> | elastic in eqiad plugin updates: restarting elastic1001 | [production] | 
            
  | 07:42 | <moritzm> | depooled mw1180-mw1188 (T104968) | [production] | 
            
  | 07:36 | <moritzm> | repooled mw1170-mw1179 (T104968) | [production] | 
            
  | 07:36 | <dcausse> | elastic in eqiad plugin updates: freezing indices | [production] | 
            
  | 07:27 | <moritzm> | depooled mw1170-mw1179 (T104968) | [production] | 
            
  | 07:14 | <_joe_> | uploading new HHVM package | [production] | 
            
  | 07:07 | <moritzm> | repooled mw1161-1168 (T104968) | [production] | 
            
  | 06:57 | <moritzm> | depooled mw1161-1168 (T104968) | [production] | 
            
  | 06:45 | <moritzm> | repooled mw1209-mw1220 with ferm enabled | [production] | 
            
  | 06:33 | <moritzm> | depooling mw1209-mw1220 (in two steps) | [production] | 
            
  | 05:47 | <l10nupdate@tin> | ResourceLoader cache refresh completed at Thu Sep 17 05:47:47 UTC 2015 (duration 47m 46s) | [production] | 
            
  | 03:06 | <l10nupdate@tin> | LocalisationUpdate completed (1.26wmf23) at 2015-09-17 03:06:33+00:00 | [production] | 
            
  | 03:03 | <l10nupdate@tin> | Synchronized php-1.26wmf23/cache/l10n: l10nupdate for 1.26wmf23 (duration: 06m 30s) | [production] | 
            
  | 02:45 | <l10nupdate@tin> | LocalisationUpdate completed (1.26wmf22) at 2015-09-17 02:45:48+00:00 | [production] | 
            
  | 02:39 | <l10nupdate@tin> | Synchronized php-1.26wmf22/cache/l10n: l10nupdate for 1.26wmf22 (duration: 11m 11s) | [production] | 
            
  
    | 2015-09-16
      
      § | 
    
  | 23:27 | <bblack> | updating eqiad switch configs for lvs1007-1012 vlan/trunk settings | [production] | 
            
  | 23:19 | <krenair@tin> | Synchronized php-1.26wmf23/extensions/MobileFrontend/resources/mobile.overlays/Overlay.less: https://gerrit.wikimedia.org/r/#/c/238865/ (duration: 00m 11s) | [production] | 
            
  | 23:13 | <gwicke> | started `nodetool rebuild -- eqiad` on restbase-test200{1,2 | [production] | 
            
  | 22:08 | <urandom> | disabling puppet in RESTBase eqiad staging cluster to test new code and config | [production] | 
            
  | 22:08 | <ottomata> | powercycling  analytcis1029, it is down? | [production] | 
            
  | 20:47 | <cscott> | updated OCG to version 4032a596ce6eb442b02cc6ee9b79263b1eb23275 | [production] | 
            
  | 19:38 | <ori> | Deployed statsv 0bfd9f06f / change I050a12d3b | [production] | 
            
  | 18:47 | <twentyafterfour@tin> | rebuilt wikiversions.cdb and synchronized wikiversions files: group1 wikis to 1.26wmf23 | [production] | 
            
  | 18:38 | <twentyafterfour@tin> | Synchronized php-1.26wmf23: syncing wmf23 ahead of deployment to group1 (duration: 01m 35s) | [production] | 
            
  | 17:34 | <paravoid> | asw-d-eqiad: toggling RE mastership again | [production] | 
            
  | 17:26 | <godog> | stop puppet on restbase* to apply https://gerrit.wikimedia.org/r/#/c/238738/ / merge / reenable puppet | [production] | 
            
  | 16:54 | <_joe_> | turned on the hhvm tmh, stopping the zend ones for testing | [production] | 
            
  | 16:44 | <oblivian@tin> | Synchronized wmf-config/CommonSettings.php: use ffmpeg whereever possible (duration: 00m 12s) | [production] | 
            
  | 16:16 | <bblack> | upgrading pybal on lvs400[12] | [production] | 
            
  | 16:12 | <bblack> | upgrading pybal on lvs400[34], lvs300[34] | [production] | 
            
  | 16:08 | <bblack> | upgrading pybal on lvs200[123] | [production] | 
            
  | 16:05 | <bblack> | upgrading pybal on lvs200[456] | [production] | 
            
  | 15:44 | <_joe_> | uploading pybal 1.10 to reprepro, installing to the test cluster | [production] | 
            
  | 15:24 | <moritzm> | uploaded debdeploy 0.0.6 to apt.wikimedia.org | [production] | 
            
  | 15:10 | <hashar> | Started using Nodepool spawned instances.  Moved integration-jjb-config-diff Jenkins job to Nodepool with https://gerrit.wikimedia.org/r/#/c/238752/  . See also: https://phabricator.wikimedia.org/T112750 | [production] | 
            
  | 15:05 | <demon@tin> | Synchronized wmf-config/CommonSettings.php: Add m.wikidata.org to wgCrossSiteAJAXdomains (duration: 00m 12s) | [production] | 
            
  | 14:51 | <_joe_> | experimenting on testwiki for poolcounter failure scenarios | [production] | 
            
  | 14:45 | <moritzm> | enabled ferm on mw1010 (jobrunner) in eqiad | [production] | 
            
  | 14:27 | <paravoid> | asw-d-eqiad: toggling RE mastership | [production] | 
            
  | 14:18 | <paravoid> | disabling/ignoring asw-d-eqiad @ librenms | [production] | 
            
  | 14:09 | <jynus> | upgrading and restarting db1051 | [production] | 
            
  | 13:57 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Depool es1051 for maintenance (duration: 00m 12s) | [production] | 
            
  | 13:40 | <urandom> | initiating Cassandra repair on restbase1007 (nodetool repair -pr) | [production] | 
            
  | 13:40 | <catrope@tin> | Synchronized php-1.26wmf23: (no message) (duration: 01m 37s) | [production] | 
            
  | 13:35 | <moritzm> | repooled mw1149-mw1151 (with ferm enabled) | [production] | 
            
  | 13:24 | <moritzm> | depooled mw1149-mw1151 (for enabling ferm) | [production] | 
            
  | 13:19 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Reverting depool of es1055 (duration: 00m 12s) | [production] | 
            
  | 13:15 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Depool es1055 for maintenance (duration: 00m 12s) | [production] |