| 2015-09-03
      
      ยง | 
    
  | 21:18 | <urandom> | bouncing Cassandra on restbase1001 to apply temporary GC settings | [production] | 
            
  | 19:54 | <bearND> | MobileApps deployed sha1 553c399 | [production] | 
            
  | 19:31 | <twentyafterfour@tin> | rebuilt wikiversions.cdb and synchronized wikiversions files: wikipedia wikis to 1.26wmf21 | [production] | 
            
  | 18:13 | <ottomata> | rolling restart of hadoop  yarn nodemanagers to pick up Yarn AppMaster port range limitation to apply ferm rules. | [production] | 
            
  | 18:04 | <catrope@tin> | Synchronized wmf-config/CommonSettings.php: Add plumbing code for Flow beta feature (unused for now) (duration: 00m 12s) | [production] | 
            
  | 18:03 | <catrope@tin> | Synchronized wmf-config/InitialiseSettings.php: Add plumbing code for Flow beta feature (unused for now) (duration: 00m 12s) | [production] | 
            
  | 17:39 | <krenair@tin> | Synchronized php-1.26wmf21/extensions/OpenStackManager/nova/OpenStackNovaController.php: https://gerrit.wikimedia.org/r/#/c/235769/ (duration: 00m 12s) | [production] | 
            
  | 17:34 | <mutante> | bromine - deleting policy docroot | [production] | 
            
  | 17:06 | <jynus> | cloning es1006 mysql data into es1015 [ETA:8h] | [production] | 
            
  | 16:30 | <bblack> | updating nginx->1.9.4 on cp1071, cp3033 for prod validation before broader rollout | [production] | 
            
  | 16:30 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: es3 master switchover from es1009 to es1014 (eqiad) (duration: 00m 13s) | [production] | 
            
  | 16:28 | <jynus@tin> | Synchronized wmf-config/db-codfw.php: es3 master switchover from es1009 to es1014 (codfw) (duration: 00m 13s) | [production] | 
            
  | 16:26 | <mutante> | imported jenkins 1.609.3 into APT repo | [production] | 
            
  | 16:23 | <legoktm> | fixed content model of Template:Languages@metawiki | [production] | 
            
  | 16:21 | <robh> | re-enabling puppet on all mw systems | [production] | 
            
  | 16:14 | <robh> | disabling puppet on all mw systems for apache config update | [production] | 
            
  | 16:01 | <jynus> | performing es3 master switchover from es1009 to es1014 | [production] | 
            
  | 15:40 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: depool es1006 (duration: 00m 12s) | [production] | 
            
  | 15:17 | <hashar> | stopping nodepool on labnodepool1001.eqiad.wmnet not ready yet | [production] | 
            
  | 15:15 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: es2 master switchover from es1006 to es1011 (eqiad) (duration: 00m 13s) | [production] | 
            
  | 15:14 | <jynus@tin> | Synchronized wmf-config/db-codfw.php: es2 master switchover from es1006 to es1011 (codfw) (duration: 00m 12s) | [production] | 
            
  | 15:05 | <demon@tin> | Synchronized wmf-config/InitialiseSettings.php: (no message) (duration: 00m 13s) | [production] | 
            
  | 15:04 | <demon@tin> | Synchronized php-1.26wmf21/extensions/Translate/: (no message) (duration: 00m 15s) | [production] | 
            
  | 14:51 | <jynus> | performing es2 master switchover from es1006 to es1011 | [production] | 
            
  | 14:33 | <paravoid> | rebooting msw1-eqiad | [production] | 
            
  | 14:28 | <twentyafterfour> | restarted phd (phabricator daemon) to pick up new configuration | [production] | 
            
  | 14:24 | <paravoid> | changing IPv6 RA interval/lifetime/virtual-router-only @ eqiad | [production] | 
            
  | 14:21 | <paravoid> | rebooting msw1-codfw | [production] | 
            
  | 13:17 | <paravoid> | upgrading mr1-esams and mr1-eqiad to newer junos | [production] | 
            
  | 13:13 | <godog> | bounce carbon daemons on graphite1001 | [production] | 
            
  | 12:42 | <chasemp> | unban elastic1001 and put back in service | [production] | 
            
  | 12:24 | <chasemp> | move all shards off of elastic1001 | [production] | 
            
  | 12:24 | <chasemp> | disable elastic1001 in lvs as we are gonig to try fw apply round #2 | [production] | 
            
  | 11:01 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Repool db1028; increase the load of es1010, es1013 and es1017 (duration: 00m 12s) | [production] | 
            
  | 10:45 | <jynus> | applying schema change for ContentTranslation on x1-master "wikishared" | [production] | 
            
  | 10:02 | <godog> | reenable puppet on ms-be1* | [production] | 
            
  | 09:16 | <jynus> | started profiling mysql queries at phabricator. Only a 1% overhead is expected. | [production] | 
            
  | 09:12 | <moritzm> | updated rsyncd firewall rules (see https://gerrit.wikimedia.org/r/235425 for details) | [production] | 
            
  | 09:12 | <godog> | stop puppet on ms-be1* after ferm rsync change | [production] | 
            
  | 08:23 | <godog> | fixup current graphite retention T96662 | [production] | 
            
  | 07:26 | <moritzm> | enabled ferm on dbstore* servers in codfw | [production] | 
            
  | 06:29 | <l10nupdate@tin> | ResourceLoader cache refresh completed at Thu Sep  3 06:29:35 UTC 2015 (duration 29m 34s) | [production] | 
            
  | 03:09 | <l10nupdate@tin> | LocalisationUpdate completed (1.26wmf21) at 2015-09-03 03:09:20+00:00 | [production] | 
            
  | 03:06 | <l10nupdate@tin> | Synchronized php-1.26wmf21/cache/l10n: l10nupdate for 1.26wmf21 (duration: 05m 32s) | [production] | 
            
  | 02:45 | <l10nupdate@tin> | LocalisationUpdate completed (1.26wmf20) at 2015-09-03 02:45:36+00:00 | [production] | 
            
  | 02:39 | <l10nupdate@tin> | Synchronized php-1.26wmf20/cache/l10n: l10nupdate for 1.26wmf20 (duration: 10m 41s) | [production] | 
            
  | 01:32 | <krenair@tin> | Synchronized wmf-config/interwiki.cdb: Updating interwiki cache (duration: 00m 12s) | [production] | 
            
  | 00:36 | <ori@tin> | Synchronized php-1.26wmf21/includes/parser/Preprocessor_Hash.php: Idd1acd903: Decline to cache preprocessor items larger than 1 Mb (duration: 00m 11s) | [production] | 
            
  | 00:36 | <ori@tin> | Synchronized php-1.26wmf20/includes/parser/Preprocessor_Hash.php: Idd1acd903: Decline to cache preprocessor items larger than 1 Mb (duration: 00m 13s) | [production] | 
            
  | 00:27 | <RoanKattouw> | Deployed patch for T111029 | [production] |