| 2016-05-19
      
      § | 
    
  | 23:33 | <dereckson@tin> | Started scap: Revert "Convert Special:WhatLinksHere from XML form to OOUI form" ([[Gerrit:289772]], T135773) | [production] | 
            
  | 23:29 | <dereckson@tin> | Synchronized wmf-config/InitialiseSettings.php: Drop already-enabled VisualEditorNewAccountEnableProportion wikis (duration: 00m 27s) | [production] | 
            
  | 23:25 | <dereckson@tin> | Synchronized wmf-config/InitialiseSettings.php: Move VisualEditor to secondary status on English Wikipedia (T132806) (duration: 00m 29s) | [production] | 
            
  | 23:13 | <dereckson@tin> | Synchronized wmf-config/throttle.php: Add Account throttle exception for SF Edit-a-thon (T135777) (duration: 00m 27s) | [production] | 
            
  | 23:10 | <dereckson@tin> | Synchronized wmf-config/CommonSettings.php: Enable experimental Video.js player on test2wiki (2/2) (duration: 00m 27s) | [production] | 
            
  | 23:09 | <dereckson@tin> | Synchronized wmf-config/InitialiseSettings.php: Enable experimental Video.js player on test2wiki (1/2) (duration: 00m 26s) | [production] | 
            
  | 23:09 | <dereckson@tin> | Synchronized wmf-config/InitialiseSettings.php: Enable experimental Video.js player on test2wiki (1/2) (duration: 00m 30s) | [production] | 
            
  | 22:13 | <krenair@tin> | Synchronized php-1.28.0-wmf.2/extensions/VisualEditor/ApiVisualEditor.php: rv for now, have another idea for later (duration: 00m 40s) | [production] | 
            
  | 21:28 | <krenair@tin> | Synchronized php-1.28.0-wmf.2/extensions/VisualEditor/ApiVisualEditor.php: more debug logs (duration: 00m 25s) | [production] | 
            
  | 21:02 | <yurik_> | deployed tilerator service update https://gerrit.wikimedia.org/r/#/c/289736/ | [production] | 
            
  | 21:01 | <krenair@tin> | Synchronized php-1.28.0-wmf.2/extensions/VisualEditor/ApiVisualEditor.php: (no message) (duration: 00m 27s) | [production] | 
            
  | 20:50 | <krenair@tin> | Synchronized php-1.28.0-wmf.2/extensions/VisualEditor/ApiVisualEditor.php: (no message) (duration: 00m 25s) | [production] | 
            
  | 20:49 | <krenair@tin> | Synchronized php-1.28.0-wmf.2/extensions/VisualEditor/ApiVisualEditor.php: update to the debug logging to try to find other params (duration: 00m 28s) | [production] | 
            
  | 20:45 | <urandom> | RESTBase staging Cassandra upgrade complete : T126629 | [production] | 
            
  | 20:44 | <urandom> | restbase-test2003.codfw.wmnet upgraded and online : T126629 | [production] | 
            
  | 20:42 | <urandom> | Upgrading Cassandra to 2.2.6 on restbase-test2003.codfw.wmnet : T126629 | [production] | 
            
  | 20:41 | <urandom> | Stopping Cassandra on restbase-test2003.codfw.wmnet : T126629 | [production] | 
            
  | 20:38 | <urandom> | restbase-test2002.codfw.wmnet upgraded and online : T126629 | [production] | 
            
  | 20:36 | <krenair@tin> | Synchronized php-1.28.0-wmf.2/extensions/VisualEditor/ApiVisualEditor.php: slight update to the debug logging to try to find the callers (duration: 00m 31s) | [production] | 
            
  | 20:36 | <urandom> | Upgrading Cassandra to 2.2.6 on restbase-test2002.codfw.wmnet : T126629 | [production] | 
            
  | 20:35 | <urandom> | Stopping Cassandra on restbase-test2002.codfw.wmnet : T126629 | [production] | 
            
  | 20:35 | <chasemp> | iridium try to block vandal by ip temp so puppet disable and edit of /etc/apache2/phabbanlist.conf | [production] | 
            
  | 20:31 | <urandom> | restbase-test2001.codfw.wmnet upgraded and online : T126629 | [production] | 
            
  | 20:29 | <urandom> | Upgrading Cassandra to 2.2.6 on restbase-test2001.codfw.wmnet : T126629 | [production] | 
            
  | 20:28 | <urandom> | Stopping Cassandra on restbase-test2001.codfw.wmnet : T126629 | [production] | 
            
  | 20:19 | <urandom> | praseodymium.eqiad.wmnet upgraded and online : T126629 | [production] | 
            
  | 20:16 | <urandom> | Upgrading Cassandra to 2.2.6 on praseodymium.eqiad.wmnet : T126629 | [production] | 
            
  | 20:16 | <aaron@tin> | Synchronized wmf-config/CommonSettings.php: Lowered  to 5 (duration: 00m 29s) | [production] | 
            
  | 20:15 | <urandom> | Stopping Cassandra on praseodymium.eqiad.wmnet : T126629 | [production] | 
            
  | 20:00 | <urandom> | Upgrading Cassandra on cerium.eqiad.wmnet, and forcing puppet run : T126629 | [production] | 
            
  | 19:59 | <urandom> | Stopping Cassanra on cerium.eqiad.wmnet for Cassandra 2.2.6 upgrade : T126629 | [production] | 
            
  | 19:32 | <urandom> | Disabling puppet in RESTBase staging : T126629 | [production] | 
            
  | 19:26 | <twentyafterfour@tin> | rebuilt wikiversions.php and synchronized wikiversions files: all wikis to 1.28.0-wmf.2 | [production] | 
            
  | 18:38 | <cscott> | updated Parsoid to version 67816adf (T100681, T130638) | [production] | 
            
  | 18:27 | <urandom> | Enabling puppet on xenon.eqiad.wmnet and forcing run : T126629 | [production] | 
            
  | 18:26 | <urandom> | Upgrading Cassandra on xenon.eqiad.wmnet to 2.2.6 : T126629 | [production] | 
            
  | 18:26 | <cscott> | synced; restarted Parsoid on wtp1001.eqiad as a canary | [production] | 
            
  | 18:25 | <urandom> | Stopping Cassandra on xenon.eqiad.wmnet : T126629 | [production] | 
            
  | 18:23 | <cscott> | re-attempting parsoid deploy of 67816adf | [production] | 
            
  | 18:22 | <cscott> | to cleanup Parsoid repos ori ran: salt 'wtp*' cmd.run "sed -i -e '/106801025/d' /srv/deployment/parsoid/deploy/src/lib/api/routes.js" | [production] | 
            
  | 18:13 | <cscott> | parsoid deploy reverted to parsoid/deploy-sync-20160504-200410 tag (b0d015fa); 21 repos still dirty | [production] | 
            
  | 18:09 | <cscott> | starting to revert Parsoid deploy due to unresolved dirty repos | [production] | 
            
  | 18:05 | <cscott> | git-deploy of Parsoid failed with "21/44 minions completed checkout" due to dirty repos, root had applied patch during the restbase/changeprop/parsoid outage. | [production] | 
            
  | 17:39 | <cscott> | starting Parsoid deploy (of 67816adf) | [production] | 
            
  | 17:25 | <elukey> | execute sysctl -w net.netfilter.nf_conntrack_max=512000 on kafka1013 as temporary measure (investigating why conntrack count is higher after leader election) - T135557 | [production] | 
            
  | 17:13 | <jynus@tin> | Synchronized wmf-config/db-eqiad.php: Repool db1033 after maintenance with low weight; increase db1029 weight (duration: 00m 29s) | [production] | 
            
  | 16:31 | <volans> | Set runtime value for max_allowed_packet, innodb_buffer_pool_dump_at_shutdown, innodb_buffer_pool_load_at_startup to their configured values for s1-s7, es1-es3, x1 T133333 | [production] | 
            
  | 16:31 | <urandom> | Disabling puppet on xenon.eqiad.wmnet in preparation for Cassandra upgrade : T126629 | [production] |