| 2018-04-03
      
      ยง | 
    
  | 23:16 | <dereckson@tin> | Synchronized wmf-config/CommonSettings.php: Make a note about the loading order of GlobalPreferences and Echo ([[Gerrit:422642]]) (no-op) (duration: 01m 17s) | [production] | 
            
  | 23:10 | <dereckson@tin> | Synchronized wmf-config/InitialiseSettings.php: Rollout VirtualPageViews (final stage) (T189906) (duration: 01m 19s) | [production] | 
            
  | 22:34 | <mutante> | cobalt - puppet disabled temporarily to apply fix to "simplify directory structure" change .. on gerrit2001 first | [production] | 
            
  | 22:25 | <mutante> | restarting Apache on phab1001 - T182832 | [production] | 
            
  | 22:14 | <twentyafterfour> | Finished MediaWiki Train for group0, 1.31.0-wmf.28 refs T183967 | [production] | 
            
  | 22:12 | <twentyafterfour@tin> | Pruned MediaWiki: 1.31.0-wmf.25 [keeping static files] (duration: 01m 55s) | [production] | 
            
  | 22:10 | <twentyafterfour@tin> | Pruned MediaWiki: 1.31.0-wmf.24 [keeping static files] (duration: 04m 18s) | [production] | 
            
  | 21:30 | <twentyafterfour@tin> | rebuilt and synchronized wikiversions files: group0 wikis to 1.31.0-wmf.28 refs T183967 | [production] | 
            
  | 21:15 | <twentyafterfour@tin> | Finished scap: testwikis wikis to 1.31.0-wmf.28 refs T183967 (duration: 46m 38s) | [production] | 
            
  | 21:13 | <urandom> | (re)starting restbase-dev1004-{a,b} (ooms), and enabling alternately patched cassandra 3.11.2 build - T186751 | [production] | 
            
  | 20:28 | <twentyafterfour@tin> | Started scap: testwikis wikis to 1.31.0-wmf.28 refs T183967 | [production] | 
            
  | 20:22 | <ejegg> | disabled thank you mail sender | [production] | 
            
  | 19:46 | <urandom> | restarting restbase-dev1004-{a,b} to enable patched cassandra 3.11.2 build - T186751 | [production] | 
            
  | 19:07 | <twentyafterfour> | Preparing to deploy 1.31.0-wmf.28 refs T183967 | [production] | 
            
  | 18:25 | <urandom> | upgrading restbase-dev1006-b to cassandra 3.11.2 - T186751 | [production] | 
            
  | 18:23 | <urandom> | upgrading restbase-dev1006-a to cassandra 3.11.2 - T186751 | [production] | 
            
  | 18:20 | <urandom> | upgrading restbase-dev1005-b to cassandra 3.11.2 - T186751 | [production] | 
            
  | 18:18 | <urandom> | upgrading restbase-dev1005-a to cassandra 3.11.2 - T186751 | [production] | 
            
  | 18:15 | <urandom> | upgrading restbase-dev1004-b to cassandra 3.11.2 - T186751 | [production] | 
            
  | 18:13 | <urandom> | upgrading restbase-dev1004-a to cassandra 3.11.2 - T186751 | [production] | 
            
  | 18:05 | <mutante> | rhodium - closing idle screen session from maintenance work on puppetmasters | [production] | 
            
  | 18:03 | <mutante> | elnath - fixing and re-enabling Icinga alert about screens, none are running, spare hosts should not have these | [production] | 
            
  | 17:59 | <mutante> | restarting ferm on bromine | [production] | 
            
  | 17:40 | <elukey> | manually set net.ipv4.tcp_tw_reuse=1 on restbase1007 as test for T190213 | [production] | 
            
  | 17:35 | <sbisson@tin> | Finished deploy [tilerator/deploy@8e68cb8]: Deploying tilerator i18n to maps-test* (take 2) (duration: 00m 25s) | [production] | 
            
  | 17:35 | <sbisson@tin> | Started deploy [tilerator/deploy@8e68cb8]: Deploying tilerator i18n to maps-test* (take 2) | [production] | 
            
  | 17:28 | <awight@tin> | Finished deploy [ores/deploy@7701cee]: ORES versioned virtualenv, T181071 (duration: 01m 27s) | [production] | 
            
  | 17:28 | <sbisson@tin> | Finished deploy [tilerator/deploy@03add2d]: Deploying tilerator i18n to maps-test* (duration: 04m 09s) | [production] | 
            
  | 17:27 | <awight@tin> | Started deploy [ores/deploy@7701cee]: ORES versioned virtualenv, T181071 | [production] | 
            
  | 17:25 | <awight@tin> | Finished deploy [ores/deploy@7701cee]: ORES versioned virtualenv, T181071 (duration: 00m 27s) | [production] | 
            
  | 17:25 | <awight@tin> | Started deploy [ores/deploy@7701cee]: ORES versioned virtualenv, T181071 | [production] | 
            
  | 17:24 | <sbisson@tin> | Started deploy [tilerator/deploy@03add2d]: Deploying tilerator i18n to maps-test* | [production] | 
            
  | 17:24 | <moritzm> | upgrading HHVM on labweb* | [production] | 
            
  | 17:18 | <jynus> | reloading labsdb proxy configuration | [production] | 
            
  | 17:08 | <elukey> | manually set net.ipv4.tcp_tw_reuse=1 on restbase2001 as test for T190213 | [production] | 
            
  | 16:53 | <demon@tin> | Finished deploy [gerrit/gerrit@aa1a1a0]: no-op, pushing empty motd.config file (duration: 00m 11s) | [production] | 
            
  | 16:53 | <demon@tin> | Started deploy [gerrit/gerrit@aa1a1a0]: no-op, pushing empty motd.config file | [production] | 
            
  | 16:33 | <urandom> | rebooting restbase-dev1006 - T186751 | [production] | 
            
  | 16:10 | <urandom> | rebooting restbase-dev1004 - T186751 | [production] | 
            
  | 16:04 | <ariel@tin> | Finished deploy [dumps/dumps@77dc467]: split up some large modules, prep work for prefetch changes (duration: 00m 04s) | [production] | 
            
  | 16:04 | <ariel@tin> | Started deploy [dumps/dumps@77dc467]: split up some large modules, prep work for prefetch changes | [production] | 
            
  | 15:39 | <elukey> | roll restart of zookeeper on conf100[123] to pick up prometheus monitoring | [production] | 
            
  | 15:09 | <godog> | depool ms-fe2005 to test rewrite.py - T183902 | [production] | 
            
  | 14:40 | <mobrovac@tin> | Synchronized wmf-config/InitialiseSettings.php: Switch the remaining high-traffic jobs for all wikis, file 2/2 - T190327 (duration: 00m 59s) | [production] | 
            
  | 14:39 | <ppchelko@tin> | Finished deploy [cpjobqueue/deploy@60a2292]: Switch all high traffic jobs to kafka T190327 (duration: 00m 44s) | [production] | 
            
  | 14:39 | <mobrovac@tin> | Synchronized wmf-config/jobqueue.php: Switch the remaining high-traffic jobs for all wikis, file 1/2 - T190327 (duration: 00m 59s) | [production] | 
            
  | 14:38 | <ppchelko@tin> | Started deploy [cpjobqueue/deploy@60a2292]: Switch all high traffic jobs to kafka T190327 | [production] | 
            
  | 13:48 | <anomie@tin> | Synchronized php-1.31.0-wmf.27/extensions/intersection/DynamicPageList.hooks.php: Backporting fix for T191116 ([[gerrit:423689]]) (duration: 00m 58s) | [production] | 
            
  | 13:47 | <anomie@tin> | Synchronized php-1.31.0-wmf.27/includes/specials/SpecialWhatlinkshere.php: Backporting fix for T191116 ([[gerrit:423688]]) (duration: 00m 58s) | [production] | 
            
  | 13:41 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Restore original weight for db1077 after alter table (duration: 00m 58s) | [production] |