| 2016-11-10
      
      § | 
    
  | 23:10 | <mutante> | mw1185 - service hhvm restart | [production] | 
            
  | 23:00 | <maxsem@tin> | Finished scap: https://gerrit.wikimedia.org/r/#/c/320864/ (duration: 22m 42s) | [production] | 
            
  | 22:38 | <maxsem@tin> | Started scap: https://gerrit.wikimedia.org/r/#/c/320864/ | [production] | 
            
  | 22:35 | <maxsem@tin> | scap sync-l10n completed (1.29.0-wmf.2) (duration: 01m 01s) | [production] | 
            
  | 21:15 | <papaul> | kafka2003 - signing puppet certs, salt-key, initial run | [production] | 
            
  | 21:09 | <twentyafterfour@tin> | rebuilt wikiversions.php and synchronized wikiversions files: all wikis to 1.29.0-wmf.2 | [production] | 
            
  | 21:08 | <twentyafterfour@tin> | Synchronized php-1.29.0-wmf.2/extensions/ORES/includes/ApiHooks.php: deploy I86e97b05b56b90d956616ef16e8aa86d96403b8c (duration: 00m 47s) | [production] | 
            
  | 20:54 | <twentyafterfour@tin> | rebuilt wikiversions.php and synchronized wikiversions files: Revert "all wikis to 1.29.0-wmf.2" (Fatal errors spike) | [production] | 
            
  | 20:49 | <twentyafterfour@tin> | rebuilt wikiversions.php and synchronized wikiversions files: all wikis to 1.29.0-wmf.2 | [production] | 
            
  | 20:33 | <mutante> | neon - deactivated puppet node, scheduled icinga downtime, shutdown server permanently (T125023) | [production] | 
            
  | 20:32 | <mutante> | neon - shutdown -h now (scheduled 3 days downtime, nothing that looked worth saving in homes) | [production] | 
            
  | 20:28 | <mutante> | neon - deactivate puppet node | [production] | 
            
  | 20:25 | <twentyafterfour@tin> | Synchronized wmf-config/InitialiseSettings-labs.php: sync LABS: enable mapframe everywhere (I35709ed2903b28a2d4d6e8528ac1fcf361483e76) (duration: 00m 50s) | [production] | 
            
  | 19:37 | <dcausse> | elastic@eqiad: reindexing commonswiki (logs in terbium.eqiad.wmnet:~dcausse/commons_reindex/cirrus_log) - T150232 | [production] | 
            
  | 19:33 | <bblack> | cache_*: restarting nginx for libssl update (seamless) | [production] | 
            
  | 19:31 | <bblack> | upgrading libssl1.1 to 1.1.0c on other misc hosts... | [production] | 
            
  | 19:30 | <Pchelolo> | update RESTBase to 6bfa0f75f | [production] | 
            
  | 19:24 | <Pchelolo> | update RESTBase to 6bfa0f75f - canary on restbase1007 | [production] | 
            
  | 19:22 | <bblack> | upgrading openssl to 1.1.0c on cache_* | [production] | 
            
  | 19:19 | <Pchelolo> | update RESTBase to 6bfa0f75f - staging | [production] | 
            
  | 19:16 | <thcipriani@tin> | Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:320552|Add PageViewInfo log channel (T129602)]] (duration: 00m 49s) | [production] | 
            
  | 19:09 | <thcipriani@tin> | Synchronized wmf-config/InitialiseSettings-labs.php: SWAT: [[gerrit:320450|Make beta PageViewInfo use the production pageview API (T129602)]] (labs-only-change) (duration: 00m 48s) | [production] | 
            
  | 19:02 | <cwd> | updated payments from c1fa73c649986be89a6a038b9e19f6b3ea19e537 to 3b3c8ce6c7bc24beb520a3e2f507df4514de4679 | [production] | 
            
  | 18:57 | <moritzm> | uploaded openssl 1.1.0c for jessie-wikimedia to carbon | [production] | 
            
  | 18:42 | <paravoid> | upgrading (and restarting) nginx on sodium | [production] | 
            
  | 17:46 | <urandom> | T133395: Convert final 25 RESTBase tables to TWCS | [production] | 
            
  | 17:34 | <cwd> | deployed EVERYTHING... changed config tree to c6a7b17432188a0a4f990061b488208e64fe39dd and civicrm-authonly from 56eadabf705b3d035e11bb9fd3478457c159e40d to df50d2d7b6a8494fb3e049df6b1e101611270 | [production] | 
            
  | 17:33 | <cwd> | restarted slander | [production] | 
            
  | 17:32 | <cwd> | restarted dash | [production] | 
            
  | 16:58 | <urandom> | T133395: Performing next 25 RESTBase table conversions to TWCS | [production] | 
            
  | 15:09 | <marostegui> | Deploy schema change s4 commonswiki.template links (db1068) - https://phabricator.wikimedia.org/T149079 | [production] | 
            
  | 15:01 | <elukey> | restored mw1284 to its settings | [production] | 
            
  | 14:53 | <jynus> | applying schema change on s3 (page) T69223 | [production] | 
            
  | 14:47 | <elukey> | de-pooling mw1284 to raise mod_proxy_fcgi log level manually (temporary for an ongoing investigation) | [production] | 
            
  | 14:31 | <bblack> | cache_text: upgrade nginx to 1.11.4-1+wmf14 | [production] | 
            
  | 14:20 | <bblack> | cache_upload: upgrade nginx to 1.11.4-1+wmf14 | [production] | 
            
  | 14:04 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Depool db1068 - T149079 (duration: 00m 48s) | [production] | 
            
  | 13:54 | <bblack> | restarting varnishes on cache_maps + cache_misc | [production] | 
            
  | 13:51 | <marostegui> | Enabled gtid+ssl on db2010,db2012,db2030 | [production] | 
            
  | 13:45 | <bblack> | cache_misc: upgrade nginx to 1.11.4-1+wmf14 | [production] | 
            
  | 13:29 | <bblack> | cache_maps: upgrade nginx to 1.11.4-1+wmf14 | [production] | 
            
  | 13:25 | <marostegui> | Restarting mysql in misc shard slaves (only codfw - db2010,db2012,db2030) to apply a MySQL config - T149418 | [production] | 
            
  | 13:20 | <gehel> | restart wdqs1* for jvm update | [production] | 
            
  | 13:07 | <gehel> | restart wdqs2* for jvm update | [production] | 
            
  | 12:01 | <ema> | upgrading cp1068 (text-eqiad) to varnish 4 -- T131503 | [production] | 
            
  | 11:45 | <ema> | upgrading cp1067 (text-eqiad) to varnish 4 -- T131503 | [production] | 
            
  | 11:25 | <ema> | upgrading cp1066 (text-eqiad) to varnish 4 -- T131503 | [production] |