| 2016-11-11
      
      § | 
    
  | 21:15 | <hashar> | Restarted Jenkins. This time ZMQ managed to bind to port 8888 | [production] | 
            
  | 21:11 | <hashar> | jenkins: disabled/reenabled the ZMQ Event Publisher. Apparently it refused to start | [production] | 
            
  | 21:06 | <hashar> | Restarted Jenkins | [production] | 
            
  | 14:03 | <mobrovac> | restarting RESTBase to pick up https://gerrit.wikimedia.org/r/#/c/320529/ | [production] | 
            
  | 14:02 | <moritzm> | restarting hhvm on canary app servers to pick up libcurl update | [production] | 
            
  | 13:11 | <moritzm> | installing curl security updates | [production] | 
            
  | 11:10 | <ema> | cp3043 repooled with gethdr_extrachance=100 (T150503) | [production] | 
            
  | 10:59 | <ema> | cp3043 depooled, testing https://phabricator.wikimedia.org/P4406 (T150503) | [production] | 
            
  | 10:51 | <elukey> | restored mw1284 to its normal settings | [production] | 
            
  | 10:14 | <marostegui> | Deploy alter table dbstore1002 s4 commonswiki.revision - T147305 | [production] | 
            
  | 10:05 | <elukey> | increasing apache log level on mw1284 (depooling, applying config manually, re-pooling with lower weight) for a 503 investigation | [production] | 
            
  | 09:38 | <marostegui> | Deploy schema change s4 commonswiki.revision db1069 - T147305 | [production] | 
            
  | 07:33 | <marostegui@tin> | Synchronized wmf-config/db-eqiad.php: Repool db1068 - T149079 (duration: 00m 48s) | [production] | 
            
  | 02:28 | <l10nupdate@tin> | ResourceLoader cache refresh completed at Fri Nov 11 02:28:24 UTC 2016 (duration 5m 14s) | [production] | 
            
  | 02:23 | <l10nupdate@tin> | scap sync-l10n completed (1.29.0-wmf.2) (duration: 04m 56s) | [production] | 
            
  | 01:45 | <mutante> | gerrit now has higher "packedGitLimit" of 2g, goal is to reduce Gerrit slowdowns | [production] | 
            
  | 01:39 | <mutante> | gerrit restarting for config change 317322 (T148478) | [production] | 
            
  | 01:04 | <godog> | revert swift ring change for ms-be1027 | [production] | 
            
  | 00:23 | <godog> | swift eqiad-prod: ms-be1027 to weight 1000 - T136631 | [production] | 
            
  | 00:18 | <catrope@tin> | Synchronized wmf-config/InitialiseSettings.php: Remove registered trademark symbol from officewiki footer (T95007) (duration: 00m 48s) | [production] | 
            
  | 00:15 | <catrope@tin> | Synchronized wmf-config/InitialiseSettings.php: Enable {{NOINDEX}} as a noindex template on enwiki (2/2) (T149538) (duration: 00m 47s) | [production] | 
            
  | 00:13 | <catrope@tin> | Synchronized wmf-config/CommonSettings.php: Enable {{NOINDEX}} as a noindex template on enwiki (1/2) (T149538) (duration: 00m 49s) | [production] | 
            
  
    | 2016-11-10
      
      § | 
    
  | 23:10 | <mutante> | mw1185 - service hhvm restart | [production] | 
            
  | 23:00 | <maxsem@tin> | Finished scap: https://gerrit.wikimedia.org/r/#/c/320864/ (duration: 22m 42s) | [production] | 
            
  | 22:38 | <maxsem@tin> | Started scap: https://gerrit.wikimedia.org/r/#/c/320864/ | [production] | 
            
  | 22:35 | <maxsem@tin> | scap sync-l10n completed (1.29.0-wmf.2) (duration: 01m 01s) | [production] | 
            
  | 21:15 | <papaul> | kafka2003 - signing puppet certs, salt-key, initial run | [production] | 
            
  | 21:09 | <twentyafterfour@tin> | rebuilt wikiversions.php and synchronized wikiversions files: all wikis to 1.29.0-wmf.2 | [production] | 
            
  | 21:08 | <twentyafterfour@tin> | Synchronized php-1.29.0-wmf.2/extensions/ORES/includes/ApiHooks.php: deploy I86e97b05b56b90d956616ef16e8aa86d96403b8c (duration: 00m 47s) | [production] | 
            
  | 20:54 | <twentyafterfour@tin> | rebuilt wikiversions.php and synchronized wikiversions files: Revert "all wikis to 1.29.0-wmf.2" (Fatal errors spike) | [production] | 
            
  | 20:49 | <twentyafterfour@tin> | rebuilt wikiversions.php and synchronized wikiversions files: all wikis to 1.29.0-wmf.2 | [production] | 
            
  | 20:33 | <mutante> | neon - deactivated puppet node, scheduled icinga downtime, shutdown server permanently (T125023) | [production] | 
            
  | 20:32 | <mutante> | neon - shutdown -h now (scheduled 3 days downtime, nothing that looked worth saving in homes) | [production] | 
            
  | 20:28 | <mutante> | neon - deactivate puppet node | [production] | 
            
  | 20:25 | <twentyafterfour@tin> | Synchronized wmf-config/InitialiseSettings-labs.php: sync LABS: enable mapframe everywhere (I35709ed2903b28a2d4d6e8528ac1fcf361483e76) (duration: 00m 50s) | [production] | 
            
  | 19:37 | <dcausse> | elastic@eqiad: reindexing commonswiki (logs in terbium.eqiad.wmnet:~dcausse/commons_reindex/cirrus_log) - T150232 | [production] | 
            
  | 19:33 | <bblack> | cache_*: restarting nginx for libssl update (seamless) | [production] | 
            
  | 19:31 | <bblack> | upgrading libssl1.1 to 1.1.0c on other misc hosts... | [production] | 
            
  | 19:30 | <Pchelolo> | update RESTBase to 6bfa0f75f | [production] | 
            
  | 19:24 | <Pchelolo> | update RESTBase to 6bfa0f75f - canary on restbase1007 | [production] | 
            
  | 19:22 | <bblack> | upgrading openssl to 1.1.0c on cache_* | [production] | 
            
  | 19:19 | <Pchelolo> | update RESTBase to 6bfa0f75f - staging | [production] | 
            
  | 19:16 | <thcipriani@tin> | Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:320552|Add PageViewInfo log channel (T129602)]] (duration: 00m 49s) | [production] | 
            
  | 19:09 | <thcipriani@tin> | Synchronized wmf-config/InitialiseSettings-labs.php: SWAT: [[gerrit:320450|Make beta PageViewInfo use the production pageview API (T129602)]] (labs-only-change) (duration: 00m 48s) | [production] |