| 2016-06-09
      
      § | 
    
  | 17:43 | <urandom> | Restarting Cassandra on xenon.eqiad.wmnet (use exponentially decaying resevoirs for metrics histograms) : T126629 | [production] | 
            
  | 17:19 | <mobrovac> | change-prop deploying ecfda93f09d | [production] | 
            
  | 17:09 | <ejegg> | updated payments-wiki from 3dcf58e3b4e1d02ad4f1874a3e87e55b7e169bfe to 053aaa259382c94aa59e4d0da7317fcafab635cd | [production] | 
            
  | 15:31 | <elukey> | added topic override retention.bytes=536870912000 to Kafka webrequest_text (T136690) | [production] | 
            
  | 15:22 | <hashar> | Cleaning git-daemon on gallium (was used by zuul-merger) T137418 | [production] | 
            
  | 15:19 | <aude@tin> | Synchronized wmf-config/InitialiseSettings.php: Add *.nara.gov to wgCopyUploadDomains (duration: 00m 40s) | [production] | 
            
  | 14:47 | <mobrovac> | change-prop stopped on scb1002 | [production] | 
            
  | 14:38 | <elukey> | Tested temp setting retention.bytes=2G for Analytics kafka topic webrequest_misc | [production] | 
            
  | 14:37 | <hashar> | Removing zuul-merger from gallium | [production] | 
            
  | 14:33 | <hashar> | stopped / disabled zuul-merger on gallium T137418 | [production] | 
            
  | 14:12 | <mobrovac> | change-prop restarting on scb1001 for update | [production] | 
            
  | 14:07 | <urandom> | Re-enabling puppet on xenon.eqiad.wmnet, forcing a run, and restarting Cassandra : T137419 | [production] | 
            
  | 13:52 | <mobrovac> | change-prop restarting on scb1002 for update | [production] | 
            
  | 13:45 | <mobrovac> | change-prop deploying 2161403c | [production] | 
            
  | 13:26 | <urandom> | Restarting Cassandra on xenon.eqiad.wmnet to apply 2G file cache : T137419 | [production] | 
            
  | 12:50 | <urandom> | Restarting Cassandra on xenon.eqiad.wmnet : T126629 | [production] | 
            
  | 12:33 | <tgr@tin> | Synchronized php-1.28.0-wmf.5/extensions/LdapAuthentication/LdapPrimaryAuthenticationProvider.php: deploy [[gerrit:293459]] to fix wikitech API login / morebots (T137377) (duration: 00m 47s) | [production] | 
            
  | 12:19 | <tgr> | deploying [[gerrit:293459]] to fix morebots (T137377) | [production] | 
            
  | 12:11 | <urandom> | Restarting Cassandra on xenon.eqiad.wmnet : T126629 | [production] | 
            
  | 12:06 | <urandom> | Temporarily disabling puppet on xenon.eqiad.wmnet to test settings : T126629 | [production] | 
            
  | 11:33 | <Amir1> | manually restarting ores-uwsgi and celery-ores-worker in scb100[12] | [production] | 
            
  | 10:51 | <urandom> | Restarting Cassandra on {cerium,praseodymium}.eqiad.wmnet (RESTBase staging) : T126629 | [production] | 
            
  | 09:16 | <gehel> | lowering disk high watermark to rebalance disk usage on elasticsearch eqiad cluster | [production] | 
            
  | 09:05 | <Amir1> | restarting uwsgi-ores celery-ores-worker in scb1001 and scb1002 | [production] | 
            
  | 08:54 | <moritzm> | installing libtasn security updates | [production] | 
            
  | 08:38 | <moritzm> | rolling restart of app server canaries for libtasn security update | [production] | 
            
  | 07:22 | <moritzm> | removed /var/log/logstash/logstash.1 on logstash1001, logspam (similar to the what is described in https://github.com/logstash-plugins/logstash-output-elasticsearch/issues/144) depleted the space on the root partition | [production] | 
            
  | 02:55 | <l10nupdate@tin> | ResourceLoader cache refresh completed at Thu Jun  9 02:55:20 UTC 2016 (duration 6m 19s) | [production] | 
            
  | 02:53 | <mutante> | ms-be2012 ran out of disk due to huge syslog, deleted log, restarted rsyslogd | [production] | 
            
  | 02:49 | <mwdeploy@tin> | scap sync-l10n completed (1.28.0-wmf.5) (duration: 11m 02s) | [production] | 
            
  | 02:26 | <mwdeploy@tin> | scap sync-l10n completed (1.28.0-wmf.4) (duration: 11m 00s) | [production] | 
            
  | 00:16 | <ejegg> | updated payments from de11bfe2273d0bcaa0e713389b2d91e8b3567a1d to c6ea2ca699aa78a90f64c77d860f69421e9a7782 | [production] | 
            
  | 00:15 | <mutante> | restarted the log bot | [production] | 
            
  | 00:11 | <mutante> | restarted the log bot | [production] | 
            
  | 00:06 | <mutante> | meta T46791 | [production] | 
            
  | 00:03 | <twentyafterfour> | Preparing to deploy phabricator update. Tagged release/2016-06-08/1 | [production] | 
            
  
    | 2016-06-08
      
      § | 
    
  | 23:17 | <maxsem@tin> | Synchronized php-1.28.0-wmf.5/extensions/WikimediaEvents/: https://gerrit.wikimedia.org/r/#/c/293439/ (duration: 00m 23s) | [production] | 
            
  | 23:15 | <maxsem@tin> | Synchronized wmf-config/InitialiseSettings.php: https://gerrit.wikimedia.org/r/293438 (duration: 00m 25s) | [production] | 
            
  | 23:07 | <maxsem@tin> | Synchronized php-1.28.0-wmf.4/extensions/LiquidThreads/: https://gerrit.wikimedia.org/r/#/c/293247/ (duration: 00m 26s) | [production] | 
            
  | 23:05 | <maxsem@tin> | Synchronized php-1.28.0-wmf.5/extensions/LiquidThreads/: https://gerrit.wikimedia.org/r/#/c/293247/ (duration: 00m 26s) | [production] | 
            
  | 22:51 | <hoo> | Re-started dumpwikidatattl on snapshot1003 | [production] | 
            
  | 22:44 | <tgr@tin> | Synchronized wmf-config/InitialiseSettings.php: enable AuthManager on group1 for reals T135504 (duration: 00m 25s) | [production] | 
            
  | 22:26 | <tgr@tin> | Synchronized wmf-config/InitialiseSettings.php: enable AuthManager on group1 T135504 (duration: 00m 23s) | [production] | 
            
  | 22:20 | <tgr@tin> | Synchronized php-1.28.0-wmf.5/extensions/OpenStackManager/: backport [[gerrit:293130]] for AuthManager deploy T135504 (duration: 00m 28s) | [production] | 
            
  | 22:05 | <ottomata> | starting kafka broker on kafka1012 after swapping disk and copying data directory | [production] | 
            
  | 22:01 | <krinkle@tin> | Synchronized wmf-config/CommonSettings.php: Bump wgResourceLoaderStorageVersion (T134368) (duration: 00m 28s) | [production] | 
            
  | 21:16 | <mobrovac> | change-prop reverting back to 96337cd540a2 | [production] | 
            
  | 21:11 | <thcipriani@tin> | rebuilt wikiversions.php and synchronized wikiversions files: group1 wikis to 1.28.0-wmf.5 | [production] | 
            
  | 21:04 | <thcipriani@tin> | Synchronized php-1.28.0-wmf.5/includes/specials/SpecialSearch.php: [[gerrit:293422|Add a visual clear to Special:Search input box and profile-tabs]] (duration: 00m 23s) | [production] | 
            
  | 20:57 | <thcipriani@tin> | Synchronized php-1.28.0-wmf.5/extensions/Renameuser/RenameuserSQL.php: [[gerrit:293383|Use master DB when touching the user to signal rename end]] (duration: 00m 22s) | [production] |