| 2017-05-15
      
      § | 
    
  | 11:09 | <Amir1_> | cleaning up ores_classification has finished 18M rows deleted, current number of rows 38,937,217  (T159753) | [production] | 
            
  | 10:36 | <moritzm> | rebooting mw2224-mw2242 for update to Linux 4.9 | [production] | 
            
  | 10:18 | <moritzm> | installing batik security updates on trusty | [production] | 
            
  | 10:14 | <moritzm> | installing fop security updates on trusty | [production] | 
            
  | 09:34 | <moritzm> | installing bind security updates (we're using client-side libs/tools only) | [production] | 
            
  | 09:10 | <godog> | swift codfw-prod: more ms-be2001/ms-be2012 decom - T162785 | [production] | 
            
  | 08:29 | <godog> | swift eqiad-prod: ms-be1028/ms-be1039 object weight 3000 - T160640 | [production] | 
            
  | 08:26 | <moritzm> | installing rtmpdump security updates on jessie | [production] | 
            
  | 08:17 | <Amir1_> | start of cleaning up ores_classification table | [production] | 
            
  | 02:27 | <l10nupdate@tin> | ResourceLoader cache refresh completed at Mon May 15 02:27:02 UTC 2017 (duration 5m 59s) | [production] | 
            
  | 02:21 | <l10nupdate@tin> | scap sync-l10n completed (1.30.0-wmf.1) (duration: 07m 42s) | [production] | 
            
  | 01:25 | <bblack> | depooled cp1053 from all services (possible hardware issues) | [production] | 
            
  
    | 2017-05-12
      
      § | 
    
  | 20:55 | <demon@tin> | Synchronized wmf-config/InitialiseSettings.php: Touch (duration: 00m 39s) | [production] | 
            
  | 20:49 | <demon@tin> | Synchronized wmf-config/: Swapping DynamicSidebar to normal extension registration (duration: 00m 19s) | [production] | 
            
  | 19:20 | <thcipriani@tin> | Synchronized php-1.30.0-wmf.1/extensions/TextExtracts/includes/ApiQueryExtracts.php: [[gerrit:353593|API: Change memcache key to clear cache]] T165161 (duration: 00m 39s) | [production] | 
            
  | 19:02 | <thcipriani@tin> | Synchronized wmf-config/CommonSettings.php: [[gerrit:353597|Add RejectParserCacheValue handler for mw-parser-output]] T165161 (duration: 00m 40s) | [production] | 
            
  | 18:47 | <bblack> | starting spaced-out ~4h run of "run-no-puppet varnish-frontend-restart" on cache_upload+cache_text to re-set transient storage levels (in screen on neodymium) | [production] | 
            
  | 18:10 | <thcipriani@tin> | Finished scap: [[gerrit:353584|Revert "Wrap parser output in <div class="mw-parser-output">"]] 4/4 (duration: 19m 13s) | [production] | 
            
  | 17:51 | <thcipriani@tin> | Started scap: [[gerrit:353584|Revert "Wrap parser output in <div class="mw-parser-output">"]] 4/4 | [production] | 
            
  | 17:51 | <thcipriani@tin> | Synchronized php-1.30.0-wmf.1/includes/api/ApiParse.php: [[gerrit:353584|Revert "Wrap parser output in <div class="mw-parser-output">"]] 3/4 (duration: 00m 42s) | [production] | 
            
  | 17:50 | <thcipriani@tin> | Synchronized php-1.30.0-wmf.1/includes/cache/MessageCache.php: [[gerrit:353584|Revert "Wrap parser output in <div class="mw-parser-output">"]] 2/4 (duration: 00m 39s) | [production] | 
            
  | 17:49 | <thcipriani@tin> | Synchronized php-1.30.0-wmf.1/includes/parser/Parser.php: [[gerrit:353584|Revert "Wrap parser output in <div class="mw-parser-output">"]] 1/4 (duration: 00m 39s) | [production] | 
            
  | 17:38 | <ema> | cp4010: upgrade varnish back to 4.1.6-1wm1, transient storage issues are unrelated | [production] | 
            
  | 17:33 | <krinkle@tin> | Synchronized php-1.30.0-wmf.1/includes/resourceloader/ResourceLoaderClientHtml.php: (no justification provided) (duration: 00m 40s) | [production] | 
            
  | 16:53 | <moritzm> | powercycling mw1294 (machine unacessible/locked up) | [production] | 
            
  | 16:23 | <moritzm> | repooled mw1172 after scap pull (was down with hardware error) | [production] | 
            
  | 14:10 | <moritzm> | rebooting mw2163-mw2179 for update to Linux 4.9 | [production] | 
            
  | 13:47 | <moritzm> | rebooting mw2110-mw2117 for update to Linux 4.9 | [production] | 
            
  | 13:06 | <moritzm> | repooled mw2098 (was down with hardware error) | [production] | 
            
  | 12:53 | <moritzm> | downgrading mw1161 (job runner) to HHVM 3.12, some known instabilities and fix for one HHVM 3.18 will likely be available next week, so going the conversative way over the weekend | [production] | 
            
  | 11:35 | <gehel> | cleaning old elasticsearch and logstash logs on logstash cluster | [production] | 
            
  | 10:38 | <_joe_> | moved hpssacli.tar.gz to /root on puppetmaster1001 | [production] | 
            
  | 09:59 | <hashar@tin> | Synchronized php-1.30.0-wmf.1/extensions/MobileFrontend: Correctly handle the mw-parser-output wrapper - T164733 (duration: 00m 43s) | [production] | 
            
  | 09:02 | <akosiaris> | move planet2001 to ganeti nodegroup row_A | [production] | 
            
  | 08:58 | <marostegui> | Rename semantic tables before dropping them on wikitech hosts (silver and labtestweb2001) - T164887 | [production] | 
            
  | 06:05 | <marostegui> | Deploy alter table on s2 (revision table) db2063 - https://phabricator.wikimedia.org/T162611 | [production] | 
            
  | 06:05 | <marostegui@tin> | Synchronized wmf-config/db-codfw.php: Repool db2064, depool db2063 - T162611 (duration: 00m 39s) | [production] | 
            
  | 05:53 | <marostegui> | Stop MySQL dbstore2001 for testing - T165033 | [production] | 
            
  | 02:30 | <l10nupdate@tin> | ResourceLoader cache refresh completed at Fri May 12 02:30:11 UTC 2017 (duration 6m 16s) | [production] | 
            
  | 02:23 | <l10nupdate@tin> | scap sync-l10n completed (1.30.0-wmf.1) (duration: 06m 49s) | [production] |