| 2019-11-08
      
      § | 
    
  | 09:39 | <jynus@cumin1001> | dbctl commit (dc=all): 'Repool db1106 at 50%', diff saved to https://phabricator.wikimedia.org/P9566 and previous config saved to /var/cache/conftool/dbconfig/20191108-093958-jynus.json | [production] | 
            
  | 09:35 | <elukey@cumin1001> | END (PASS) - Cookbook sre.druid.roll-restart-workers (exit_code=0) | [production] | 
            
  | 09:29 | <jynus> | update and restart db2094 | [production] | 
            
  | 09:27 | <jynus@cumin1001> | dbctl commit (dc=all): 'Repool db1106 at 10%', diff saved to https://phabricator.wikimedia.org/P9565 and previous config saved to /var/cache/conftool/dbconfig/20191108-092735-jynus.json | [production] | 
            
  | 09:10 | <jynus> | update and restart db1106 | [production] | 
            
  | 09:08 | <moritzm> | installing Java security updates on kafka-jumbo | [production] | 
            
  | 09:07 | <jynus@cumin1001> | dbctl commit (dc=all): 'Depool db1106 fully', diff saved to https://phabricator.wikimedia.org/P9564 and previous config saved to /var/cache/conftool/dbconfig/20191108-090746-jynus.json | [production] | 
            
  | 09:05 | <elukey@cumin1001> | START - Cookbook sre.druid.roll-restart-workers | [production] | 
            
  | 09:04 | <jynus@cumin1001> | dbctl commit (dc=all): 'Depool db1106 at 10%', diff saved to https://phabricator.wikimedia.org/P9563 and previous config saved to /var/cache/conftool/dbconfig/20191108-090451-jynus.json | [production] | 
            
  | 09:00 | <jynus@cumin1001> | dbctl commit (dc=all): 'Depool db1106 at 50%', diff saved to https://phabricator.wikimedia.org/P9562 and previous config saved to /var/cache/conftool/dbconfig/20191108-090012-jynus.json | [production] | 
            
  | 08:57 | <elukey@cumin1001> | END (PASS) - Cookbook sre.druid.roll-restart-workers (exit_code=0) | [production] | 
            
  | 08:52 | <jynus> | stop and upgrade db1124 (may create temporary lag on wikireplicas) | [production] | 
            
  | 08:31 | <elukey@cumin1001> | START - Cookbook sre.druid.roll-restart-workers | [production] | 
            
  | 08:23 | <elukey> | restart kafka on kafka-jumbo1001 to test the new openjdk | [production] | 
            
  | 08:07 | <moritzm> | installing fribidi security updates on Buster | [production] | 
            
  | 03:03 | <vgutierrez> | Switch from nginx to ats-tls on cp3054 - T231627 | [production] | 
            
  | 02:42 | <vgutierrez> | Switch from nginx to ats-tls on cp3052 - T231627 | [production] | 
            
  | 01:23 | <reedy@deploy1001> | Synchronized php-1.35.0-wmf.5/extensions/GlobalBlocking/: Prevent some extra db queries (duration: 00m 53s) | [production] | 
            
  | 01:14 | <catrope@deploy1001> | Synchronized php-1.35.0-wmf.5/resources/: Use internationalized semicolon separators (T233649) (duration: 00m 53s) | [production] | 
            
  | 01:09 | <twentyafterfour@deploy1001> | Finished deploy [releng/phatality@11d4ad8]: deploying one more time, hopefully without killing elastic (duration: 03m 04s) | [production] | 
            
  | 01:06 | <twentyafterfour@deploy1001> | Started deploy [releng/phatality@11d4ad8]: deploying one more time, hopefully without killing elastic | [production] | 
            
  | 00:44 | <catrope@deploy1001> | Synchronized php-1.35.0-wmf.5/extensions/GrowthExperiments/modules/homepage/ext.growthExperiments.Homepage.Logging.js: Fix homepage instrumentation (T237600) (duration: 00m 52s) | [production] | 
            
  | 00:40 | <catrope@deploy1001> | Synchronized php-1.35.0-wmf.5/includes: Sync DiffEngine changes that were needed to unbreak CI (duration: 00m 55s) | [production] | 
            
  | 00:34 | <catrope@deploy1001> | Synchronized php-1.35.0-wmf.5/resources/: Semicolon should appear after log entries (T237500) (duration: 00m 53s) | [production] | 
            
  | 00:26 | <catrope@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: Fix remote API configs for GrowthExperiments (duration: 00m 51s) | [production] | 
            
  | 00:19 | <catrope@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: Enable suggested edits as hidden preference on arwiki, cswiki, kowiki, viwiki (T236968) (duration: 00m 53s) | [production] | 
            
  
    | 2019-11-07
      
      § | 
    
  | 23:49 | <foks> | removing one file for legal compliance | [production] | 
            
  | 23:47 | <twentyafterfour@deploy1001> | Finished deploy [releng/phatality@62e2870]: revert phatalaty again (duration: 03m 04s) | [production] | 
            
  | 23:44 | <shdubsh> | start elasticsearch on logstash1008 | [production] | 
            
  | 23:44 | <twentyafterfour@deploy1001> | Started deploy [releng/phatality@62e2870]: revert phatalaty again | [production] | 
            
  | 23:41 | <twentyafterfour@deploy1001> | Finished deploy [releng/phatality@11d4ad8]: one more time (duration: 03m 00s) | [production] | 
            
  | 23:38 | <twentyafterfour@deploy1001> | Started deploy [releng/phatality@11d4ad8]: one more time | [production] | 
            
  | 23:31 | <twentyafterfour@deploy1001> | Finished deploy [releng/phatality@11d4ad8]: trying again with a longer scap timeout (duration: 03m 02s) | [production] | 
            
  | 23:28 | <twentyafterfour@deploy1001> | Started deploy [releng/phatality@11d4ad8]: trying again with a longer scap timeout | [production] | 
            
  | 23:23 | <twentyafterfour@deploy1001> | Finished deploy [releng/phatality@62e2870]: revert to previous phatality plugin version (duration: 02m 55s) | [production] | 
            
  | 23:20 | <twentyafterfour@deploy1001> | Started deploy [releng/phatality@62e2870]: revert to previous phatality plugin version | [production] | 
            
  | 23:09 | <twentyafterfour@deploy1001> | Finished deploy [releng/phatality@11d4ad8]: (no justification provided) (duration: 00m 06s) | [production] | 
            
  | 23:09 | <twentyafterfour@deploy1001> | Started deploy [releng/phatality@11d4ad8]: (no justification provided) | [production] | 
            
  | 23:04 | <twentyafterfour@deploy1001> | Finished deploy [releng/phatality@11d4ad8]: (no justification provided) (duration: 06m 48s) | [production] | 
            
  | 23:00 | <XenoRyet> | updated payments-wiki from aac3d93f70 to bd907656fb | [production] | 
            
  | 22:57 | <twentyafterfour@deploy1001> | Started deploy [releng/phatality@11d4ad8]: (no justification provided) | [production] | 
            
  | 22:53 | <volker-e@deploy1001> | Finished deploy [design/style-guide@4abbc70]: Update responsive Illustrations styles changes (duration: 00m 05s) | [production] | 
            
  | 22:53 | <volker-e@deploy1001> | Started deploy [design/style-guide@4abbc70]: Update responsive Illustrations styles changes | [production] | 
            
  | 22:32 | <mholloway-shell@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: MachineVision: Remove annotation job delay (duration: 00m 53s) | [production] | 
            
  | 22:03 | <volker-e@deploy1001> | Finished deploy [design/style-guide@4abbc70]: Update to latest master with components overview additions (duration: 00m 06s) | [production] | 
            
  | 22:03 | <volker-e@deploy1001> | Started deploy [design/style-guide@4abbc70]: Update to latest master with components overview additions | [production] | 
            
  | 21:54 | <andrewbogott> | rebuilding labtestpuppetmaster2001 w/Stretch | [production] | 
            
  | 21:43 | <dzahn@cumin1001> | conftool action : set/pooled=yes; selector: name=wtp2020.codfw.wmnet | [production] | 
            
  | 21:28 | <mutante> | boron apt-get clean (saved 9G on /) (T237649) | [production] | 
            
  | 20:42 | <twentyafterfour@deploy1001> | rebuilt and synchronized wikiversions files: group2 wikis to 1.35.0-wmf.5  refs T233853 | [production] |