| 2020-04-30
      
      § | 
    
  | 05:15 | <marostegui@cumin1001> | dbctl commit (dc=all): 'Repool db1099:3318', diff saved to https://phabricator.wikimedia.org/P11091 and previous config saved to /var/cache/conftool/dbconfig/20200430-051506-marostegui.json | [production] | 
            
  | 05:13 | <marostegui@cumin1001> | dbctl commit (dc=all): 'Depool db1099:3318', diff saved to https://phabricator.wikimedia.org/P11090 and previous config saved to /var/cache/conftool/dbconfig/20200430-051329-marostegui.json | [production] | 
            
  | 05:02 | <marostegui> | Restart x1 master finished - T250701 | [production] | 
            
  | 05:00 | <marostegui> | Restart x1 master (db1120) - T250701 | [production] | 
            
  | 04:52 | <marostegui@cumin1001> | dbctl commit (dc=all): 'Depool db1084', diff saved to https://phabricator.wikimedia.org/P11089 and previous config saved to /var/cache/conftool/dbconfig/20200430-045159-marostegui.json | [production] | 
            
  | 04:38 | <marostegui@cumin1001> | dbctl commit (dc=all): 'Repool db1097:3314', diff saved to https://phabricator.wikimedia.org/P11088 and previous config saved to /var/cache/conftool/dbconfig/20200430-043803-marostegui.json | [production] | 
            
  | 00:26 | <twentyafterfour> | phabricator update finished | [production] | 
            
  | 00:15 | <twentyafterfour> | deploying phabricator update: https://phabricator.wikimedia.org/project/view/4620/ | [production] | 
            
  | 00:11 | <eileen> | process-control config revision is 1f31dd21c5 | [production] | 
            
  
    | 2020-04-29
      
      § | 
    
  | 23:55 | <jforrester@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: T91649 Drop Sentry, Part II: Stop configuring it for production or Beta Cluster (duration: 01m 05s) | [production] | 
            
  | 23:51 | <jforrester@deploy1001> | Synchronized wmf-config/CommonSettings.php: T91649 Drop Sentry, Part I: Stop loading it anywhere (duration: 01m 05s) | [production] | 
            
  | 23:37 | <catrope@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: Set Growth Screener survey sample rate to 0.1% and limit to anons only (T248421) (duration: 01m 05s) | [production] | 
            
  | 23:26 | <RoanKattouw> | Ran updateArticleCount.php on trwikisource | [production] | 
            
  | 23:22 | <catrope@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: Set $wgArticleCount to any on trwikisource (duration: 01m 06s) | [production] | 
            
  | 21:59 | <bstorm_> | upgrading RAID firmware on labsdb1011 T249188 | [production] | 
            
  | 21:34 | <volker-e@deploy1001> | Finished deploy [design/style-guide@c4956c3]: Deploy design/style-guide:  (duration: 00m 08s) | [production] | 
            
  | 21:34 | <volker-e@deploy1001> | Started deploy [design/style-guide@c4956c3]: Deploy design/style-guide: | [production] | 
            
  | 21:22 | <jforrester@deploy1001> | Synchronized php-1.35.0-wmf.28/extensions/GlobalBlocking/includes/api/ApiQueryGlobalBlocks.php: T251430 Unconditionally select gb_timestamp (duration: 01m 06s) | [production] | 
            
  | 21:19 | <jforrester@deploy1001> | Synchronized php-1.35.0-wmf.30/extensions/GlobalBlocking/includes/api/ApiQueryGlobalBlocks.php: T251430 Unconditionally select gb_timestamp (duration: 01m 05s) | [production] | 
            
  | 21:17 | <jforrester@deploy1001> | Synchronized php-1.35.0-wmf.30/extensions/Quiz/includes/Quiz.php: Don't crash if quiz attempts to include a bad title T251409 (duration: 01m 06s) | [production] | 
            
  | 20:11 | <andrew@cumin1001> | END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) | [production] | 
            
  | 20:09 | <andrew@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 19:46 | <jforrester@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: (doc-only) Fix Phabricator task reference for jvwiki logo (duration: 01m 05s) | [production] | 
            
  | 19:34 | <addshore> | repool wdqs2008 | [production] | 
            
  | 19:02 | <addshore> | depooling and stopping the updater on wdqs2008 for some query tests (wdqs-internal) | [production] | 
            
  | 17:56 | <joal@deploy1001> | Finished deploy [analytics/refinery@6460d05] (thin): Regular analytics weekly train THIN [6460d05] (duration: 00m 08s) | [production] | 
            
  | 17:56 | <joal@deploy1001> | Started deploy [analytics/refinery@6460d05] (thin): Regular analytics weekly train THIN [6460d05] | [production] | 
            
  | 17:54 | <elukey@cumin1001> | END (PASS) - Cookbook sre.presto.roll-restart-workers (exit_code=0) | [production] | 
            
  | 17:44 | <elukey@cumin1001> | START - Cookbook sre.presto.roll-restart-workers | [production] | 
            
  | 17:24 | <joal@deploy1001> | Finished deploy [analytics/refinery@6460d05]: Regular analytics weekly train [6460d05] (duration: 77m 08s) | [production] | 
            
  | 16:55 | <jforrester@deploy1001> | Synchronized php-1.35.0-wmf.30/extensions/MachineVision/src/Hooks.php: Fix hook handling for  hook T251408 (duration: 01m 05s) | [production] | 
            
  | 16:54 | <jforrester@deploy1001> | sync-file aborted: Fix hook handling for  hook T251408 (duration: 00m 02s) | [production] | 
            
  | 16:53 | <jforrester@deploy1001> | Synchronized php-1.35.0-wmf.30/includes/EditPage.php: EditPage::showHeader - only warn editing an old revision if it exists T251404 (duration: 01m 06s) | [production] | 
            
  | 16:45 | <hnowlan@deploy1001> | helmfile [STAGING] Ran 'sync' command on namespace 'changeprop' for release 'staging' . | [production] | 
            
  | 16:07 | <joal@deploy1001> | Started deploy [analytics/refinery@6460d05]: Regular analytics weekly train [6460d05] | [production] | 
            
  | 16:05 | <joal@deploy1001> | Finished deploy [analytics/aqs/deploy@c87c8e2]: Analytics regular weekly deploy (duration: 06m 59s) | [production] | 
            
  | 15:58 | <joal@deploy1001> | Started deploy [analytics/aqs/deploy@c87c8e2]: Analytics regular weekly deploy | [production] | 
            
  | 15:12 | <kormat@cumin1001> | dbctl commit (dc=all): 'Repooling db2087 in s6 and s7 after reimaging T250666', diff saved to https://phabricator.wikimedia.org/P11085 and previous config saved to /var/cache/conftool/dbconfig/20200429-151219-kormat.json | [production] | 
            
  | 14:56 | <sukhe> | upload cescout 0.1.2-1 to apt.wm.o (buster) | [production] | 
            
  | 14:49 | <mdholloway> | re-ran extension/MachineVision/maintenance/withholdImages.php on commonswiki | [production] | 
            
  | 14:39 | <mholloway-shell@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: MachineVision: Update image withholding term list (duration: 01m 06s) | [production] | 
            
  | 13:05 | <liw@deploy1001> | Synchronized php: group1 wikis to 1.35.0-wmf.30 (duration: 01m 04s) | [production] | 
            
  | 13:04 | <liw@deploy1001> | rebuilt and synchronized wikiversions files: group1 wikis to 1.35.0-wmf.30 | [production] | 
            
  | 12:50 | <kormat@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | [production] | 
            
  | 12:47 | <kormat@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 12:26 | <kormat@cumin1001> | dbctl commit (dc=all): 'Depool db2087 for reimaging T250666', diff saved to https://phabricator.wikimedia.org/P11081 and previous config saved to /var/cache/conftool/dbconfig/20200429-122602-kormat.json | [production] | 
            
  | 11:58 | <mutante> | running puppet on cp-ats - switching backends of wikiworkshop.org | [production] | 
            
  | 11:05 | <hoo@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: Add new properties to wmgWBRepoPreferredPageImagesProperties (T249811) (duration: 01m 18s) | [production] | 
            
  | 10:00 | <kormat> | reimaging db2087 to buster T250666 | [production] | 
            
  | 09:56 | <marostegui@cumin1001> | dbctl commit (dc=all): 'Depool db1097:3314', diff saved to https://phabricator.wikimedia.org/P11080 and previous config saved to /var/cache/conftool/dbconfig/20200429-095629-marostegui.json | [production] |