| 2019-08-28
      
      § | 
    
  | 08:10 | <vgutierrez> | uploaded trafficserver-8.0.5-1wm4 to apt.wikimedia.org (stretch) - T231287 | [production] | 
            
  | 08:08 | <marostegui@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | [production] | 
            
  | 08:06 | <marostegui@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 06:41 | <marostegui> | Upgrade mysql on s7 codfw hosts: db2054, db2061, db2068, db2077 - T230106 | [production] | 
            
  | 06:35 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Remove db2053 from config T231407 (duration: 00m 53s) | [production] | 
            
  | 06:34 | <marostegui@deploy1001> | Synchronized wmf-config/db-codfw.php: Remove db2053 from config T231407 (duration: 00m 55s) | [production] | 
            
  | 05:54 | <marostegui> | Remove old rows from pc1010 - T210725 | [production] | 
            
  | 05:19 | <marostegui> | Start dropping neodymium grants across all the databases, parsercache, es, dbstore... T229796 | [production] | 
            
  | 05:03 | <marostegui@deploy1001> | Synchronized wmf-config/db-eqiad.php: Repool pc1009 after optimize T210725 (duration: 00m 54s) | [production] | 
            
  
    | 2019-08-27
      
      § | 
    
  | 23:45 | <Urbanecm> | Evening SWAT done | [production] | 
            
  | 23:44 | <urbanecm@deploy1001> | Synchronized php-1.34.0-wmf.20/skins/MinervaNeue/: SWAT: 4d04797: Restore contributions icon to non-AMC menu (T231363) (duration: 00m 54s) | [production] | 
            
  | 23:39 | <urbanecm@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT: 1422870: [sqwikiquote] Enable WikiLove and SandboxLink (T230390) (duration: 00m 54s) | [production] | 
            
  | 23:36 | <Urbanecm> | Run mwscript extensions/WikimediaMaintenance/createExtensionTables.php sqwikiquote wikilove (T230390) | [production] | 
            
  | 23:30 | <urbanecm@deploy1001> | Synchronized php-1.34.0-wmf.20/extensions/MobileFrontend/resources/dist/: SWAT: a109b25: Build assets reflecting edit change (duration: 00m 55s) | [production] | 
            
  | 23:22 | <urbanecm@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT: 3704bb7: Enable partial blocks on ruwiki (T231298) (duration: 00m 54s) | [production] | 
            
  | 23:16 | <urbanecm@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT: 1687ec9: Whitelist *.wikimedia.cz in wgCopyUploadsDomains for commonswiki (T231247) (duration: 00m 54s) | [production] | 
            
  | 23:02 | <eileen> | civicrm revision is 049c9666b6, config revision is 24aed9745e | [production] | 
            
  | 22:40 | <eileen> | civicrm revision changed from 517e6ee4e0 to 049c9666b6, config revision is 24aed9745e | [production] | 
            
  | 22:29 | <jforrester@deploy1001> | Synchronized php-1.34.0-wmf.20/extensions/VisualEditor/lib/ve/src/ce/nodes/ve.ce.GeneratedContentNode.js: T231381 Follow-up I196f5bd88: Fix typo (set node=this) (duration: 00m 57s) | [production] | 
            
  | 21:51 | <bblack@cumin1001> | conftool action : set/pooled=no; selector: name=cp1075.eqiad.wmnet,service=ats-be | [production] | 
            
  | 21:11 | <XioNoX> | disable both sides of the reline link between knams and esams - T230448 | [production] | 
            
  | 20:36 | <jforrester@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: Remove unusued wgEnableBlockNoticeStats setting (duration: 00m 54s) | [production] | 
            
  | 19:08 | <gehel> | starting deployment of Apache config for lexemes / SDoC - T222321 | [production] | 
            
  | 18:59 | <jforrester@deploy1001> | Synchronized php-1.34.0-wmf.20/includes/gallery/ImageGalleryBase.php: T231340 T231353 BadFileLookup::isBadFile() expects null, not false for galleries (duration: 00m 53s) | [production] | 
            
  | 18:58 | <jforrester@deploy1001> | Synchronized php-1.34.0-wmf.20/includes/api/ApiQueryImageInfo.php: T231340 T231353 BadFileLookup::isBadFile() expects null, not false for the API (duration: 00m 53s) | [production] | 
            
  | 18:56 | <jforrester@deploy1001> | Synchronized php-1.34.0-wmf.20/skins/MinervaNeue/skin.json: T231358 Fix userSandbox image path (duration: 00m 53s) | [production] | 
            
  | 17:46 | <mholloway-shell@deploy1001> | Finished deploy [mobileapps/deploy@1869f79]: Fix definition endpoint TypeError (T230503) (duration: 04m 39s) | [production] | 
            
  | 17:42 | <krinkle@deploy1001> | Synchronized php-1.34.0-wmf.20/includes/password/PasswordPolicyChecks.php: 098755622f7 (duration: 00m 54s) | [production] | 
            
  | 17:41 | <mholloway-shell@deploy1001> | Started deploy [mobileapps/deploy@1869f79]: Fix definition endpoint TypeError (T230503) | [production] | 
            
  | 17:04 | <krinkle@deploy1001> | Synchronized php-1.34.0-wmf.20/extensions/Echo: 34084279089f (duration: 00m 55s) | [production] | 
            
  | 16:38 | <krinkle@deploy1001> | Synchronized php-1.34.0-wmf.20/extensions/TwoColConflict/extension.json: d6b5d441b, T229791 (duration: 00m 55s) | [production] | 
            
  | 15:41 | <James_F> | That was T231279 Set `$wgRelatedArticlesDescriptionSource` to `wikidata` | [production] | 
            
  | 15:41 | <jforrester@deploy1001> | Synchronized wmf-config/CommonSettings.php: T231279 Set to (duration: 00m 54s) | [production] | 
            
  | 14:52 | <_joe_> | running scap pull on mw1280 | [production] | 
            
  | 14:50 | <oblivian@puppetmaster1001> | conftool action : set/pooled=inactive; selector: name=mw1280.eqiad.wmnet | [production] | 
            
  | 14:49 | <_joe_> | powercycling mw1280 | [production] | 
            
  | 14:48 | <bblack> | deploying anycast recdns resolv.conf setting to all codfw - T228190 | [production] | 
            
  | 14:45 | <zfilipin@deploy1001> | rebuilt and synchronized wikiversions files: group0 to 1.34.0-wmf.20 | [production] | 
            
  | 14:39 | <ema> | cp1081: restart crashed services varnishkafka-{statsv,webrequest}.service | [production] | 
            
  | 14:38 | <vgutierrez> | depool cp5001 - T231287 | [production] | 
            
  | 14:33 | <zfilipin@deploy1001> | Finished scap: testwiki to php-1.34.0-wmf.20 and rebuild l10n cache (duration: 30m 48s) | [production] | 
            
  | 14:03 | <zfilipin@deploy1001> | Started scap: testwiki to php-1.34.0-wmf.20 and rebuild l10n cache | [production] | 
            
  | 13:52 | <zfilipin@deploy1001> | Pruned MediaWiki: 1.34.0-wmf.16 [keeping static files] (duration: 01m 35s) | [production] | 
            
  | 13:47 | <zfilipin@deploy1001> | Pruned MediaWiki: 1.34.0-wmf.15 (duration: 06m 44s) | [production] | 
            
  | 13:43 | <vgutierrez> | repool cp5001 - T231287 | [production] | 
            
  | 12:36 | <ema> | pool cp1075 w/ ATS backend (for real) T228629 | [production] | 
            
  | 12:29 | <marostegui> | Rename table filejournal on enwiki on db1089 - T51195 | [production] | 
            
  | 12:17 | <ema> | depool cp1075, confd is not watching the key "ats-be" | [production] | 
            
  | 12:15 | <ema> | pool cp1075 w/ ATS backend T228629 | [production] | 
            
  | 11:54 | <mutante> | miscweb1001 - a2dismod mpm_event ; a2enmod php7.0 ; systemctl restart apache2 (T224247, T196968) please also see https://gerrit.wikimedia.org/r/c/operations/puppet/+/451206 | [production] |