| 2020-04-14
      
      § | 
    
  | 10:15 | <XioNoX> | update prefix-list LVS-service-ips to add missing prefixes | [production] | 
            
  | 09:49 | <XioNoX> | re-order aggregate routes to standardize order | [production] | 
            
  | 09:48 | <XioNoX> | cleanup 2620:0:860::/46 and 208.80.152.0/22 aggregates from cr2-eqdfw - T246721 | [production] | 
            
  | 09:47 | <XioNoX> | cleanup 2620:0:860::/46 and 208.80.152.0/22 aggregates from cr2-eqord - T246721 | [production] | 
            
  | 09:37 | <XioNoX> | cleanup 2620:0:860::/46 and 208.80.152.0/22 aggregates from cr1/2-codfw - T246721 | [production] | 
            
  | 09:16 | <XioNoX> | add missing `routing-options rib inet6.0 aggregate defaults discard` where missing (cr3-knams, cr3-esams, cr2-eqord, cr2-eqdfw, cr1/2-eqiad/codfw) | [production] | 
            
  | 09:13 | <godog> | add mwilliams to 'wmf' ldap group - T249844 | [production] | 
            
  | 09:08 | <marostegui> | Add kormat to ops and wmf ldap groups - T250134 | [production] | 
            
  | 08:49 | <elukey> | restart elastic-chi on cloudelastic1001 with -XX:NewSize=10G - T231517 | [production] | 
            
  | 08:05 | <sonic2020> | Upgrading to ATS 8.0.7, stage at 50% completed. | [production] | 
            
  | 08:04 | <sonic2020> | joaquin: Upgrading to ATS 8.0.7, stage at 0% completed. | [production] | 
            
  | 07:33 | <elukey> | apply CMS GC settings to chi on cloudelastic1001 - T231517 | [production] | 
            
  | 05:30 | <vgutierrez> | rolling upgrade to ats 8.0.7-rc0-1wm1 in esams and eqiad | [production] | 
            
  | 05:01 | <marostegui@deploy1001> | Synchronized wmf-config/db-codfw.php: Repool pc2008 after upgrade (duration: 01m 00s) | [production] | 
            
  
    | 2020-04-13
      
      § | 
    
  | 23:24 | <mdholloway> | re-ran extensions/MachineVision/maintenance/withholdImages.php on commonswiki | [production] | 
            
  | 23:14 | <mholloway-shell@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: MachineVision withholding list additions (T249939) (duration: 00m 59s) | [production] | 
            
  | 22:41 | <cdanis> | repool codfw | [production] | 
            
  | 22:35 | <ebernhardson> | restart elasticsearch_6@production-search-psi-eqiad on elastic1052 for excessive old gc over last few hours | [production] | 
            
  | 22:35 | <ebernhardson> | restart elasticsearch_6@production-search-psi-eqiad on elastic1052 | [production] | 
            
  | 22:08 | <cdanis> | depool codfw | [production] | 
            
  | 21:43 | <mdholloway> | ran extensions/MachineVision/maintenance/removeBlacklistedSuggestions.php on commonswiki (T249273) | [production] | 
            
  | 21:34 | <mdholloway> | ran extensions/MachineVision/maintenance/removeBlacklistedSuggestions.php on testcommonswiki | [production] | 
            
  | 21:32 | <mholloway-shell@deploy1001> | Synchronized php-1.35.0-wmf.27/extensions/MachineVision: Add script to apply blacklist to current labels (T249273) (duration: 00m 58s) | [production] | 
            
  | 20:49 | <mholloway-shell@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: MachineVision blocklist update (T249895) (duration: 00m 59s) | [production] | 
            
  | 19:56 | <mdholloway> | finished running extensions/MachineVision/maintenance/withholdImages.php on commonswiki (T249939) | [production] | 
            
  | 19:51 | <mdholloway> | running extensions/MachineVision/maintenance/withholdImages.php on commonswiki | [production] | 
            
  | 19:41 | <mdholloway> | ran extensions/MachineVision/maintenance/withholdImages.php on testcommonswiki | [production] | 
            
  | 19:37 | <mholloway-shell@deploy1001> | Synchronized php-1.35.0-wmf.27/extensions/MachineVision: Add support for WITHHOLD_ALL review state (T249939) (duration: 01m 23s) | [production] | 
            
  | 19:13 | <mholloway-shell@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: MachineVision: Add MachineVisionWithholdImageList config (T249939) (duration: 01m 03s) | [production] | 
            
  | 19:06 | <niedzielski> | Morning SWAT done | [production] | 
            
  | 19:02 | <niedzielski@deploy1001> | Synchronized php-1.35.0-wmf.27/skins/MinervaNeue: SWAT: [[gerrit:588405|Update the icon glyph (T249864)]] (duration: 01m 00s) | [production] | 
            
  | 18:49 | <niedzielski@deploy1001> | Synchronized php-1.35.0-wmf.27/extensions/TwoColConflict: SWAT: [[gerrit:588370|Fix double HTML escaping of "copytext" lines in the diff (T249986)]] (duration: 01m 01s) | [production] | 
            
  | 17:01 | <XioNoX> | sample before any other border-in terms in eqiad | [production] | 
            
  | 16:57 | <XioNoX> | sample before any other border-in terms in esams | [production] | 
            
  | 16:50 | <XioNoX> | sample before any other border-in terms in dfw | [production] | 
            
  | 16:46 | <XioNoX> | sample before any other border-in terms in ulsfo | [production] | 
            
  | 16:36 | <XioNoX> | sample before any other border-in terms in eqsin | [production] | 
            
  | 16:36 | <mholloway-shell@deploy1001> | helmfile [CODFW] Ran 'apply' command on namespace 'wikifeeds' for release 'production' . | [production] | 
            
  | 16:33 | <mholloway-shell@deploy1001> | helmfile [EQIAD] Ran 'apply' command on namespace 'wikifeeds' for release 'production' . | [production] | 
            
  | 16:31 | <XioNoX> | Sample all inbound v6 traffic on cr2-eqsin | [production] | 
            
  | 16:31 | <cmjohnson1> | replacing msw-c6-eqiad | [production] | 
            
  | 16:30 | <mholloway-shell@deploy1001> | helmfile [STAGING] Ran 'apply' command on namespace 'wikifeeds' for release 'staging' . | [production] | 
            
  | 15:56 | <marostegui> | Deploy schema change on s4 codfw master - T250067 | [production] | 
            
  | 12:12 | <vgutierrez> | rolling upgrade to ats 8.0.7-rc0-1wm1 in eqsin and codfw | [production] | 
            
  | 11:58 | <akosiaris@deploy1001> | helmfile [STAGING] Ran 'sync' command on namespace 'kube-system' for release 'coredns' . | [production] | 
            
  | 11:57 | <marostegui> | Deploy schema change on eqiad s8 hosts - T250062 | [production] | 
            
  | 11:53 | <akosiaris@deploy1001> | helmfile [STAGING] Ran 'apply' command on namespace 'kube-system' for release 'calico-policy-controller' . | [production] | 
            
  | 11:53 | <akosiaris@deploy1001> | helmfile [EQIAD] Ran 'apply' command on namespace 'kube-system' for release 'calico-policy-controller' . | [production] | 
            
  | 11:53 | <akosiaris@deploy1001> | helmfile [CODFW] Ran 'apply' command on namespace 'kube-system' for release 'calico-policy-controller' . | [production] | 
            
  | 11:53 | <marostegui> | Deploy schema change on codfw master (lag will appear on codfw) - T250062 | [production] |