| 2019-06-28
      
      § | 
    
  | 21:19 | <otto@deploy1001> | Finished deploy [eventstreams/deploy@2af2719]: Manually blacklisting IP - T226808 (duration: 03m 07s) | [production] | 
            
  | 21:16 | <otto@deploy1001> | Started deploy [eventstreams/deploy@2af2719]: Manually blacklisting IP - T226808 | [production] | 
            
  | 20:09 | <jforrester@deploy1001> | Synchronized php-1.34.0-wmf.11/extensions/Wikibase/repo/RepoHooks.php: Make it possible for File pages to be moved on Commons again T224303 T226672 (duration: 00m 50s) | [production] | 
            
  | 19:49 | <jforrester@deploy1001> | Synchronized wmf-config/mobile.php: T221196 VE mobile A/B test part 2 (duration: 00m 49s) | [production] | 
            
  | 19:48 | <jforrester@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: T221196 VE mobile A/B test part 1 (duration: 00m 50s) | [production] | 
            
  | 19:05 | <joal@deploy1001> | Finished deploy [analytics/refinery@de8eb99]: Missing bit of regular analytics deploy (duration: 02m 08s) | [production] | 
            
  | 19:03 | <joal@deploy1001> | Started deploy [analytics/refinery@de8eb99]: Missing bit of regular analytics deploy | [production] | 
            
  | 18:51 | <joal@deploy1001> | Finished deploy [analytics/refinery@de8eb99]: Missing bit of regular analytics deploy (duration: 17m 47s) | [production] | 
            
  | 18:33 | <joal@deploy1001> | Started deploy [analytics/refinery@de8eb99]: Missing bit of regular analytics deploy | [production] | 
            
  | 18:14 | <joal@deploy1001> | Finished deploy [analytics/refinery@8d6fa30]: Late regular analytics weekly deploy - notebook1004 only (duration: 01m 03s) | [production] | 
            
  | 18:13 | <joal@deploy1001> | Started deploy [analytics/refinery@8d6fa30]: Late regular analytics weekly deploy - notebook1004 only | [production] | 
            
  | 18:12 | <elukey> | systemctl reset-failed kafka* units on kafka2001 (in decom phase) | [production] | 
            
  | 18:12 | <joal@deploy1001> | Finished deploy [analytics/refinery@8d6fa30]: Late regular analytics weekly deploy - notebook1003 only again (duration: 00m 26s) | [production] | 
            
  | 18:11 | <joal@deploy1001> | Started deploy [analytics/refinery@8d6fa30]: Late regular analytics weekly deploy - notebook1003 only again | [production] | 
            
  | 18:09 | <joal@deploy1001> | Finished deploy [analytics/refinery@8d6fa30]: Late regular analytics weekly deploy - notebook1003 only (duration: 00m 05s) | [production] | 
            
  | 18:09 | <joal@deploy1001> | Started deploy [analytics/refinery@8d6fa30]: Late regular analytics weekly deploy - notebook1003 only | [production] | 
            
  | 18:08 | <joal@deploy1001> | Finished deploy [analytics/refinery@8d6fa30]: Late regular analytics weekly deploy - notebook1003 only (duration: 00m 04s) | [production] | 
            
  | 18:08 | <joal@deploy1001> | Started deploy [analytics/refinery@8d6fa30]: Late regular analytics weekly deploy - notebook1003 only | [production] | 
            
  | 18:06 | <joal@deploy1001> | Finished deploy [analytics/refinery@8d6fa30]: Laste regular analytics weekly deploy (duration: 53m 35s) | [production] | 
            
  | 17:53 | <cdanis> | increasing nginx proxy_buffer_size / proxy_buffers 02d7bcaa | [production] | 
            
  | 17:36 | <ottomata> | restarting eventstreams on scb1001 with trace logging of X-Client-IP  for  T226808 | [production] | 
            
  | 17:13 | <joal@deploy1001> | Started deploy [analytics/refinery@8d6fa30]: Laste regular analytics weekly deploy | [production] | 
            
  | 16:35 | <bblack> | Raising varnish max_http_hdr (max allowed applayer response header count) from 64->128 in systemd config and live tuning - https://gerrit.wikimedia.org/r/519661 - T226840 | [production] | 
            
  | 15:04 | <eevans@deploy1001> | scap-helm sessionstore finished | [production] | 
            
  | 15:04 | <eevans@deploy1001> | scap-helm sessionstore cluster codfw completed | [production] | 
            
  | 15:04 | <eevans@deploy1001> | scap-helm sessionstore upgrade production -f sessionstore-codfw-values.yaml stable/kask [namespace: sessionstore, clusters: codfw] | [production] | 
            
  | 15:02 | <eevans@deploy1001> | scap-helm sessionstore finished | [production] | 
            
  | 15:02 | <eevans@deploy1001> | scap-helm sessionstore cluster eqiad completed | [production] | 
            
  | 15:02 | <eevans@deploy1001> | scap-helm sessionstore upgrade production -f sessionstore-eqiad-values.yaml stable/kask [namespace: sessionstore, clusters: eqiad] | [production] | 
            
  | 14:48 | <ema> | pool cp2011 w/ ATS backend T226637 | [production] | 
            
  | 14:47 | <XioNoX> | upload kafkatee to buster-wikimedia | [production] | 
            
  | 14:11 | <eevans@deploy1001> | scap-helm sessionstore finished | [production] | 
            
  | 14:11 | <eevans@deploy1001> | scap-helm sessionstore cluster staging completed | [production] | 
            
  | 14:11 | <eevans@deploy1001> | scap-helm sessionstore upgrade staging -f sessionstore-staging-values.yaml stable/kask [namespace: sessionstore, clusters: staging] | [production] | 
            
  | 14:07 | <eevans@deploy1001> | scap-helm sessionstore upgrade production -f sessionstore-staging-values.yaml stable/kask [namespace: sessionstore, clusters: staging] | [production] | 
            
  | 14:06 | <ema> | depool cp2011 and reimage as upload_ats T226637 | [production] | 
            
  | 11:36 | <elukey> | roll restart eventstreams on all scb1* nodes | [production] | 
            
  | 11:33 | <elukey> | restart eventstreams on scb1001 | [production] | 
            
  | 11:18 | <fsero> | draining kubernetes1006 for applying updates | [production] | 
            
  | 11:14 | <fsero> | draining kubernetes1005 for applying updates | [production] | 
            
  | 11:13 | <fsero> | draining kubernetes2006 for applying updates | [production] |