| 2020-04-13
      
      ยง | 
    
  | 23:24 | <mdholloway> | re-ran extensions/MachineVision/maintenance/withholdImages.php on commonswiki | [production] | 
            
  | 23:14 | <mholloway-shell@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: MachineVision withholding list additions (T249939) (duration: 00m 59s) | [production] | 
            
  | 22:41 | <cdanis> | repool codfw | [production] | 
            
  | 22:35 | <ebernhardson> | restart elasticsearch_6@production-search-psi-eqiad on elastic1052 for excessive old gc over last few hours | [production] | 
            
  | 22:35 | <ebernhardson> | restart elasticsearch_6@production-search-psi-eqiad on elastic1052 | [production] | 
            
  | 22:08 | <cdanis> | depool codfw | [production] | 
            
  | 21:43 | <mdholloway> | ran extensions/MachineVision/maintenance/removeBlacklistedSuggestions.php on commonswiki (T249273) | [production] | 
            
  | 21:34 | <mdholloway> | ran extensions/MachineVision/maintenance/removeBlacklistedSuggestions.php on testcommonswiki | [production] | 
            
  | 21:32 | <mholloway-shell@deploy1001> | Synchronized php-1.35.0-wmf.27/extensions/MachineVision: Add script to apply blacklist to current labels (T249273) (duration: 00m 58s) | [production] | 
            
  | 20:49 | <mholloway-shell@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: MachineVision blocklist update (T249895) (duration: 00m 59s) | [production] | 
            
  | 19:56 | <mdholloway> | finished running extensions/MachineVision/maintenance/withholdImages.php on commonswiki (T249939) | [production] | 
            
  | 19:51 | <mdholloway> | running extensions/MachineVision/maintenance/withholdImages.php on commonswiki | [production] | 
            
  | 19:41 | <mdholloway> | ran extensions/MachineVision/maintenance/withholdImages.php on testcommonswiki | [production] | 
            
  | 19:37 | <mholloway-shell@deploy1001> | Synchronized php-1.35.0-wmf.27/extensions/MachineVision: Add support for WITHHOLD_ALL review state (T249939) (duration: 01m 23s) | [production] | 
            
  | 19:13 | <mholloway-shell@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: MachineVision: Add MachineVisionWithholdImageList config (T249939) (duration: 01m 03s) | [production] | 
            
  | 19:06 | <niedzielski> | Morning SWAT done | [production] | 
            
  | 19:02 | <niedzielski@deploy1001> | Synchronized php-1.35.0-wmf.27/skins/MinervaNeue: SWAT: [[gerrit:588405|Update the icon glyph (T249864)]] (duration: 01m 00s) | [production] | 
            
  | 18:49 | <niedzielski@deploy1001> | Synchronized php-1.35.0-wmf.27/extensions/TwoColConflict: SWAT: [[gerrit:588370|Fix double HTML escaping of "copytext" lines in the diff (T249986)]] (duration: 01m 01s) | [production] | 
            
  | 17:01 | <XioNoX> | sample before any other border-in terms in eqiad | [production] | 
            
  | 16:57 | <XioNoX> | sample before any other border-in terms in esams | [production] | 
            
  | 16:50 | <XioNoX> | sample before any other border-in terms in dfw | [production] | 
            
  | 16:46 | <XioNoX> | sample before any other border-in terms in ulsfo | [production] | 
            
  | 16:36 | <XioNoX> | sample before any other border-in terms in eqsin | [production] | 
            
  | 16:36 | <mholloway-shell@deploy1001> | helmfile [CODFW] Ran 'apply' command on namespace 'wikifeeds' for release 'production' . | [production] | 
            
  | 16:33 | <mholloway-shell@deploy1001> | helmfile [EQIAD] Ran 'apply' command on namespace 'wikifeeds' for release 'production' . | [production] | 
            
  | 16:31 | <XioNoX> | Sample all inbound v6 traffic on cr2-eqsin | [production] | 
            
  | 16:31 | <cmjohnson1> | replacing msw-c6-eqiad | [production] | 
            
  | 16:30 | <mholloway-shell@deploy1001> | helmfile [STAGING] Ran 'apply' command on namespace 'wikifeeds' for release 'staging' . | [production] | 
            
  | 15:56 | <marostegui> | Deploy schema change on s4 codfw master - T250067 | [production] | 
            
  | 12:12 | <vgutierrez> | rolling upgrade to ats 8.0.7-rc0-1wm1 in eqsin and codfw | [production] | 
            
  | 11:58 | <akosiaris@deploy1001> | helmfile [STAGING] Ran 'sync' command on namespace 'kube-system' for release 'coredns' . | [production] | 
            
  | 11:57 | <marostegui> | Deploy schema change on eqiad s8 hosts - T250062 | [production] | 
            
  | 11:53 | <akosiaris@deploy1001> | helmfile [STAGING] Ran 'apply' command on namespace 'kube-system' for release 'calico-policy-controller' . | [production] | 
            
  | 11:53 | <akosiaris@deploy1001> | helmfile [EQIAD] Ran 'apply' command on namespace 'kube-system' for release 'calico-policy-controller' . | [production] | 
            
  | 11:53 | <akosiaris@deploy1001> | helmfile [CODFW] Ran 'apply' command on namespace 'kube-system' for release 'calico-policy-controller' . | [production] | 
            
  | 11:53 | <marostegui> | Deploy schema change on codfw master (lag will appear on codfw) - T250062 | [production] | 
            
  | 11:15 | <urbanecm@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT: efe2feb: robots.txt: Disable indexing user (sub)pages and draft-related pages on srwiki (T248860; take II) (duration: 00m 58s) | [production] | 
            
  | 11:14 | <urbanecm@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT: efe2feb: robots.txt: Disable indexing user (sub)pages and draft-related pages on srwiki (T248860) (duration: 00m 58s) | [production] | 
            
  | 10:37 | <jdrewniak@deploy1001> | Synchronized portals: Wikimedia Portals Update: [[gerrit:588383| Bumping portals to master (563985)]] (duration: 00m 58s) | [production] | 
            
  | 10:36 | <jdrewniak@deploy1001> | Synchronized portals/wikipedia.org/assets: Wikimedia Portals Update: [[gerrit:588383| Bumping portals to master (563985)]] (duration: 01m 00s) | [production] | 
            
  | 10:24 | <mutante> | depooled wdqs1004 by request because of high lag | [production] | 
            
  | 10:19 | <marostegui> | Kill updateSpecialPages.php --only=Fewestrevisions for s8 in mwmaint1002, the vslow host is lagging and creating errors | [production] | 
            
  | 10:12 | <mutante> | mwmaint1002 - sudo systemctl status mediawiki_job_translationnotifications-mediawikiwiki.service | [production] | 
            
  | 09:52 | <Urbanecm> | Rename user account Gerakiw@grwikimedia to Geraki@grwikimedia (T245911) | [production] | 
            
  | 09:47 | <Urbanecm> | mwscript createAndPromote.php --wiki=grwikimedia --force Gerakiw <redacted> (T245911) | [production] | 
            
  | 08:15 | <marostegui> | Remove grants for haproxy@10.64.37.15 from labsdb hosts T231280 | [production] | 
            
  | 07:50 | <vgutierrez> | enable memory tracking in ats-tls on cp1085 - T249335 | [production] | 
            
  | 07:43 | <marostegui> | Compress db1092 T232446 | [production] | 
            
  | 07:41 | <marostegui@cumin1001> | dbctl commit (dc=all): 'Temporary pool db1111 in s8 API', diff saved to https://phabricator.wikimedia.org/P10964 and previous config saved to /var/cache/conftool/dbconfig/20200413-074158-marostegui.json | [production] | 
            
  | 07:40 | <vgutierrez> | rolling upgrade to ats 8.0.7-rc0-1wm1 in ulsfo | [production] |