| 2020-01-23
      
      § | 
    
  | 16:02 | <ladsgroup@deploy1001> | Synchronized php-1.35.0-wmf.16/extensions/Wikibase/data-access/src/EntitySourceDefinitions.php: [[gerrit:566721|EntitySourceDefitions::getEntityTypeToSourceMapping fix for sub entities (T242415 T214557)]] (duration: 01m 08s) | [production] | 
            
  | 16:00 | <rlazarus> | Starting etcd main cluster switchover from codfw to eqiad | [production] | 
            
  | 15:45 | <vgutierrez> | restarting high-traffic1 && high-traffic2 primary LVSs - T236120 T238625 | [production] | 
            
  | 15:32 | <vgutierrez> | restarting secondary LVSs - T236120 T238625 | [production] | 
            
  | 15:22 | <moritzm> | mask uwsgi.service on debmonitor2001 T222874 | [production] | 
            
  | 15:06 | <vgutierrez@puppetmaster1001> | conftool action : set/weight=1; selector: name=cp4026.ulsfo.wmnet,service=nginx | [production] | 
            
  | 14:39 | <vgutierrez@puppetmaster1001> | conftool action : set/weight=1; selector: service=ats-tls,name=cp4026.ulsfo.wmnet | [production] | 
            
  | 14:17 | <marostegui> | Remove wikiadmin2 user from codfw x1 hosts - T243512 | [production] | 
            
  | 13:34 | <vgutierrez@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | [production] | 
            
  | 13:32 | <vgutierrez@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 13:19 | <jmm@cumin2001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | [production] | 
            
  | 13:19 | <jmm@cumin2001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 12:50 | <Amir1> | EU SWAT is done | [production] | 
            
  | 12:48 | <ladsgroup@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:566716|Set EntitySourceBasedFederation true for testwiki (T243395)]] (duration: 01m 06s) | [production] | 
            
  | 12:47 | <ladsgroup@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:566716|Set EntitySourceBasedFederation true for testwiki (T243395)]] (duration: 01m 05s) | [production] | 
            
  | 12:46 | <Urbanecm> | Run renameRestrictions.php 'autopatrol' 'editautopatrolprotected' for all Serbian wikis (T230103) | [production] | 
            
  | 12:44 | <Urbanecm> | mwscript renameRestrictions.php --wiki=hewiki 'autopatrol' 'editautopatrolprotected' (T230103) | [production] | 
            
  | 12:44 | <Urbanecm> | mwscript renameRestrictions.php --wiki=etwiki 'autopatrol' 'editautopatrolprotected' (T230103) | [production] | 
            
  | 12:40 | <urbanecm@deploy1001> | Synchronized wmf-config/flaggedrevs.php: SWAT: 0c2fb70: Use editautopatrolprotected right for pages protected for autopatrollers (3/3; T230103) (duration: 01m 05s) | [production] | 
            
  | 12:39 | <urbanecm@deploy1001> | Synchronized wmf-config/CommonSettings.php: SWAT: 0c2fb70: Use editautopatrolprotected right for pages protected for autopatrollers (2/3; T230103) (duration: 01m 08s) | [production] | 
            
  | 12:35 | <Urbanecm> | mwscript renameRestrictions.php --wiki=ckbwiki 'autopatrol' 'editautopatrolprotected' (T230103) | [production] | 
            
  | 12:33 | <urbanecm@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT: 0c2fb70: Use editautopatrolprotected right for pages protected for autopatrollers; fixing broken cache (T230103) (duration: 01m 04s) | [production] | 
            
  | 12:31 | <twentyafterfour> | Deploying hotfix for T243479, restarting php7.3-fpm on phab1003 | [production] | 
            
  | 12:31 | <urbanecm@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT: 0c2fb70: Use editautopatrolprotected right for pages protected for autopatrollers (T230103) (duration: 01m 06s) | [production] | 
            
  | 12:15 | <ladsgroup@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:566701|Set useEntitySourceBasedFederation to true for Wikidata (T241972)]] (duration: 01m 04s) | [production] | 
            
  | 12:14 | <ladsgroup@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:566701|Set useEntitySourceBasedFederation to true for Wikidata (T241972)]] (duration: 01m 06s) | [production] | 
            
  | 12:10 | <ladsgroup@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:566123|Move CX out of beta for af, is, lv and ne WPs (T242011 T242012 T242014 T242016)]] (duration: 01m 05s) | [production] | 
            
  | 12:08 | <ladsgroup@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:566123|Move CX out of beta for af, is, lv and ne WPs (T242011 T242012 T242014 T242016)]] (duration: 01m 08s) | [production] | 
            
  | 11:37 | <jbond42> | updating order in resolve search list https://gerrit.wikimedia.org/r/c/operations/puppet/+/566567 | [production] | 
            
  | 10:25 | <vgutierrez> | depooling and reimaging cp4026 as buster - T242093 | [production] | 
            
  | 09:13 | <moritzm> | installing xen updates (only pulled in via deps, otherwise unused) | [production] | 
            
  | 08:45 | <marostegui> | Stop mysql on es2024 to "clone" es2025 - T243052 | [production] | 
            
  | 06:05 | <marostegui> | Remove partitions from db1097:3314 - T239453 | [production] | 
            
  | 06:03 | <marostegui@cumin1001> | dbctl commit (dc=all): 'Depool db1097:3314 - T239453', diff saved to https://phabricator.wikimedia.org/P10248 and previous config saved to /var/cache/conftool/dbconfig/20200123-060308-marostegui.json | [production] | 
            
  | 05:59 | <marostegui@cumin1001> | dbctl commit (dc=all): 'Repool db1103:3314 - T239453', diff saved to https://phabricator.wikimedia.org/P10247 and previous config saved to /var/cache/conftool/dbconfig/20200123-055919-marostegui.json | [production] | 
            
  | 05:55 | <marostegui> | Compress some tables on db1124:3318, this might generate lag on s8 labs - T232446 | [production] | 
            
  | 01:40 | <jforrester@deploy1001> | Synchronized php-1.35.0-wmf.16/extensions/AbuseFilter/includes/AFComputedVariable.php: T243469 When no registration date is recorded, use 2008-01-15 (duration: 01m 08s) | [production] | 
            
  | 01:37 | <twentyafterfour> | Phabricator deployment completed with no apparent issues. | [production] | 
            
  | 01:27 | <twentyafterfour> | Deploying phabricator update tagged release/2020-01-23/1 | [production] | 
            
  | 00:41 | <catrope@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: resync (duration: 01m 07s) | [production] | 
            
  | 00:40 | <RoanKattouw> | Deployment freeze lifted | [production] | 
            
  
    | 2020-01-22
      
      § | 
    
  | 23:46 | <James_F> | <RoanKattouw> T236104 happened again, and this time I'm leaving it broken so I can investigate. Please don't use do any MW deployments (use scap) for now | [production] | 
            
  | 23:31 | <eileen> | civicrm revision changed from 036b742316 to fbd5c35fb0, config revision is 74a355670a | [production] | 
            
  | 23:28 | <eileen> | civicrm revision changed from 7595104180 to 036b742316, config revision is 74a355670a | [production] | 
            
  | 23:14 | <eileen> | civicrm revision changed from c74092ad63 to 7595104180, config revision is 74a355670a | [production] | 
            
  | 23:06 | <XioNoX> | configure flowspec on cr3-knams | [production] | 
            
  | 22:39 | <catrope@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: Enable homepage on ukwiki, huwiki, hywiki (T238320, T231720, T230478, T230676) (duration: 01m 05s) | [production] | 
            
  | 22:30 | <catrope@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: Enable help panel on ukwiki, huwiki, hywiki (T238319, T231720, T230478, T230676) (duration: 01m 04s) | [production] | 
            
  | 22:19 | <catrope@deploy1001> | Synchronized php-1.35.0-wmf.16/extensions/CodeReview/: T243337 (duration: 01m 06s) | [production] | 
            
  | 22:13 | <catrope@deploy1001> | Finished scap: i18n changes for SWAT: Special page aliases for GrowthExperiments (T230676); messages for machinevision-tester group (T243440); fix namespace names for atj (T243125) (duration: 40m 48s) | [production] |