| 2020-01-15
      
      § | 
    
  | 22:40 | <mutante> | phabricator - disabling 'bzimport' user (T242860) | [production] | 
            
  | 21:03 | <jforrester@deploy1001> | Synchronized php-1.35.0-wmf.14/languages/messages/MessagesMrj.php: Fix fallbacks of mrj (Hill Mari) T242409 T242796 (duration: 01m 05s) | [production] | 
            
  | 20:47 | <mutante> | gerrit - adding Zoranzoki to members of extension-GoogleAdSense (endorsed by extension owner Siebrand) (T241509) | [production] | 
            
  | 20:28 | <jforrester@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: Touched IS.php for sync (duration: 01m 05s) | [production] | 
            
  | 20:27 | <jforrester@deploy1001> | sync-file aborted: Enable partial blocks on last wiki,  (duration: 00m 01s) | [production] | 
            
  | 20:17 | <krinkle@deploy1001> | Synchronized php-1.35.0-wmf.15/extensions/MultimediaViewer/resources/: T229484 (duration: 01m 06s) | [production] | 
            
  | 19:57 | <jforrester@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: Enable partial blocks on last wiki, Commons T242570 (duration: 01m 03s) | [production] | 
            
  | 19:54 | <jforrester@deploy1001> | Synchronized wmf-config/InitialiseSettings.php: Enable banner for wikis that recently opted in to partial blocks T240300 T242570 T242569 (duration: 01m 05s) | [production] | 
            
  | 18:10 | <anomie@deploy1001> | Synchronized wmf-config/CommonSettings.php: Set OAuth 2 access token expiry to "infinity" (duration: 01m 04s) | [production] | 
            
  | 17:50 | <anomie@deploy1001> | Synchronized private/PrivateSettings.php: Setting RSA keys for OAuth 2.0 (T242872) (duration: 01m 05s) | [production] | 
            
  | 16:27 | <elukey> | import key 0xDBBF9D42B7B4BD70 (Apache BigTop) manually on install1002's gpg | [production] | 
            
  | 15:54 | <ladsgroup@deploy1001> | Synchronized php-1.35.0-wmf.15/extensions/WikibaseQualityConstraints/extension.json: [[gerrit:565012|Fix service injection for special page (T242846)]] (duration: 01m 08s) | [production] | 
            
  | 15:40 | <ladsgroup@deploy1001> | Synchronized php-1.35.0-wmf.15/extensions/Wikibase/client/includes/Api/PageTerms.php: [[gerrit:565034|Fix invalid iteration over false in PageTerms (T242856)]] (duration: 01m 06s) | [production] | 
            
  | 15:37 | <vgutierrez> | rolling restart of ats-tls instances - T196558 T242778 | [production] | 
            
  | 15:28 | <ema> | cp3064: ats-tls-restart to apply https://gerrit.wikimedia.org/r/559711 T196558 | [production] | 
            
  | 15:20 | <moritzm> | installing OpenSSL security updates on db* hosts | [production] | 
            
  | 15:02 | <moritzm> | installing OpenSSL security updates on mw* | [production] | 
            
  | 14:54 | <jiji@cumin1001> | conftool action : set/weight=10; selector: dc=eqiad,cluster=appserver,service=nginx,name=mw1252.eqiad.wmnet | [production] | 
            
  | 14:54 | <jiji@cumin1001> | conftool action : set/weight=10; selector: dc=eqiad,cluster=appserver,service=nginx,name=mw1251.eqiad.wmnet | [production] | 
            
  | 14:54 | <jiji@cumin1001> | conftool action : set/weight=10; selector: dc=eqiad,cluster=appserver,service=nginx,name=mw1250.eqiad.wmnet | [production] | 
            
  | 14:54 | <jiji@cumin1001> | conftool action : set/weight=10; selector: dc=eqiad,cluster=appserver,service=nginx,name=mw1249.eqiad.wmnet | [production] | 
            
  | 14:54 | <jiji@cumin1001> | conftool action : set/weight=10; selector: dc=eqiad,cluster=appserver,service=nginx,name=mw1248.eqiad.wmnet | [production] | 
            
  | 14:54 | <jiji@cumin1001> | conftool action : set/weight=10; selector: dc=eqiad,cluster=appserver,service=nginx,name=mw1247.eqiad.wmnet | [production] | 
            
  | 14:54 | <jiji@cumin1001> | conftool action : set/weight=10; selector: dc=eqiad,cluster=appserver,service=nginx,name=mw1246.eqiad.wmnet | [production] | 
            
  | 14:54 | <jiji@cumin1001> | conftool action : set/weight=10; selector: dc=eqiad,cluster=appserver,service=nginx,name=mw1245.eqiad.wmnet | [production] | 
            
  | 14:54 | <jiji@cumin1001> | conftool action : set/weight=10; selector: dc=eqiad,cluster=appserver,service=nginx,name=mw1244.eqiad.wmnet | [production] | 
            
  | 14:54 | <jiji@cumin1001> | conftool action : set/weight=10; selector: dc=eqiad,cluster=appserver,service=nginx,name=mw1243.eqiad.wmnet | [production] | 
            
  | 14:54 | <jiji@cumin1001> | conftool action : set/weight=10; selector: dc=eqiad,cluster=appserver,service=nginx,name=mw1242.eqiad.wmnet | [production] | 
            
  | 14:54 | <jiji@cumin1001> | conftool action : set/weight=10; selector: dc=eqiad,cluster=appserver,service=nginx,name=mw1241.eqiad.wmnet | [production] | 
            
  | 14:54 | <jiji@cumin1001> | conftool action : set/weight=10; selector: dc=eqiad,cluster=appserver,service=nginx,name=mw1240.eqiad.wmnet | [production] | 
            
  | 14:54 | <jiji@cumin1001> | conftool action : set/weight=10; selector: dc=eqiad,cluster=appserver,service=nginx,name=mw1239.eqiad.wmnet | [production] | 
            
  | 14:54 | <jiji@cumin1001> | conftool action : set/weight=10; selector: dc=eqiad,cluster=appserver,service=nginx,name=mw1238.eqiad.wmnet | [production] | 
            
  | 14:54 | <effie> | lower weights on slower servers mw1238-mw1252 | [production] | 
            
  | 14:53 | <effie> | pool mw1238, mw1240, mw1246 | [production] | 
            
  | 14:43 | <XioNoX> | reject RPKI invalids in dfw - T220669 | [production] | 
            
  | 14:30 | <moritzm> | rolling restart of FPM on mw1261-mw1265 to pick up OpenSSL security update | [production] | 
            
  | 14:25 | <XioNoX> | reject RPKI invalids in ams - T220669 | [production] | 
            
  | 14:18 | <godog> | reenable puppet on cp hosts, after https://gerrit.wikimedia.org/r/c/operations/puppet/+/563430 deployment | [production] | 
            
  | 14:08 | <effie> | depool mw1238, mw1240, mw1246 | [production] | 
            
  | 14:06 | <liw@deploy1001> | Synchronized php: group1 wikis to 1.35.0-wmf.15 (duration: 01m 07s) | [production] | 
            
  | 14:05 | <liw@deploy1001> | rebuilt and synchronized wikiversions files: group1 wikis to 1.35.0-wmf.15 | [production] | 
            
  | 13:58 | <filippo@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | [production] | 
            
  | 13:56 | <filippo@cumin1001> | START - Cookbook sre.hosts.downtime | [production] | 
            
  | 13:54 | <akosiaris@deploy1001> | helmfile [EQIAD] Ran 'sync' command on namespace 'kube-system' for release 'calico-policy-controller' . | [production] | 
            
  | 13:54 | <akosiaris@deploy1001> | helmfile [CODFW] Ran 'sync' command on namespace 'kube-system' for release 'calico-policy-controller' . | [production] | 
            
  | 13:53 | <akosiaris> | update calico policy on eqiad/codfw/staging. Add new urldownloaders. T224551 | [production] |