| 
      
        2019-04-08
      
      ยง
     | 
  
    
  | 23:57 | 
  <ebernhardson@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: T218954: Enable WBCS search on commons too (duration: 00m 50s) | 
  [production] | 
            
  | 23:45 | 
  <ebernhardson@deploy1001> | 
  Synchronized wmf-config: T218954: Disable wbcs dispatching query builder on commons (3/3) (duration: 00m 52s) | 
  [production] | 
            
  | 23:41 | 
  <ebernhardson@deploy1001> | 
  Synchronized wmf-config: T218954: Disable wbcs dispatching query builder on commons (3/3) (duration: 00m 51s) | 
  [production] | 
            
  | 23:33 | 
  <ebernhardson@deploy1001> | 
  Synchronized wmf-config/Wikibase.php: T218954: Disable wbcs dispatching query builder on commons (2/3) (duration: 00m 52s) | 
  [production] | 
            
  | 23:10 | 
  <ebernhardson@deploy1001> | 
  Synchronized wmf-config/: T218954: Disable wbcs dispatching query builder on commons (1/3) (duration: 00m 52s) | 
  [production] | 
            
  | 22:45 | 
  <XioNoX> | 
  rollback enable sampling on cr2-eqiad external links | 
  [production] | 
            
  | 22:29 | 
  <XioNoX> | 
  enable sampling on cr2-eqiad external links | 
  [production] | 
            
  | 22:18 | 
  <XioNoX> | 
  enable sampling on eqiad Telia transit link | 
  [production] | 
            
  | 22:04 | 
  <jforrester@deploy1001> | 
  Synchronized php-1.33.0-wmf.24/extensions/WikibaseMediaInfo/src/WikibaseMediaInfoHooks.php: WBMI  T220277 (duration: 00m 57s) | 
  [production] | 
            
  | 22:00 | 
  <XioNoX> | 
  pfw firewall rules update - T217355 | 
  [production] | 
            
  | 20:49 | 
  <bsitzmann@deploy1001> | 
  Finished deploy [mobileapps/deploy@c7fa522]: Update mobileapps to cdb9928 (T220045 T219411 T219667) (duration: 07m 55s) | 
  [production] | 
            
  | 20:41 | 
  <bsitzmann@deploy1001> | 
  Started deploy [mobileapps/deploy@c7fa522]: Update mobileapps to cdb9928 (T220045 T219411 T219667) | 
  [production] | 
            
  | 20:24 | 
  <urandom> | 
  bootstrapping cassandra-b, restbase2019 -- T208087 | 
  [production] | 
            
  | 20:08 | 
  <bearND> | 
  mobileapps deploy failed on canary (Check 'endpoints' failed). Rolled back canary. | 
  [production] | 
            
  | 20:07 | 
  <bsitzmann@deploy1001> | 
  Finished deploy [mobileapps/deploy@c7fa522]: Update mobileapps to cdb9928 (T220045 T219411 T219667) (duration: 02m 10s) | 
  [production] | 
            
  | 20:05 | 
  <bsitzmann@deploy1001> | 
  Started deploy [mobileapps/deploy@c7fa522]: Update mobileapps to cdb9928 (T220045 T219411 T219667) | 
  [production] | 
            
  | 19:59 | 
  <marxarelli> | 
  promotion of 1.33.0-wmf.24 to all wikis completed. error rates nominal aside from usual timeouts. cc: T206678, T220037 | 
  [production] | 
            
  | 19:51 | 
  <dduvall@deploy1001> | 
  rebuilt and synchronized wikiversions files: all wikis to 1.33.0-wmf.24 | 
  [production] | 
            
  | 19:48 | 
  <marxarelli> | 
  promoting 1.33.0-wmf.24 to all wikis. cc: T220037, T206678 | 
  [production] | 
            
  | 19:41 | 
  <dduvall@deploy1001> | 
  Synchronized php: group1 wikis to 1.33.0-wmf.24 (duration: 01m 46s) | 
  [production] | 
            
  | 19:41 | 
  <marxarelli> | 
  dduvall@deploy1001 rebuilt and synchronized wikiversions files: group1 wikis to 1.33.0-wmf.24 | 
  [production] | 
            
  | 19:41 | 
  <marxarelli> | 
  dduvall@deploy1001 rebuilt and synchronized wikiversions files: group1 wikis to 1.33.0-wmf.2 | 
  [production] | 
            
  | 19:39 | 
  <dduvall@deploy1001> | 
  rebuilt and synchronized wikiversions files: group1 wikis to 1.33.0-wmf.24 | 
  [production] | 
            
  | 19:35 | 
  <marxarelli> | 
  starting promotion of 1.33.0-wmf.24 to group1 | 
  [production] | 
            
  | 18:45 | 
  <Lucas_WMDE> | 
  Morning SWAT done | 
  [production] | 
            
  | 18:31 | 
  <bblack> | 
  deploying wiktionary CNAME experiment - https://phabricator.wikimedia.org/T208263#5094712 | 
  [production] | 
            
  | 18:27 | 
  <mobrovac@deploy1001> | 
  Finished deploy [restbase/deploy@9cf5364]: Lower AQS rate limits and fix recommendation-api spec - T219910 T220221 (duration: 21m 14s) | 
  [production] | 
            
  | 18:27 | 
  <otto@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: Enable eventgate-analytics api-request logging for group0 wikis - T214080 (duration: 00m 56s) | 
  [production] | 
            
  | 18:24 | 
  <mobrovac> | 
  restart pdfrender on scb2001 - T174916 | 
  [production] | 
            
  | 18:13 | 
  <otto@deploy1001> | 
  scap-helm eventgate-analytics finished | 
  [production] | 
            
  | 18:13 | 
  <otto@deploy1001> | 
  scap-helm eventgate-analytics cluster eqiad completed | 
  [production] | 
            
  | 18:12 | 
  <otto@deploy1001> | 
  scap-helm eventgate-analytics upgrade production -f eventgate-analytics-eqiad-values.yaml --reset-values stable/eventgate-analytics [namespace: eventgate-analytics, clusters: eqiad] | 
  [production] | 
            
  | 18:12 | 
  <otto@deploy1001> | 
  scap-helm eventgate-analytics finished | 
  [production] | 
            
  | 18:12 | 
  <otto@deploy1001> | 
  scap-helm eventgate-analytics cluster codfw completed | 
  [production] | 
            
  | 18:12 | 
  <otto@deploy1001> | 
  scap-helm eventgate-analytics upgrade production -f eventgate-analytics-codfw-values.yaml --reset-values stable/eventgate-analytics [namespace: eventgate-analytics, clusters: codfw] | 
  [production] | 
            
  | 18:10 | 
  <otto@deploy1001> | 
  scap-helm eventgate-analytics upgrade production -f eventgate-analytics-codfw-values.yaml --reset-values stable/eventgate-analytics [namespace: eventgate-analytics, clusters: codfw] | 
  [production] | 
            
  | 18:09 | 
  <otto@deploy1001> | 
  scap-helm eventgate-analytics finished | 
  [production] | 
            
  | 18:09 | 
  <otto@deploy1001> | 
  scap-helm eventgate-analytics cluster staging completed | 
  [production] | 
            
  | 18:09 | 
  <otto@deploy1001> | 
  scap-helm eventgate-analytics upgrade staging -f eventgate-analytics-staging-values.yaml --reset-values stable/eventgate-analytics [namespace: eventgate-analytics, clusters: staging] | 
  [production] | 
            
  | 18:06 | 
  <mobrovac@deploy1001> | 
  Started deploy [restbase/deploy@9cf5364]: Lower AQS rate limits and fix recommendation-api spec - T219910 T220221 | 
  [production] | 
            
  | 17:50 | 
  <arturo> | 
  T220129 renaming labtestmetal2001.codfw.wmnet to clouddb2001-dev.codfw.wmnet | 
  [production] | 
            
  | 17:42 | 
  <XioNoX> | 
  add swift term to cr1/2-eqiad - T220081 | 
  [production] | 
            
  | 17:14 | 
  <onimisionipe@deploy1001> | 
  Finished deploy [wdqs/wdqs@c30a540]: GUI updates, Updater with redirect fix and Blazegraph with XSS fix (duration: 11m 17s) | 
  [production] | 
            
  | 17:03 | 
  <onimisionipe@deploy1001> | 
  Started deploy [wdqs/wdqs@c30a540]: GUI updates, Updater with redirect fix and Blazegraph with XSS fix | 
  [production] | 
            
  | 16:59 | 
  <mobrovac@deploy1001> | 
  Finished deploy [mobileapps/deploy@64f09a0]: Force-deploy to scb1001 to test the config perms (duration: 00m 16s) | 
  [production] | 
            
  | 16:59 | 
  <mobrovac@deploy1001> | 
  Started deploy [mobileapps/deploy@64f09a0]: Force-deploy to scb1001 to test the config perms | 
  [production] | 
            
  | 16:55 | 
  <mholloway-shell@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings-labs.php: Replace needed WikimediaEditorTasks Beta Cluster config (T220153) (duration: 00m 58s) | 
  [production] | 
            
  | 16:31 | 
  <urandom> | 
  bootstrapping cassandra-a, restbase2019 -- T208087 | 
  [production] | 
            
  | 15:35 | 
  <herron> | 
  aborting ores to logstash kafka logging pipeline switchover for now. puppet applied only to ores2009, reverting now | 
  [production] | 
            
  | 15:19 | 
  <herron> | 
  switching ores to logstash kafka logging pipeline (via temporary puppet disable and rolling puppet agent runs) | 
  [production] |