| 
      
        2019-03-21
      
      ยง
     | 
  
    
  | 18:25 | 
  <sbisson@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:498166|Disable Welcome survey on viwiki]] (duration: 00m 49s) | 
  [production] | 
            
  | 18:23 | 
  <otto@deploy1001> | 
  scap-helm eventgate-analytics finished | 
  [production] | 
            
  | 18:23 | 
  <otto@deploy1001> | 
  scap-helm eventgate-analytics cluster staging completed | 
  [production] | 
            
  | 18:23 | 
  <otto@deploy1001> | 
  scap-helm eventgate-analytics upgrade staging -f eventgate-analytics-staging-values.yaml stable/eventgate-analytics [namespace: eventgate-analytics, clusters: staging] | 
  [production] | 
            
  | 18:17 | 
  <gehel@cumin2001> | 
  END (PASS) - Cookbook sre.elasticsearch.force-shard-allocation (exit_code=0) | 
  [production] | 
            
  | 18:16 | 
  <sbisson@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:496857|Enable logging for CitationUsage and CitationUsagePageLoad]] (duration: 00m 49s) | 
  [production] | 
            
  | 18:13 | 
  <gehel@cumin2001> | 
  START - Cookbook sre.elasticsearch.force-shard-allocation | 
  [production] | 
            
  | 18:12 | 
  <otto@deploy1001> | 
  scap-helm eventgate-analytics finished | 
  [production] | 
            
  | 18:12 | 
  <otto@deploy1001> | 
  scap-helm eventgate-analytics cluster staging completed | 
  [production] | 
            
  | 18:12 | 
  <otto@deploy1001> | 
  scap-helm eventgate-analytics upgrade staging -f eventgate-analytics-staging-values.yaml stable/eventgate-analytics [namespace: eventgate-analytics, clusters: staging] | 
  [production] | 
            
  | 18:11 | 
  <sbisson@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit:494552|Disable reader trust survey v2]] (duration: 00m 50s) | 
  [production] | 
            
  | 18:08 | 
  <gehel@cumin1001> | 
  START - Cookbook sre.elasticsearch.e6-upgrade | 
  [production] | 
            
  | 18:05 | 
  <otto@deploy1001> | 
  scap-helm eventgate-analytics finished | 
  [production] | 
            
  | 18:05 | 
  <otto@deploy1001> | 
  scap-helm eventgate-analytics cluster staging completed | 
  [production] | 
            
  | 18:05 | 
  <otto@deploy1001> | 
  scap-helm eventgate-analytics upgrade staging -f eventgate-analytics-staging-values.yaml stable/eventgate-analytics [namespace: eventgate-analytics, clusters: staging] | 
  [production] | 
            
  | 18:01 | 
  <otto@deploy1001> | 
  scap-helm eventgate-analytics finished | 
  [production] | 
            
  | 18:01 | 
  <otto@deploy1001> | 
  scap-helm eventgate-analytics cluster staging completed | 
  [production] | 
            
  | 18:01 | 
  <otto@deploy1001> | 
  scap-helm eventgate-analytics upgrade staging -f eventgate-analytics-staging-values.yaml stable/eventgate-analytics [namespace: eventgate-analytics, clusters: staging] | 
  [production] | 
            
  | 17:56 | 
  <bblack> | 
  everything back to normal for lvs1002/lvs1005 (high-traffic2 @ eqiad) | 
  [production] | 
            
  | 17:55 | 
  <bblack> | 
  restarting pybal on lvs1002 | 
  [production] | 
            
  | 17:54 | 
  <otto@deploy1001> | 
  scap-helm eventgate-analytics finished | 
  [production] | 
            
  | 17:54 | 
  <otto@deploy1001> | 
  scap-helm eventgate-analytics cluster staging completed | 
  [production] | 
            
  | 17:54 | 
  <otto@deploy1001> | 
  scap-helm eventgate-analytics upgrade staging -f eventgate-analytics-staging-values.yaml stable/eventgate-analytics [namespace: eventgate-analytics, clusters: staging] | 
  [production] | 
            
  | 17:49 | 
  <reedy@deploy1001> | 
  Synchronized php-1.33.0-wmf.22/includes/user/User.php: Iab24923c613d6aeed4b574f587fc4cee8f33077c (duration: 00m 51s) | 
  [production] | 
            
  | 17:43 | 
  <bblack> | 
  restarting pybal on lvs1005 | 
  [production] | 
            
  | 17:43 | 
  <jforrester@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: SDC: Enable EntitySourceBasedFederation on TestCommons (duration: 00m 50s) | 
  [production] | 
            
  | 17:37 | 
  <gehel@cumin1001> | 
  END (FAIL) - Cookbook sre.elasticsearch.e6-upgrade (exit_code=99) | 
  [production] | 
            
  | 17:35 | 
  <bblack> | 
  disabled puppet on lvs1002 + lvs1005 for new service rollout | 
  [production] | 
            
  | 17:28 | 
  <gehel@cumin1001> | 
  START - Cookbook sre.elasticsearch.e6-upgrade | 
  [production] | 
            
  | 17:27 | 
  <jforrester@deploy1001> | 
  Synchronized wmf-config/CommonSettings.php: SDC: Add test-commons.wikimedia.org to wgCrossSiteAJAXdomains (duration: 00m 49s) | 
  [production] | 
            
  | 17:11 | 
  <gehel@cumin1001> | 
  END (FAIL) - Cookbook sre.elasticsearch.e6-upgrade (exit_code=99) | 
  [production] | 
            
  | 17:07 | 
  <jforrester@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: SDC: Enable Depicts on TestCommons, with related config (duration: 00m 50s) | 
  [production] | 
            
  | 17:03 | 
  <gehel@cumin1001> | 
  START - Cookbook sre.elasticsearch.e6-upgrade | 
  [production] | 
            
  | 17:03 | 
  <gehel@cumin1001> | 
  END (ERROR) - Cookbook sre.elasticsearch.e6-upgrade (exit_code=97) | 
  [production] | 
            
  | 17:02 | 
  <gehel@cumin2001> | 
  END (PASS) - Cookbook sre.elasticsearch.force-shard-allocation (exit_code=0) | 
  [production] | 
            
  | 17:02 | 
  <gehel@cumin2001> | 
  START - Cookbook sre.elasticsearch.force-shard-allocation | 
  [production] | 
            
  | 16:39 | 
  <gehel@cumin1001> | 
  START - Cookbook sre.elasticsearch.e6-upgrade | 
  [production] | 
            
  | 16:38 | 
  <gehel@cumin1001> | 
  END (ERROR) - Cookbook sre.elasticsearch.e6-upgrade (exit_code=97) | 
  [production] | 
            
  | 16:38 | 
  <gehel@cumin1001> | 
  START - Cookbook sre.elasticsearch.e6-upgrade | 
  [production] | 
            
  | 16:38 | 
  <gehel@cumin1001> | 
  END (ERROR) - Cookbook sre.elasticsearch.e6-upgrade (exit_code=97) | 
  [production] | 
            
  | 16:38 | 
  <gehel@cumin2001> | 
  END (PASS) - Cookbook sre.elasticsearch.force-shard-allocation (exit_code=0) | 
  [production] | 
            
  | 16:38 | 
  <gehel@cumin2001> | 
  START - Cookbook sre.elasticsearch.force-shard-allocation | 
  [production] | 
            
  | 16:29 | 
  <gehel@cumin2001> | 
  END (PASS) - Cookbook sre.elasticsearch.force-shard-allocation (exit_code=0) | 
  [production] | 
            
  | 16:29 | 
  <gehel@cumin2001> | 
  START - Cookbook sre.elasticsearch.force-shard-allocation | 
  [production] | 
            
  | 16:03 | 
  <marostegui@deploy1001> | 
  Synchronized wmf-config/db-codfw.php: Depool db2096 for onsite maintenance (duration: 00m 50s) | 
  [production] | 
            
  | 16:01 | 
  <marostegui> | 
  Poweroff db2096 for onsite maintenance T218336 | 
  [production] | 
            
  | 15:20 | 
  <moritzm> | 
  rebooting flerovium/furud for kernel updates | 
  [production] | 
            
  | 14:35 | 
  <moritzm> | 
  restarging jenkins on releases* after Java update | 
  [production] | 
            
  | 14:18 | 
  <gtirloni> | 
  downtimed labtestweb2001 (T218881) | 
  [production] | 
            
  | 14:11 | 
  <vgutierrez> | 
  re-enabling puppet in acme-chief clients - T218862 | 
  [production] |