| 
      
        2020-10-08
      
      ยง
     | 
  
    
  | 19:24 | 
  <ebernhardson@deploy1001> | 
  Finished deploy [wikimedia/discovery/analytics@3b11443]: search_satisfaction: Alias sample multiplier to expected name (duration: 01m 09s) | 
  [production] | 
            
  | 19:23 | 
  <ebernhardson@deploy1001> | 
  Started deploy [wikimedia/discovery/analytics@3b11443]: search_satisfaction: Alias sample multiplier to expected name | 
  [production] | 
            
  | 18:57 | 
  <volker-e@deploy1001> | 
  Finished deploy [design/style-guide@b1166af]: Deploy design/style-guide:  (duration: 00m 06s) | 
  [production] | 
            
  | 18:57 | 
  <volker-e@deploy1001> | 
  Started deploy [design/style-guide@b1166af]: Deploy design/style-guide: | 
  [production] | 
            
  | 18:17 | 
  <tchanders@deploy1001> | 
  Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:632908|Enable Special:Investigate by default on production (T264357)]] (duration: 01m 06s) | 
  [production] | 
            
  | 17:50 | 
  <root@cumin1001> | 
  END (PASS) - Cookbook sre.dns.netbox (exit_code=0) | 
  [production] | 
            
  | 17:49 | 
  <ebernhardson@deploy1001> | 
  Finished deploy [wikimedia/discovery/analytics@945e5c1]: airflow: Set search satisfaction dag start date to oldest current available data (duration: 11m 55s) | 
  [production] | 
            
  | 17:44 | 
  <root@cumin1001> | 
  START - Cookbook sre.dns.netbox | 
  [production] | 
            
  | 17:37 | 
  <ebernhardson@deploy1001> | 
  Started deploy [wikimedia/discovery/analytics@945e5c1]: airflow: Set search satisfaction dag start date to oldest current available data | 
  [production] | 
            
  | 17:31 | 
  <volans@cumin1001> | 
  START - Cookbook sre.dns.netbox | 
  [production] | 
            
  | 17:30 | 
  <volans@cumin1001> | 
  END (PASS) - Cookbook sre.dns.netbox (exit_code=0) | 
  [production] | 
            
  | 17:27 | 
  <andrew@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | 
  [production] | 
            
  | 17:26 | 
  <andrew@cumin1001> | 
  START - Cookbook sre.hosts.downtime | 
  [production] | 
            
  | 17:23 | 
  <volans@cumin1001> | 
  START - Cookbook sre.dns.netbox | 
  [production] | 
            
  | 17:16 | 
  <shdubsh> | 
  install prometheus-rsyslog-exporter_0.0.0+git20201008 on centrallog1001 - T210137 | 
  [production] | 
            
  | 16:25 | 
  <mutante> | 
  rebooting cloudvirt1023 - trying PXE boot | 
  [production] | 
            
  | 16:19 | 
  <hashar> | 
  Restarting CI Jenkins | 
  [production] | 
            
  | 16:15 | 
  <volans@cumin1001> | 
  END (PASS) - Cookbook sre.dns.netbox (exit_code=0) | 
  [production] | 
            
  | 16:09 | 
  <volans@cumin1001> | 
  START - Cookbook sre.dns.netbox | 
  [production] | 
            
  | 16:08 | 
  <volans@cumin1001> | 
  END (ERROR) - Cookbook sre.dns.netbox (exit_code=97) | 
  [production] | 
            
  | 16:08 | 
  <volans@cumin1001> | 
  START - Cookbook sre.dns.netbox | 
  [production] | 
            
  | 14:21 | 
  <marostegui> | 
  Set  global innodb_change_buffering = all; on pc2009 T263443 | 
  [production] | 
            
  | 14:17 | 
  <moritzm> | 
  importing icu 63.1-6+deb10u1~wmf5 to component/icu63 T264991 | 
  [production] | 
            
  | 13:37 | 
  <hnowlan@deploy1001> | 
  helmfile [codfw] Ran 'sync' command on namespace 'api-gateway' for release 'staging' . | 
  [production] | 
            
  | 13:37 | 
  <hnowlan@deploy1001> | 
  helmfile [codfw] Ran 'sync' command on namespace 'api-gateway' for release 'production' . | 
  [production] | 
            
  | 12:29 | 
  <kart_> | 
  Updated cxserver to 2020-10-08-053343-production (T264407, T264859) | 
  [production] | 
            
  | 12:26 | 
  <kartik@deploy1001> | 
  helmfile [codfw] Ran 'sync' command on namespace 'cxserver' for release 'production' . | 
  [production] | 
            
  | 12:24 | 
  <kartik@deploy1001> | 
  helmfile [eqiad] Ran 'sync' command on namespace 'cxserver' for release 'production' . | 
  [production] | 
            
  | 12:21 | 
  <kartik@deploy1001> | 
  helmfile [staging] Ran 'sync' command on namespace 'cxserver' for release 'staging' . | 
  [production] | 
            
  | 12:10 | 
  <hnowlan@deploy1001> | 
  helmfile [eqiad] Ran 'sync' command on namespace 'api-gateway' for release 'staging' . | 
  [production] | 
            
  | 12:10 | 
  <hnowlan@deploy1001> | 
  helmfile [eqiad] Ran 'sync' command on namespace 'api-gateway' for release 'production' . | 
  [production] | 
            
  | 12:08 | 
  <kormat@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | 
  [production] | 
            
  | 12:07 | 
  <kormat@cumin1001> | 
  START - Cookbook sre.hosts.downtime | 
  [production] | 
            
  | 12:07 | 
  <hnowlan@deploy1001> | 
  helmfile [staging] Ran 'sync' command on namespace 'api-gateway' for release 'staging' . | 
  [production] | 
            
  | 12:07 | 
  <hnowlan@deploy1001> | 
  helmfile [staging] Ran 'sync' command on namespace 'api-gateway' for release 'production' . | 
  [production] | 
            
  | 12:05 | 
  <hnowlan@deploy1001> | 
  helmfile [staging] Ran 'sync' command on namespace 'api-gateway' for release 'production' . | 
  [production] | 
            
  | 12:05 | 
  <hnowlan@deploy1001> | 
  helmfile [staging] Ran 'sync' command on namespace 'api-gateway' for release 'staging' . | 
  [production] | 
            
  | 10:54 | 
  <aborrero@cumin2001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) | 
  [production] | 
            
  | 10:52 | 
  <aborrero@cumin2001> | 
  START - Cookbook sre.hosts.downtime | 
  [production] | 
            
  | 10:45 | 
  <hnowlan@puppetmaster1001> | 
  conftool action : set/pooled=yes:weight=10; selector: dc=eqiad,cluster=restbase,service=restbase-backend,name=restbase1030.eqiad.wmnet | 
  [production] | 
            
  | 10:45 | 
  <hnowlan@puppetmaster1001> | 
  conftool action : set/pooled=yes:weight=10; selector: dc=eqiad,cluster=restbase,service=restbase-ssl,name=restbase1030.eqiad.wmnet | 
  [production] | 
            
  | 10:45 | 
  <hnowlan@puppetmaster1001> | 
  conftool action : set/pooled=yes:weight=10; selector: dc=eqiad,cluster=restbase,service=restbase,name=restbase1030.eqiad.wmnet | 
  [production] | 
            
  | 10:37 | 
  <moritzm> | 
  installing Postgres security updates on netboxdb1001 | 
  [production] | 
            
  | 10:34 | 
  <hnowlan@puppetmaster1001> | 
  conftool action : set/pooled=yes:weight=10; selector: dc=eqiad,cluster=restbase,service=restbase-backend,name=restbase1029.eqiad.wmnet | 
  [production] | 
            
  | 10:34 | 
  <hnowlan@puppetmaster1001> | 
  conftool action : set/pooled=yes:weight=10; selector: dc=eqiad,cluster=restbase,service=restbase-ssl,name=restbase1029.eqiad.wmnet | 
  [production] | 
            
  | 10:34 | 
  <hnowlan@puppetmaster1001> | 
  conftool action : set/pooled=yes:weight=10; selector: dc=eqiad,cluster=restbase,service=restbase,name=restbase1029.eqiad.wmnet | 
  [production] | 
            
  | 10:32 | 
  <moritzm> | 
  installing Postgres security updates on netboxdb2001 | 
  [production] | 
            
  | 10:29 | 
  <mvolz@deploy1001> | 
  helmfile [eqiad] Ran 'sync' command on namespace 'citoid' for release 'production' . | 
  [production] | 
            
  | 10:28 | 
  <hnowlan@puppetmaster1001> | 
  conftool action : set/pooled=yes:weight=10; selector: dc=eqiad,cluster=restbase,service=restbase-backend,name=restbase1028.eqiad.wmnet | 
  [production] | 
            
  | 10:27 | 
  <hnowlan@puppetmaster1001> | 
  conftool action : set/pooled=yes:weight=10; selector: dc=eqiad,cluster=restbase,service=restbase-ssl,name=restbase1028.eqiad.wmnet | 
  [production] |