| 
      
        2023-06-15
      
      ยง
     | 
  
    
  | 18:44 | 
  <ryankemper@puppetmaster1001> | 
  conftool action : set/weight=0:pooled=inactive; selector: name=wdqs2021.* | 
  [production] | 
            
  | 18:44 | 
  <ladsgroup@deploy1002> | 
  Finished scap: Backport for [[gerrit:930542|BlockedDomains: Add logging in case of hit (T337431)]] (duration: 30m 33s) | 
  [production] | 
            
  | 18:25 | 
  <ladsgroup@deploy1002> | 
  ladsgroup: Backport for [[gerrit:930542|BlockedDomains: Add logging in case of hit (T337431)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet | 
  [production] | 
            
  | 18:13 | 
  <ladsgroup@deploy1002> | 
  Started scap: Backport for [[gerrit:930542|BlockedDomains: Add logging in case of hit (T337431)]] | 
  [production] | 
            
  | 17:13 | 
  <bd808@deploy1002> | 
  helmfile [eqiad] DONE helmfile.d/services/developer-portal: apply | 
  [production] | 
            
  | 17:13 | 
  <bd808@deploy1002> | 
  helmfile [eqiad] START helmfile.d/services/developer-portal: apply | 
  [production] | 
            
  | 17:12 | 
  <bd808@deploy1002> | 
  helmfile [codfw] DONE helmfile.d/services/developer-portal: apply | 
  [production] | 
            
  | 17:12 | 
  <bd808@deploy1002> | 
  helmfile [codfw] START helmfile.d/services/developer-portal: apply | 
  [production] | 
            
  | 17:12 | 
  <bd808@deploy1002> | 
  helmfile [staging] DONE helmfile.d/services/developer-portal: apply | 
  [production] | 
            
  | 17:11 | 
  <bd808@deploy1002> | 
  helmfile [staging] START helmfile.d/services/developer-portal: apply | 
  [production] | 
            
  | 17:02 | 
  <joal@deploy1002> | 
  Finished deploy [airflow-dags/analytics@bba655e]: (no justification provided) (duration: 00m 11s) | 
  [production] | 
            
  | 17:02 | 
  <joal@deploy1002> | 
  Started deploy [airflow-dags/analytics@bba655e]: (no justification provided) | 
  [production] | 
            
  | 17:00 | 
  <jnuche@deploy1002> | 
  Installation of scap version "4.53.0" completed for 594 hosts | 
  [production] | 
            
  | 16:59 | 
  <jnuche@deploy1002> | 
  Installing scap version "4.53.0" for 594 hosts | 
  [production] | 
            
  | 16:55 | 
  <jnuche@deploy1002> | 
  Installing scap version "4.53.0" for 595 hosts | 
  [production] | 
            
  | 16:53 | 
  <jnuche@deploy1002> | 
  Installing scap version "4.53.0" for 595 hosts | 
  [production] | 
            
  | 16:52 | 
  <aborrero@cumin2002> | 
  END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudservices2004-dev.codfw.wmnet with OS bullseye | 
  [production] | 
            
  | 16:52 | 
  <aborrero@cumin2002> | 
  END (FAIL) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=99) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - aborrero@cumin2002" | 
  [production] | 
            
  | 16:51 | 
  <aborrero@cumin2002> | 
  START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - aborrero@cumin2002" | 
  [production] | 
            
  | 16:38 | 
  <vgutierrez@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on acmechief2001.codfw.wmnet with reason: https://letsencrypt.status.io/pages/55957a99e800baa4470002da | 
  [production] | 
            
  | 16:38 | 
  <vgutierrez@cumin1001> | 
  START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on acmechief2001.codfw.wmnet with reason: https://letsencrypt.status.io/pages/55957a99e800baa4470002da | 
  [production] | 
            
  | 16:16 | 
  <vgutierrez@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on acmechief2001.codfw.wmnet with reason: https://letsencrypt.status.io/pages/55957a99e800baa4470002da | 
  [production] | 
            
  | 16:15 | 
  <vgutierrez@cumin1001> | 
  START - Cookbook sre.hosts.downtime for 1:00:00 on acmechief2001.codfw.wmnet with reason: https://letsencrypt.status.io/pages/55957a99e800baa4470002da | 
  [production] | 
            
  | 16:14 | 
  <vgutierrez@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for acmechief2001.codfw.wmnet | 
  [production] | 
            
  | 16:14 | 
  <vgutierrez@cumin1001> | 
  START - Cookbook sre.hosts.remove-downtime for acmechief2001.codfw.wmnet | 
  [production] | 
            
  | 15:58 | 
  <hnowlan@deploy1002> | 
  helmfile [eqiad] DONE helmfile.d/services/thumbor: apply | 
  [production] | 
            
  | 15:58 | 
  <hnowlan@deploy1002> | 
  helmfile [eqiad] START helmfile.d/services/thumbor: apply | 
  [production] | 
            
  | 15:56 | 
  <joal@deploy1002> | 
  Finished deploy [airflow-dags/analytics@c584b62]: (no justification provided) (duration: 00m 12s) | 
  [production] | 
            
  | 15:56 | 
  <joal@deploy1002> | 
  Started deploy [airflow-dags/analytics@c584b62]: (no justification provided) | 
  [production] | 
            
  | 15:51 | 
  <mutante> | 
  phabricator - made jnuche (https://phabricator.wikimedia.org/people/manage/32076/) an Administrator T339174 | 
  [production] | 
            
  | 15:46 | 
  <milimetric@deploy1002> | 
  Finished deploy [analytics/refinery@106bf30] (thin): Patch for HiveToDruid with snapshots [thin] (duration: 00m 04s) | 
  [production] | 
            
  | 15:45 | 
  <milimetric@deploy1002> | 
  Started deploy [analytics/refinery@106bf30] (thin): Patch for HiveToDruid with snapshots [thin] | 
  [production] | 
            
  | 15:44 | 
  <claime> | 
  mw2323.codfw.wmnet repooled following T326564 | 
  [production] | 
            
  | 15:44 | 
  <cgoubert@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for mw2323.codfw.wmnet | 
  [production] | 
            
  | 15:44 | 
  <cgoubert@cumin1001> | 
  START - Cookbook sre.hosts.remove-downtime for mw2323.codfw.wmnet | 
  [production] | 
            
  | 15:44 | 
  <milimetric@deploy1002> | 
  Finished deploy [analytics/refinery@106bf30]: Patch for HiveToDruid with snapshots (duration: 07m 01s) | 
  [production] | 
            
  | 15:43 | 
  <claime> | 
  mw2324.codfw.wmnet repooled following T326564 | 
  [production] | 
            
  | 15:39 | 
  <cgoubert@cumin1001> | 
  conftool action : set/pooled=yes; selector: name=mw2323.codfw.wmnet | 
  [production] | 
            
  | 15:37 | 
  <milimetric@deploy1002> | 
  Started deploy [analytics/refinery@106bf30]: Patch for HiveToDruid with snapshots | 
  [production] | 
            
  | 15:36 | 
  <cgoubert@cumin1001> | 
  conftool action : set/pooled=yes; selector: name=mw2324.codfw.wmnet | 
  [production] | 
            
  | 15:36 | 
  <cgoubert@cumin1001> | 
  END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for mw2324.codfw.wmnet | 
  [production] | 
            
  | 15:36 | 
  <cgoubert@cumin1001> | 
  START - Cookbook sre.hosts.remove-downtime for mw2324.codfw.wmnet | 
  [production] | 
            
  | 15:33 | 
  <cgoubert@cumin1001> | 
  conftool action : set/pooled=no; selector: name=mw2324.codfw.wmnet | 
  [production] | 
            
  | 15:33 | 
  <cgoubert@deploy1002> | 
  helmfile [eqiad] DONE helmfile.d/services/mw-web: apply | 
  [production] | 
            
  | 15:33 | 
  <cgoubert@deploy1002> | 
  helmfile [eqiad] START helmfile.d/services/mw-web: apply | 
  [production] | 
            
  | 15:31 | 
  <cgoubert@deploy1002> | 
  helmfile [codfw] DONE helmfile.d/services/mw-web: apply | 
  [production] | 
            
  | 15:28 | 
  <cgoubert@deploy1002> | 
  helmfile [codfw] START helmfile.d/services/mw-web: apply | 
  [production] | 
            
  | 15:28 | 
  <cgoubert@deploy1002> | 
  helmfile [eqiad] DONE helmfile.d/services/mw-api-int: apply | 
  [production] | 
            
  | 15:27 | 
  <claime> | 
  mw2411.codfw.wmnet repooled following T326564 | 
  [production] | 
            
  | 15:26 | 
  <cgoubert@cumin1001> | 
  conftool action : set/pooled=yes; selector: name=mw2411.codfw.wmnet | 
  [production] |