| 2023-10-18
      
      ยง | 
    
  | 16:43 | <jclark@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cp1101.eqiad.wmnet with reason: host reimage | [production] | 
            
  | 16:40 | <jclark@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cp1102.eqiad.wmnet with reason: host reimage | [production] | 
            
  | 16:40 | <jclark@cumin1001> | START - Cookbook sre.hosts.downtime for 2:00:00 on cp1101.eqiad.wmnet with reason: host reimage | [production] | 
            
  | 16:39 | <jclark@cumin1001> | START - Cookbook sre.hosts.downtime for 2:00:00 on cp1100.eqiad.wmnet with reason: host reimage | [production] | 
            
  | 16:37 | <jclark@cumin1001> | START - Cookbook sre.hosts.downtime for 2:00:00 on cp1102.eqiad.wmnet with reason: host reimage | [production] | 
            
  | 16:33 | <eevans@cumin1001> | END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host aqs1010.eqiad.wmnet with OS bullseye | [production] | 
            
  | 16:30 | <jforrester@deploy2002> | helmfile [staging] DONE helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 16:30 | <jforrester@deploy2002> | helmfile [staging] START helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 16:29 | <jforrester@deploy2002> | helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 16:28 | <jclark@cumin1001> | END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp1103.eqiad.wmnet with OS bullseye | [production] | 
            
  | 16:28 | <jclark@cumin1001> | END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1001" | [production] | 
            
  | 16:28 | <jforrester@deploy2002> | helmfile [eqiad] START helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 16:28 | <jforrester@deploy2002> | helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 16:26 | <jforrester@deploy2002> | helmfile [codfw] START helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 16:25 | <jclark@cumin1001> | START - Cookbook sre.hosts.reimage for host cp1102.eqiad.wmnet with OS bullseye | [production] | 
            
  | 16:24 | <jclark@cumin1001> | START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1001" | [production] | 
            
  | 16:24 | <jclark@cumin1001> | END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['cp1102'] | [production] | 
            
  | 16:23 | <jclark@cumin1001> | START - Cookbook sre.hosts.reimage for host cp1101.eqiad.wmnet with OS bullseye | [production] | 
            
  | 16:22 | <jclark@cumin1001> | START - Cookbook sre.hosts.reimage for host cp1100.eqiad.wmnet with OS bullseye | [production] | 
            
  | 16:22 | <jclark@cumin1001> | END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['cp1100'] | [production] | 
            
  | 16:20 | <jclark@cumin1001> | START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['cp1100'] | [production] | 
            
  | 16:20 | <jclark@cumin1001> | END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['cp1101'] | [production] | 
            
  | 16:20 | <jclark@cumin1001> | START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['cp1101'] | [production] | 
            
  | 16:19 | <jclark@cumin1001> | START - Cookbook sre.hosts.reimage for host cp1110.eqiad.wmnet with OS bullseye | [production] | 
            
  | 16:19 | <jclark@cumin1001> | END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['cp1110'] | [production] | 
            
  | 16:18 | <jclark@cumin1001> | START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['cp1110'] | [production] | 
            
  | 16:18 | <jforrester@deploy2002> | helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 16:18 | <jclark@cumin1001> | START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['cp1102'] | [production] | 
            
  | 16:18 | <jclark@cumin1001> | END (PASS) - Cookbook sre.dns.netbox (exit_code=0) | [production] | 
            
  | 16:18 | <jclark@cumin1001> | END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update  mgmt cp1102 - jclark@cumin1001" | [production] | 
            
  | 16:17 | <jforrester@deploy2002> | helmfile [codfw] START helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 16:17 | <jforrester@deploy2002> | helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 16:17 | <jclark@cumin1001> | START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update  mgmt cp1102 - jclark@cumin1001" | [production] | 
            
  | 16:17 | <jforrester@deploy2002> | helmfile [eqiad] START helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 16:16 | <jforrester@deploy2002> | helmfile [staging] DONE helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 16:15 | <jforrester@deploy2002> | helmfile [staging] START helmfile.d/services/wikifunctions: apply | [production] | 
            
  | 16:15 | <jclark@cumin1001> | START - Cookbook sre.dns.netbox | [production] | 
            
  | 16:14 | <jclark@cumin1001> | START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['cp1102'] | [production] | 
            
  | 16:14 | <jclark@cumin1001> | END (ERROR) - Cookbook sre.hardware.upgrade-firmware (exit_code=97) upgrade firmware for hosts ['cp1110'] | [production] | 
            
  | 16:14 | <jclark@cumin1001> | START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['cp1110'] | [production] | 
            
  | 16:14 | <jclark@cumin1001> | END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cp1102 | [production] | 
            
  | 16:13 | <jclark@cumin1001> | START - Cookbook sre.network.configure-switch-interfaces for host cp1102 | [production] | 
            
  | 16:11 | <eevans@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on aqs1010.eqiad.wmnet with reason: host reimage | [production] | 
            
  | 16:10 | <jclark@cumin1001> | END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['cp1110'] | [production] | 
            
  | 16:08 | <jclark@cumin1001> | START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['cp1110'] | [production] | 
            
  | 16:08 | <jclark@cumin1001> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cp1103.eqiad.wmnet with reason: host reimage | [production] | 
            
  | 16:08 | <eevans@cumin1001> | START - Cookbook sre.hosts.downtime for 2:00:00 on aqs1010.eqiad.wmnet with reason: host reimage | [production] | 
            
  | 16:08 | <jclark@cumin1001> | END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['cp1102'] | [production] | 
            
  | 16:07 | <jclark@cumin1001> | START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['cp1102'] | [production] | 
            
  | 16:07 | <jclark@cumin1001> | END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cp1102 | [production] |