| 2023-06-29
      
      § | 
    
  | 09:00 | <jmm@cumin2002> | END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2025.codfw.wmnet | [production] | 
            
  | 08:59 | <jmm@cumin2002> | END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2025.codfw.wmnet | [production] | 
            
  | 08:59 | <btullis@deploy1002> | helmfile [staging] DONE helmfile.d/services/datahub: sync on main | [production] | 
            
  | 08:58 | <btullis@deploy1002> | helmfile [staging] START helmfile.d/services/datahub: apply on main | [production] | 
            
  | 08:53 | <jmm@cumin2002> | START - Cookbook sre.hosts.reboot-single for host ganeti2025.codfw.wmnet | [production] | 
            
  | 08:49 | <jmm@cumin2002> | START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2025.codfw.wmnet | [production] | 
            
  | 08:44 | <jmm@cumin2002> | END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2026.codfw.wmnet | [production] | 
            
  | 08:44 | <jmm@cumin2002> | END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2026.codfw.wmnet | [production] | 
            
  | 08:37 | <jmm@cumin2002> | START - Cookbook sre.hosts.reboot-single for host ganeti2026.codfw.wmnet | [production] | 
            
  | 08:25 | <jmm@cumin2002> | START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2026.codfw.wmnet | [production] | 
            
  | 08:18 | <jmm@cumin2002> | END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2027.codfw.wmnet | [production] | 
            
  | 08:18 | <jmm@cumin2002> | END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2027.codfw.wmnet | [production] | 
            
  | 08:12 | <jmm@cumin2002> | START - Cookbook sre.hosts.reboot-single for host ganeti2027.codfw.wmnet | [production] | 
            
  | 08:10 | <jmm@cumin2002> | START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2027.codfw.wmnet | [production] | 
            
  | 08:08 | <jmm@cumin2002> | END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti-test2003.codfw.wmnet | [production] | 
            
  | 08:08 | <jmm@cumin2002> | END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti-test2003.codfw.wmnet | [production] | 
            
  | 08:01 | <jmm@cumin2002> | START - Cookbook sre.hosts.reboot-single for host ganeti-test2003.codfw.wmnet | [production] | 
            
  | 08:01 | <jmm@cumin2002> | START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti-test2003.codfw.wmnet | [production] | 
            
  | 07:22 | <jmm@cumin2002> | END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2028.codfw.wmnet | [production] | 
            
  | 07:22 | <jmm@cumin2002> | END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2028.codfw.wmnet | [production] | 
            
  | 07:15 | <jmm@cumin2002> | START - Cookbook sre.hosts.reboot-single for host ganeti2028.codfw.wmnet | [production] | 
            
  | 07:09 | <jmm@cumin2002> | START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2028.codfw.wmnet | [production] | 
            
  | 07:08 | <jmm@cumin2002> | END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2029.codfw.wmnet | [production] | 
            
  | 07:08 | <jmm@cumin2002> | END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2029.codfw.wmnet | [production] | 
            
  | 07:02 | <jmm@cumin2002> | START - Cookbook sre.hosts.reboot-single for host ganeti2029.codfw.wmnet | [production] | 
            
  | 06:59 | <jmm@cumin2002> | START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2029.codfw.wmnet | [production] | 
            
  | 06:56 | <jmm@cumin2002> | END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti2030.codfw.wmnet | [production] | 
            
  | 06:56 | <jmm@cumin2002> | END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2030.codfw.wmnet | [production] | 
            
  | 06:49 | <jmm@cumin2002> | START - Cookbook sre.hosts.reboot-single for host ganeti2030.codfw.wmnet | [production] | 
            
  | 06:45 | <jmm@cumin2002> | START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti2030.codfw.wmnet | [production] | 
            
  | 01:33 | <rzl@deploy1002> | helmfile [staging] DONE helmfile.d/services/opentelemetry-collector: apply | [production] | 
            
  | 01:32 | <rzl@deploy1002> | helmfile [staging] START helmfile.d/services/opentelemetry-collector: apply | [production] | 
            
  
    | 2023-06-28
      
      § | 
    
  | 22:51 | <rzl@deploy1002> | helmfile [staging] DONE helmfile.d/services/opentelemetry-collector: apply | [production] | 
            
  | 22:50 | <rzl@deploy1002> | helmfile [staging] START helmfile.d/services/opentelemetry-collector: apply | [production] | 
            
  | 21:14 | <eileen_> | civicrm upgraded from 0a59d203 to 9e04c92d | [production] | 
            
  | 20:10 | <brennen@deploy1002> | Finished scap: Backport for [[gerrit:933642|Revert "Deprecate use of targets"]] (duration: 07m 23s) | [production] | 
            
  | 20:05 | <brennen@deploy1002> | jdlrobson and brennen: Backport for [[gerrit:933642|Revert "Deprecate use of targets"]] synced to the testservers: mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet | [production] | 
            
  | 20:03 | <brennen@deploy1002> | Started scap: Backport for [[gerrit:933642|Revert "Deprecate use of targets"]] | [production] | 
            
  | 19:46 | <brennen> | train 1.41.0-wmf.15 (T340243): deploying a revert for T127268 related deprecation logspam - this is likely to impinge on upcoming backport window, which currently has no patches.  will update when finished. | [production] | 
            
  | 19:13 | <mutante> | contint1002,2002,2001 - sudo chmod -R g-w /etc/zuul/wikimedia with deploying gerrit:927980 for T338277 | [production] | 
            
  | 19:03 | <mutante> | contint* - temp disabled puppet - deploying gerrit:927980 - related to git cloning zuul config on CI servers | [production] | 
            
  | 18:20 | <brennen@deploy1002> | Synchronized php: group1 wikis to 1.41.0-wmf.15  refs T340243 (duration: 06m 18s) | [production] | 
            
  | 18:14 | <brennen@deploy1002> | rebuilt and synchronized wikiversions files: group1 wikis to 1.41.0-wmf.15  refs T340243 | [production] | 
            
  | 18:02 | <brennen> | train 1.41.0-wmf.15 ) | [production] | 
            
  | 18:02 | <brennen> | train 1.41.0-wmf.15 (T340243): no current blockers, rolling to group1. | [production] | 
            
  | 18:01 | <brennen> | train 1.41.0-wmf.15 ( | [production] | 
            
  | 17:15 | <btullis@cumin1001> | END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts an-test-coord1002.eqiad.wmnet | [production] | 
            
  | 17:15 | <btullis@cumin1001> | END (PASS) - Cookbook sre.dns.netbox (exit_code=0) | [production] | 
            
  | 17:15 | <btullis@cumin1001> | END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: an-test-coord1002.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - btullis@cumin1001" | [production] | 
            
  | 17:07 | <eevans@cumin1001> | END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host sessionstore1003.eqiad.wmnet with OS bullseye | [production] |