| 2025-09-24
      
      § | 
    
  | 07:12 | <andrew@cloudcumin1001> | END (PASS) - Cookbook wmcs.ceph.osd.undrain_node (exit_code=0) | [admin] | 
            
  | 07:11 | <kharlan@deploy1003> | Finished scap sync-world: Backport for [[gerrit:1190184|hCaptcha: Enable account creation trial on phase 2 wikis (T402366)]] (duration: 19m 26s) | [production] | 
            
  | 07:06 | <kharlan@deploy1003> | kharlan: Continuing with sync | [production] | 
            
  | 07:02 | <jmm@cumin2002> | END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti7002.magru.wmnet | [production] | 
            
  | 07:01 | <jmm@cumin2002> | END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti7002.magru.wmnet | [production] | 
            
  | 06:58 | <kharlan@deploy1003> | kharlan: Backport for [[gerrit:1190184|hCaptcha: Enable account creation trial on phase 2 wikis (T402366)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. | [production] | 
            
  | 06:52 | <kharlan@deploy1003> | Started scap sync-world: Backport for [[gerrit:1190184|hCaptcha: Enable account creation trial on phase 2 wikis (T402366)]] | [production] | 
            
  | 06:51 | <jmm@cumin2002> | START - Cookbook sre.hosts.reboot-single for host ganeti7002.magru.wmnet | [production] | 
            
  | 06:49 | <jmm@cumin2002> | START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti7002.magru.wmnet | [production] | 
            
  | 06:46 | <wmbot~component-configs@tools-bastion> | Deployment completed: https://github.com/cluebotng/component-configs/actions/runs/17968643293 | [tools.cluebotng-review] | 
            
  | 06:34 | <wmbot~godog@r5> | END (PASS) - Cookbook wmcs.nfs.add_server (exit_code=0) | [toolsbeta] | 
            
  | 06:26 | <wmbot~godog@r5> | START - Cookbook wmcs.nfs.add_server | [toolsbeta] | 
            
  | 05:35 | <eileen> | * civicrm upgraded from 8cdce9e0 to 1c973d84 | [production] | 
            
  | 03:54 | <andrew@cloudcumin1001> | END (PASS) - Cookbook wmcs.toolforge.k8s.reboot (exit_code=0) for tools-k8s-worker-nfs-12 | [tools] | 
            
  | 03:52 | <andrew@cloudcumin1001> | START - Cookbook wmcs.toolforge.k8s.reboot for tools-k8s-worker-nfs-12 | [tools] | 
            
  | 02:00 | <andrew@cloudcumin1001> | START - Cookbook wmcs.ceph.osd.undrain_node | [admin] | 
            
  | 01:58 | <andrew@cloudcumin1001> | END (PASS) - Cookbook wmcs.ceph.osd.reactivate (exit_code=0) | [admin] | 
            
  | 01:58 | <andrew@cloudcumin1001> | START - Cookbook wmcs.ceph.osd.reactivate | [admin] | 
            
  | 01:57 | <andrew@cumin2002> | END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudcephosd1049.eqiad.wmnet with OS bookworm | [production] | 
            
  | 01:40 | <andrew@cumin2002> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudcephosd1049.eqiad.wmnet with reason: host reimage | [production] | 
            
  | 01:36 | <andrew@cumin2002> | START - Cookbook sre.hosts.downtime for 2:00:00 on cloudcephosd1049.eqiad.wmnet with reason: host reimage | [production] | 
            
  | 01:17 | <andrew@cumin2002> | START - Cookbook sre.hosts.reimage for host cloudcephosd1049.eqiad.wmnet with OS bookworm | [production] | 
            
  | 01:15 | <andrew@cloudcumin1001> | END (PASS) - Cookbook wmcs.ceph.osd.reactivate (exit_code=0) | [admin] | 
            
  | 01:13 | <andrew@cloudcumin1001> | START - Cookbook wmcs.ceph.osd.reactivate | [admin] | 
            
  | 01:12 | <mwpresync@deploy1003> | Finished scap build-images: Publishing wmf/next image (duration: 11m 58s) | [production] | 
            
  | 01:11 | <andrew@cloudcumin1001> | END (FAIL) - Cookbook wmcs.ceph.osd.reactivate (exit_code=99) | [admin] | 
            
  | 01:11 | <andrew@cloudcumin1001> | START - Cookbook wmcs.ceph.osd.reactivate | [admin] | 
            
  | 01:09 | <andrew@cloudcumin1001> | END (FAIL) - Cookbook wmcs.ceph.osd.reactivate (exit_code=99) | [admin] | 
            
  | 01:09 | <andrew@cloudcumin1001> | START - Cookbook wmcs.ceph.osd.reactivate | [admin] | 
            
  | 01:04 | <andrew@cloudcumin1001> | END (FAIL) - Cookbook wmcs.ceph.osd.reactivate (exit_code=99) | [admin] | 
            
  | 01:04 | <andrew@cloudcumin1001> | START - Cookbook wmcs.ceph.osd.reactivate | [admin] | 
            
  | 01:01 | <andrew@cloudcumin1001> | END (FAIL) - Cookbook wmcs.ceph.osd.reactivate (exit_code=99) | [admin] | 
            
  | 01:01 | <andrew@cloudcumin1001> | START - Cookbook wmcs.ceph.osd.reactivate | [admin] | 
            
  | 01:01 | <andrew@cloudcumin1001> | END (FAIL) - Cookbook wmcs.ceph.osd.reactivate (exit_code=99) | [admin] | 
            
  | 01:01 | <andrew@cloudcumin1001> | START - Cookbook wmcs.ceph.osd.reactivate | [admin] | 
            
  | 01:00 | <mwpresync@deploy1003> | Started scap build-images: Publishing wmf/next image | [production] | 
            
  | 01:00 | <andrew@cloudcumin1001> | END (FAIL) - Cookbook wmcs.ceph.osd.reactivate (exit_code=99) | [admin] | 
            
  | 01:00 | <andrew@cloudcumin1001> | START - Cookbook wmcs.ceph.osd.reactivate | [admin] | 
            
  | 00:59 | <andrew@cloudcumin1001> | END (FAIL) - Cookbook wmcs.ceph.osd.reactivate (exit_code=99) | [admin] | 
            
  | 00:59 | <andrew@cloudcumin1001> | START - Cookbook wmcs.ceph.osd.reactivate | [admin] | 
            
  | 00:59 | <andrew@cloudcumin1001> | END (FAIL) - Cookbook wmcs.ceph.osd.reactivate (exit_code=99) | [admin] | 
            
  | 00:59 | <andrew@cloudcumin1001> | START - Cookbook wmcs.ceph.osd.reactivate | [admin] | 
            
  | 00:58 | <andrew@cumin2002> | END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudcephosd1048.eqiad.wmnet with OS bookworm | [production] | 
            
  | 00:44 | <andrew@cumin2002> | END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudcephosd1048.eqiad.wmnet with reason: host reimage | [production] | 
            
  | 00:40 | <andrew@cumin2002> | START - Cookbook sre.hosts.downtime for 2:00:00 on cloudcephosd1048.eqiad.wmnet with reason: host reimage | [production] |