|
2025-12-22
§
|
| 10:04 |
<cgoubert@deploy2002> |
helmfile [staging] START helmfile.d/services/rest-gateway: apply |
[production] |
| 09:31 |
<kevinbazira@deploy2002> |
helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'experimental' for release 'main' . |
[production] |
| 09:02 |
<elukey@cumin1003> |
END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host db2249.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART |
[production] |
| 08:57 |
<elukey@cumin1003> |
START - Cookbook sre.hosts.provision for host db2249.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART |
[production] |
| 08:47 |
<elukey@cumin1003> |
END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host db2249.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART |
[production] |
| 08:47 |
<elukey@cumin1003> |
START - Cookbook sre.hosts.provision for host db2249.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART |
[production] |
| 08:40 |
<elukey@cumin1003> |
END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host db2249.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART |
[production] |
| 08:40 |
<elukey@cumin1003> |
START - Cookbook sre.hosts.provision for host db2249.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART |
[production] |
| 08:32 |
<elukey@cumin1003> |
END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host db2249.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART |
[production] |
| 08:28 |
<elukey@cumin1003> |
START - Cookbook sre.hosts.provision for host db2249.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART |
[production] |
| 08:14 |
<elukey@deploy2002> |
Finished deploy [docker-pkg/deploy@1664255]: (no justification provided) (duration: 00m 08s) |
[production] |
| 08:14 |
<elukey@deploy2002> |
Started deploy [docker-pkg/deploy@1664255]: (no justification provided) |
[production] |
| 08:04 |
<elukey@deploy2002> |
Finished deploy [docker-pkg/deploy@1664255]: (no justification provided) (duration: 00m 07s) |
[production] |
| 08:04 |
<elukey@deploy2002> |
Started deploy [docker-pkg/deploy@1664255]: (no justification provided) |
[production] |
| 08:03 |
<elukey@deploy2002> |
Finished deploy [docker-pkg/deploy@1664255]: (no justification provided) (duration: 00m 11s) |
[production] |
| 08:03 |
<elukey@deploy2002> |
Started deploy [docker-pkg/deploy@1664255]: (no justification provided) |
[production] |
| 07:32 |
<marostegui@cumin1003> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host es2028.codfw.wmnet with OS trixie |
[production] |
| 07:09 |
<marostegui@cumin1003> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on es2028.codfw.wmnet with reason: host reimage |
[production] |
| 07:05 |
<marostegui@cumin1003> |
START - Cookbook sre.hosts.downtime for 2:00:00 on es2028.codfw.wmnet with reason: host reimage |
[production] |
| 06:49 |
<marostegui@cumin1003> |
START - Cookbook sre.hosts.reimage for host es2028.codfw.wmnet with OS trixie |
[production] |
| 01:00 |
<mwpresync@deploy2002> |
Started scap build-images: Publishing wmf/next image |
[production] |
|
2025-12-21
§
|
| 23:48 |
<eileen> |
config revision changed from e478c565 to 8e95f98e |
[production] |
| 21:27 |
<wmbot~anticomposite@tools-bastion-15> |
Deploy 1d2a4fb, a39759c |
[tools.stewardbots] |
| 14:30 |
<wmftkbot> |
Test Kitchen edge-unique experiments (poll 50623) - adds: none; removes: logged-out-retention-round2; fields: none - xLab/MPIC/TK tips at https://w.wiki/FwuD |
[analytics] |
| 13:27 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary (exit_code=0) on codfw1dev, with recreate True, for hosts list: ['cloudvirt2004-dev'] |
[cloudvirt-canary] |
| 13:26 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary on codfw1dev, with recreate True, for hosts list: ['cloudvirt2004-dev'] |
[cloudvirt-canary] |
| 01:24 |
<mwpresync@deploy2002> |
Finished scap build-images: Publishing wmf/next image (duration: 23m 45s) |
[production] |
| 01:00 |
<mwpresync@deploy2002> |
Started scap build-images: Publishing wmf/next image |
[production] |
|
2025-12-20
§
|
| 19:47 |
<wmbot~lucaswerkmeister@tools-bastion-15> |
deployed 7719b7b594 (refactor: kw-only param) |
[tools.quickcategories] |
| 18:49 |
<wmbot~lucaswerkmeister@tools-bastion-15> |
deployed 68012a14ac (upgrade dependencies) |
[tools.quickcategories] |
| 15:24 |
<dzahn@dns1004> |
END - running authdns-update |
[production] |
| 15:23 |
<dzahn@dns1004> |
START - running authdns-update |
[production] |
| 03:10 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) on deployment eqiad1 for all services |
[admin] |
| 03:01 |
<andrewbogott> |
restarting eqiad1 openstack services in response to some fullstack failures |
[admin] |
| 02:56 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.openstack.restart_openstack on deployment eqiad1 for all services |
[admin] |
| 02:27 |
<andrew@cloudcumin1001> |
END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) on deployment eqiad1 for service: project,heat |
[admin] |
| 02:27 |
<andrew@cloudcumin1001> |
START - Cookbook wmcs.openstack.restart_openstack on deployment eqiad1 for service: project,heat |
[admin] |
| 01:23 |
<mwpresync@deploy2002> |
Finished scap build-images: Publishing wmf/next image (duration: 22m 39s) |
[production] |
| 01:00 |
<mwpresync@deploy2002> |
Started scap build-images: Publishing wmf/next image |
[production] |
|
2025-12-19
§
|
| 21:03 |
<jhancock@cumin1003> |
END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host db2249.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART |
[production] |
| 21:03 |
<jhancock@cumin1003> |
START - Cookbook sre.hosts.provision for host db2249.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART |
[production] |
| 21:02 |
<jhancock@cumin1003> |
END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host db2249.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART |
[production] |
| 21:02 |
<jhancock@cumin1003> |
START - Cookbook sre.hosts.provision for host db2249.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART |
[production] |
| 21:01 |
<jhancock@cumin1003> |
END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host db2249.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART |
[production] |
| 21:01 |
<jhancock@cumin1003> |
START - Cookbook sre.hosts.provision for host db2249.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART |
[production] |
| 21:00 |
<jhancock@cumin1003> |
END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host db2249.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART |
[production] |
| 21:00 |
<jhancock@cumin1003> |
START - Cookbook sre.hosts.provision for host db2249.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART |
[production] |
| 20:59 |
<jhancock@cumin1003> |
END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host db2249.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART |
[production] |
| 20:59 |
<jhancock@cumin1003> |
START - Cookbook sre.hosts.provision for host db2249.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART |
[production] |
| 20:58 |
<jhancock@cumin1003> |
END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host db2249.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART |
[production] |