1-50 of 10000 results (87ms)
2025-12-22 §
07:09 <marostegui@cumin1003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on es2028.codfw.wmnet with reason: host reimage [production]
07:05 <marostegui@cumin1003> START - Cookbook sre.hosts.downtime for 2:00:00 on es2028.codfw.wmnet with reason: host reimage [production]
06:49 <marostegui@cumin1003> START - Cookbook sre.hosts.reimage for host es2028.codfw.wmnet with OS trixie [production]
01:00 <mwpresync@deploy2002> Started scap build-images: Publishing wmf/next image [production]
2025-12-21 §
23:48 <eileen> config revision changed from e478c565 to 8e95f98e [production]
01:24 <mwpresync@deploy2002> Finished scap build-images: Publishing wmf/next image (duration: 23m 45s) [production]
01:00 <mwpresync@deploy2002> Started scap build-images: Publishing wmf/next image [production]
2025-12-20 §
15:24 <dzahn@dns1004> END - running authdns-update [production]
15:23 <dzahn@dns1004> START - running authdns-update [production]
01:23 <mwpresync@deploy2002> Finished scap build-images: Publishing wmf/next image (duration: 22m 39s) [production]
01:00 <mwpresync@deploy2002> Started scap build-images: Publishing wmf/next image [production]
2025-12-19 §
21:03 <jhancock@cumin1003> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host db2249.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART [production]
21:03 <jhancock@cumin1003> START - Cookbook sre.hosts.provision for host db2249.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART [production]
21:02 <jhancock@cumin1003> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host db2249.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART [production]
21:02 <jhancock@cumin1003> START - Cookbook sre.hosts.provision for host db2249.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART [production]
21:01 <jhancock@cumin1003> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host db2249.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART [production]
21:01 <jhancock@cumin1003> START - Cookbook sre.hosts.provision for host db2249.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART [production]
21:00 <jhancock@cumin1003> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host db2249.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART [production]
21:00 <jhancock@cumin1003> START - Cookbook sre.hosts.provision for host db2249.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART [production]
20:59 <jhancock@cumin1003> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host db2249.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART [production]
20:59 <jhancock@cumin1003> START - Cookbook sre.hosts.provision for host db2249.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART [production]
20:58 <jhancock@cumin1003> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host db2249.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART [production]
20:56 <jhancock@cumin1003> START - Cookbook sre.hosts.provision for host db2249.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART [production]
20:55 <jhancock@cumin1003> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host db2249 [production]
20:54 <jhancock@cumin1003> START - Cookbook sre.network.configure-switch-interfaces for host db2249 [production]
20:50 <jhancock@cumin1003> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
20:50 <jhancock@cumin1003> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: adding db2249 to codfw - jhancock@cumin1003" [production]
20:50 <jhancock@cumin1003> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: adding db2249 to codfw - jhancock@cumin1003" [production]
20:47 <jhancock@cumin1003> START - Cookbook sre.dns.netbox [production]
18:16 <mforns@deploy2002> helmfile [codfw] DONE helmfile.d/services/page-analytics: apply [production]
18:16 <mforns@deploy2002> helmfile [codfw] START helmfile.d/services/page-analytics: apply [production]
18:16 <mforns@deploy2002> helmfile [eqiad] DONE helmfile.d/services/page-analytics: apply [production]
18:15 <mforns@deploy2002> helmfile [eqiad] START helmfile.d/services/page-analytics: apply [production]
18:15 <mforns@deploy2002> helmfile [staging] DONE helmfile.d/services/page-analytics: apply [production]
18:15 <mforns@deploy2002> helmfile [staging] START helmfile.d/services/page-analytics: apply [production]
17:45 <cscott@deploy2002> Finished scap sync-world: Backport for [[gerrit:1219904|Ensure that user interface language is "used" by postprocessing pipeline (T413227)]] (duration: 09m 07s) [production]
17:41 <cscott@deploy2002> cscott: Continuing with sync [production]
17:38 <cscott@deploy2002> cscott: Backport for [[gerrit:1219904|Ensure that user interface language is "used" by postprocessing pipeline (T413227)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
17:36 <cscott@deploy2002> Started scap sync-world: Backport for [[gerrit:1219904|Ensure that user interface language is "used" by postprocessing pipeline (T413227)]] [production]
17:21 <mforns@deploy2002> helmfile [staging] DONE helmfile.d/services/edit-analytics: apply [production]
17:21 <mforns@deploy2002> helmfile [staging] START helmfile.d/services/edit-analytics: apply [production]
17:09 <btullis@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/analytics-test: apply [production]
17:09 <btullis@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/analytics-test: apply [production]
16:44 <elukey@cumin1003> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host wdqs1029.eqiad.wmnet with OS bookworm [production]
16:18 <ejegg> civicrm upgraded from 878d168c to 4eee8c62 [production]
16:09 <btullis@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/analytics-test: apply [production]
16:09 <btullis@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/analytics-test: apply [production]
15:45 <elukey@cumin1003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wdqs1029.eqiad.wmnet with reason: host reimage [production]
15:41 <elukey@cumin1003> START - Cookbook sre.hosts.downtime for 2:00:00 on wdqs1029.eqiad.wmnet with reason: host reimage [production]
15:21 <elukey> restored the correct puppetserver1001's TLS certificate for puppet following https://phabricator.wikimedia.org/T405580#11214327 [production]