551-600 of 10000 results (119ms)
2025-04-01 ยง
10:43 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on maps-test2002.codfw.wmnet with reason: host reimage [production]
10:40 <jmm@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on maps-test2002.codfw.wmnet with reason: host reimage [production]
10:36 <ladsgroup@deploy1003> Started scap sync-world: Backport for [[gerrit:1133087|Bump thumbnail steps to 55% (T360589)]] [production]
10:33 <jiji@cumin1002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host mc-gp2004.codfw.wmnet [production]
10:33 <jiji@cumin1002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host mc-gp1005.eqiad.wmnet [production]
10:27 <jiji@cumin1002> START - Cookbook sre.hosts.reboot-single for host mc-gp2004.codfw.wmnet [production]
10:26 <jiji@cumin1002> START - Cookbook sre.hosts.reboot-single for host mc-gp1005.eqiad.wmnet [production]
10:25 <akosiaris@deploy1003> Finished scap sync-world: Backport for [[gerrit:1133069|typos: Add wnmet as a typo]] (duration: 29m 34s) [production]
10:24 <jiji@cumin1002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host mc-gp1004.eqiad.wmnet [production]
10:20 <jmm@cumin2002> START - Cookbook sre.hosts.reimage for host maps-test2002.codfw.wmnet with OS bookworm [production]
10:19 <jiji@cumin1002> END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=99) for host mc-gp2004.codfw.wmnet [production]
10:19 <jiji@cumin1002> START - Cookbook sre.hosts.reboot-single for host mc-gp2004.codfw.wmnet [production]
10:19 <aqu@deploy1003> Finished deploy [airflow-dags/analytics@d96f732]: Update artifacts for analytics (duration: 00m 59s) [production]
10:18 <aqu@deploy1003> Started deploy [airflow-dags/analytics@d96f732]: Update artifacts for analytics [production]
10:17 <jiji@cumin1002> START - Cookbook sre.hosts.reboot-single for host mc-gp1004.eqiad.wmnet [production]
10:17 <aqu@deploy1003> Finished deploy [airflow-dags/analytics_test@d96f732]: Update artifacts for analytics_test (duration: 00m 12s) [production]
10:17 <aqu@deploy1003> Started deploy [airflow-dags/analytics_test@d96f732]: Update artifacts for analytics_test [production]
10:17 <jiji@cumin1002> END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=99) for host mc-gp1004.eqiad.wmnet [production]
10:16 <jiji@cumin1002> START - Cookbook sre.hosts.reboot-single for host mc-gp1004.eqiad.wmnet [production]
10:16 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host maps-test2001.codfw.wmnet with OS bookworm [production]
10:09 <akosiaris@deploy1003> akosiaris: Continuing with sync [production]
10:08 <akosiaris@deploy1003> akosiaris: Backport for [[gerrit:1133069|typos: Add wnmet as a typo]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
10:00 <joal@deploy1003> Finished deploy [analytics/refinery@efc4808] (hadoop-test): Analytics webrequest migration TEST [analytics/refinery@efc48089] (duration: 00m 40s) [production]
09:59 <joal@deploy1003> Started deploy [analytics/refinery@efc4808] (hadoop-test): Analytics webrequest migration TEST [analytics/refinery@efc48089] [production]
09:59 <joal@deploy1003> Finished deploy [analytics/refinery@efc4808] (thin): Analytics webrequest migration THIN [analytics/refinery@efc48089] (duration: 00m 55s) [production]
09:58 <joal@deploy1003> Started deploy [analytics/refinery@efc4808] (thin): Analytics webrequest migration THIN [analytics/refinery@efc48089] [production]
09:57 <joal@deploy1003> Finished deploy [analytics/refinery@efc4808]: Analytics webrequest migration [analytics/refinery@efc48089] (duration: 02m 24s) [production]
09:57 <moritzm> installing freetype security updates [production]
09:56 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on maps-test2001.codfw.wmnet with reason: host reimage [production]
09:55 <akosiaris@deploy1003> Started scap sync-world: Backport for [[gerrit:1133069|typos: Add wnmet as a typo]] [production]
09:55 <akosiaris> scap backport a noop change https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/1133069 for T390251 [production]
09:55 <joal@deploy1003> Started deploy [analytics/refinery@efc4808]: Analytics webrequest migration [analytics/refinery@efc48089] [production]
09:52 <jmm@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on maps-test2001.codfw.wmnet with reason: host reimage [production]
09:50 <elukey> restart nginx on registry* to pick up the debug changes [production]
09:42 <volans@cumin2002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:05:00 on sretest1001.eqiad.wmnet with reason: test [production]
09:39 <gmodena@deploy1003> Finished deploy [airflow-dags/search@ed0fc78]: Deploy mjolnir-2.7.0.dev.conda.tgz (duration: 01m 29s) [production]
09:38 <gmodena@deploy1003> Started deploy [airflow-dags/search@ed0fc78]: Deploy mjolnir-2.7.0.dev.conda.tgz [production]
09:32 <jmm@cumin2002> START - Cookbook sre.hosts.reimage for host maps-test2001.codfw.wmnet with OS bookworm [production]
09:27 <ayounsi@cumin1002> END (ERROR) - Cookbook sre.network.tls (exit_code=97) for network device mr1-ulsfo [production]
09:26 <ayounsi@cumin1002> START - Cookbook sre.network.tls for network device mr1-ulsfo [production]
09:20 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-test-k8s: apply [production]
09:19 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-test-k8s: apply [production]
09:00 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti4008.ulsfo.wmnet [production]
08:59 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti4008.ulsfo.wmnet [production]
08:58 <dcausse@deploy1003> Finished deploy [wdqs/wdqs@354b5ac]: revert T326311, deletion query way too slow (duration: 12m 15s) [production]
08:56 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti4008.ulsfo.wmnet [production]
08:50 <hashar@deploy1003> Finished deploy [integration/docroot@5256e19]: build: Updating eslint-config-wikimedia to 0.29.1 (duration: 00m 09s) [production]
08:50 <hashar@deploy1003> Started deploy [integration/docroot@5256e19]: build: Updating eslint-config-wikimedia to 0.29.1 [production]
08:46 <ayounsi@cumin1002> END (PASS) - Cookbook sre.network.tls (exit_code=0) for network device msw1-eqiad [production]
08:46 <topranks> Drain Lumen cct from codfw to ulsfo due to instability T390660 [production]