|
2026-10-29
ยง
|
| 10:13 |
<bwojtowicz@deploy1003> |
helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-editquality-reverted' for release 'main' . |
[production] |
| 10:12 |
<bwojtowicz@deploy1003> |
helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-editquality-reverted' for release 'main' . |
[production] |
| 10:11 |
<bwojtowicz@deploy1003> |
helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-editquality-reverted' for release 'main' . |
[production] |
| 10:08 |
<bwojtowicz@deploy1003> |
helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-editquality-goodfaith' for release 'main' . |
[production] |
| 10:07 |
<vgutierrez@cumin1004> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on lvs6002.drmrs.wmnet with reason: host reimage |
[production] |
| 10:05 |
<bwojtowicz@deploy1003> |
helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-editquality-goodfaith' for release 'main' . |
[production] |
| 10:04 |
<bwojtowicz@deploy1003> |
helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-editquality-goodfaith' for release 'main' . |
[production] |
| 10:04 |
<vgutierrez@cumin1004> |
START - Cookbook sre.hosts.downtime for 2:00:00 on lvs6002.drmrs.wmnet with reason: host reimage |
[production] |
| 10:01 |
<vgutierrez@cumin1004> |
END (PASS) - Cookbook sre.hosts.dhcp (exit_code=0) for host lvs6002.drmrs.wmnet (remove config only) |
[production] |
| 10:01 |
<vgutierrez@cumin1004> |
START - Cookbook sre.hosts.dhcp for host lvs6002.drmrs.wmnet (remove config only) |
[production] |
| 09:59 |
<bwojtowicz@deploy1003> |
helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-editquality-damaging' for release 'main' . |
[production] |
| 09:53 |
<bwojtowicz@deploy1003> |
helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-editquality-damaging' for release 'main' . |
[production] |
| 09:52 |
<bwojtowicz@deploy1003> |
helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-editquality-damaging' for release 'main' . |
[production] |
| 09:49 |
<bwojtowicz@deploy1003> |
helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-drafttopic' for release 'main' . |
[production] |
| 09:48 |
<bwojtowicz@deploy1003> |
helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-drafttopic' for release 'main' . |
[production] |
| 09:47 |
<vgutierrez@cumin1004> |
END (PASS) - Cookbook sre.hosts.dhcp (exit_code=0) for host lvs6002.drmrs.wmnet (push config only) |
[production] |
| 09:47 |
<vgutierrez@cumin1004> |
START - Cookbook sre.hosts.dhcp for host lvs6002.drmrs.wmnet (push config only) |
[production] |
| 09:46 |
<vgutierrez@cumin1004> |
START - Cookbook sre.hosts.reimage for host lvs6002.drmrs.wmnet with OS trixie |
[production] |
| 09:45 |
<bwojtowicz@deploy1003> |
helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-drafttopic' for release 'main' . |
[production] |
| 09:41 |
<bwojtowicz@deploy1003> |
helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-draftquality' for release 'main' . |
[production] |
| 09:41 |
<vgutierrez@cumin1004> |
END (PASS) - Cookbook sre.loadbalancer.admin (exit_code=0) config_reloading P{lvs6002.*} and A:liberica |
[production] |
| 09:40 |
<vgutierrez@cumin1004> |
START - Cookbook sre.loadbalancer.admin config_reloading P{lvs6002.*} and A:liberica |
[production] |
| 09:38 |
<bwojtowicz@deploy1003> |
helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-draftquality' for release 'main' . |
[production] |
| 09:37 |
<moritzm> |
installing Bind security updates |
[production] |
| 09:36 |
<bwojtowicz@deploy1003> |
helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-draftquality' for release 'main' . |
[production] |
| 09:34 |
<bwojtowicz@deploy1003> |
helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-articletopic' for release 'main' . |
[production] |
| 09:33 |
<bwojtowicz@deploy1003> |
helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-articletopic' for release 'main' . |
[production] |
| 09:33 |
<dpogorzelski@cumin1004> |
conftool action : set/pooled=true; selector: dnsdisc=k8s-ingress-ml-serve,name=eqiad |
[production] |
| 09:32 |
<dpogorzelski@cumin1004> |
conftool action : set/pooled=true; selector: dnsdisc=inference,name=eqiad |
[production] |
| 09:32 |
<bwojtowicz@deploy1003> |
helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-articletopic' for release 'main' . |
[production] |
| 09:28 |
<bwojtowicz@deploy1003> |
helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-articlequality' for release 'main' . |
[production] |
| 09:28 |
<jiji@cumin2003> |
END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1228: Depooling replica - T442815 |
[production] |
| 09:27 |
<jiji@cumin2003> |
START - Cookbook sre.mysql.depool depool db1228: Depooling replica - T442815 |
[production] |
| 09:25 |
<bwojtowicz@deploy1003> |
helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-articlequality' for release 'main' . |
[production] |
| 09:23 |
<bwojtowicz@deploy1003> |
helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-articlequality' for release 'main' . |
[production] |
| 09:17 |
<moritzm> |
installing ca-certificates updates on Bookworm |
[production] |
| 09:16 |
<vgutierrez@cumin1004> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host lvs6003.drmrs.wmnet with OS trixie |
[production] |
| 08:56 |
<vgutierrez@cumin1004> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on lvs6003.drmrs.wmnet with reason: host reimage |
[production] |
| 08:51 |
<vgutierrez@cumin1004> |
START - Cookbook sre.hosts.downtime for 2:00:00 on lvs6003.drmrs.wmnet with reason: host reimage |
[production] |
| 08:49 |
<vgutierrez@cumin1004> |
END (PASS) - Cookbook sre.hosts.dhcp (exit_code=0) for host lvs6003.drmrs.wmnet (remove config only) |
[production] |
| 08:48 |
<vgutierrez@cumin1004> |
START - Cookbook sre.hosts.dhcp for host lvs6003.drmrs.wmnet (remove config only) |
[production] |
| 08:42 |
<hashar> |
restarting Gerrit |
[production] |
| 08:38 |
<jelto@cumin1004> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:15:00 on 14 hosts with reason: Java upgrade |
[production] |
| 08:38 |
<jelto@cumin1004> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:15:00 on gerrit1003.wikimedia.org with reason: Java upgrade |
[production] |
| 08:35 |
<vgutierrez@cumin1004> |
END (PASS) - Cookbook sre.hosts.dhcp (exit_code=0) for host lvs6003.drmrs.wmnet (push config only) |
[production] |
| 08:35 |
<vgutierrez@cumin1004> |
START - Cookbook sre.hosts.dhcp for host lvs6003.drmrs.wmnet (push config only) |
[production] |
| 08:34 |
<vgutierrez@cumin1004> |
START - Cookbook sre.hosts.reimage for host lvs6003.drmrs.wmnet with OS trixie |
[production] |
| 08:30 |
<hashar> |
Restarted Gerrit on gerrit2002/gerrit2003 (spare/replica) to switch to Java 21 | T392465 |
[production] |
| 08:30 |
<moritzm> |
imported Jenkins 2.580.2 to thirdparty/jenkins for trixie-wikimedia T442030 |
[production] |
| 07:43 |
<kevinbazira@deploy1003> |
helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'experimental' for release 'llm' . |
[production] |