|
2023-09-14
ยง
|
| 09:55 |
<jmm@cumin2002> |
END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=1) for host cumin1001.eqiad.wmnet |
[production] |
| 09:52 |
<elukey> |
remove the 'mediawiki.revision-score' stream form eventstreams public API - T342116 |
[production] |
| 09:51 |
<elukey@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/eventstreams: sync |
[production] |
| 09:51 |
<elukey@deploy1002> |
helmfile [codfw] START helmfile.d/services/eventstreams: sync |
[production] |
| 09:50 |
<elukey@deploy1002> |
helmfile [staging] DONE helmfile.d/services/eventstreams: sync |
[production] |
| 09:49 |
<elukey@deploy1002> |
helmfile [staging] START helmfile.d/services/eventstreams: sync |
[production] |
| 09:49 |
<jayme> |
restarted navtiming on webperf2003 to pick up changed etcd service records |
[production] |
| 09:44 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host cumin1001.eqiad.wmnet |
[production] |
| 09:43 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ldap-rw2001.wikimedia.org |
[production] |
| 09:39 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host ldap-rw2001.wikimedia.org |
[production] |
| 09:37 |
<wm-bot> |
<root> webservice restart |
[tools.whois-referral] |
| 09:36 |
<wm-bot> |
<root> webservice restart |
[tools.ipcheck] |
| 09:36 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ldap-rw1001.wikimedia.org |
[production] |
| 09:32 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host ldap-rw1001.wikimedia.org |
[production] |
| 09:22 |
<mvernon@cumin2002> |
END (PASS) - Cookbook sre.swift.roll-restart-reboot-swift-ms-proxies (exit_code=0) rolling restart_daemons on A:swift-fe-eqiad |
[production] |
| 09:18 |
<wm-bot2> |
dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component volume-admission |
[tools] |
| 09:17 |
<wm-bot2> |
dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component volume-admission |
[tools] |
| 09:17 |
<wm-bot2> |
dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component volume-admission |
[toolsbeta] |
| 09:17 |
<mvernon@cumin2002> |
START - Cookbook sre.swift.roll-restart-reboot-swift-ms-proxies rolling restart_daemons on A:swift-fe-eqiad |
[production] |
| 09:16 |
<mvernon@cumin2002> |
END (PASS) - Cookbook sre.swift.roll-restart-reboot-swift-ms-proxies (exit_code=0) rolling restart_daemons on A:swift-fe-codfw |
[production] |
| 09:16 |
<wm-bot2> |
dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component volume-admission |
[toolsbeta] |
| 09:10 |
<mvernon@cumin2002> |
START - Cookbook sre.swift.roll-restart-reboot-swift-ms-proxies rolling restart_daemons on A:swift-fe-codfw |
[production] |
| 09:07 |
<moritzm> |
installing qemu security updates on ganeti-test |
[production] |
| 08:59 |
<btullis> |
running build-production-images on build2001 for T344910 |
[production] |
| 08:54 |
<wm-bot2> |
dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component volume-admission |
[toolsbeta] |
| 08:53 |
<godog> |
+50 to prometheus eqiad k8s-staging |
[production] |
| 08:53 |
<wm-bot2> |
dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component volume-admission |
[toolsbeta] |
| 08:45 |
<jayme> |
restarting confd fleet wide |
[production] |
| 08:45 |
<jmm@cumin2002> |
START - Cookbook sre.aqs.roll-restart-reboot rolling reboot on A:aqs-eqiad |
[production] |
| 08:43 |
<jayme> |
restarting primary lvs in codfw, eqsin, ulsfo |
[production] |
| 08:38 |
<jayme> |
restarted secondary lvs in codfw, eqsin, ulsfo |
[production] |
| 08:24 |
<jnuche@deploy1002> |
rebuilt and synchronized wikiversions files: group2 wikis to 1.41.0-wmf.26 refs T343728 |
[production] |
| 08:10 |
<wm-bot2> |
dcaro@urcuchillay END (PASS) - Cookbook wmcs.toolforge.k8s.component.deploy (exit_code=0) for component registry-admissionNone (T341084) |
[tools] |
| 08:09 |
<wm-bot2> |
dcaro@urcuchillay START - Cookbook wmcs.toolforge.k8s.component.deploy for component registry-admissionNone (T341084) |
[tools] |
| 07:57 |
<jayme@cumin1001> |
END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) _etcd._tcp.eqsin.wmnet on all recursors |
[production] |
| 07:56 |
<jayme@cumin1001> |
START - Cookbook sre.dns.wipe-cache _etcd._tcp.eqsin.wmnet on all recursors |
[production] |
| 07:56 |
<jayme@cumin1001> |
END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) _etcd._tcp.ulsfo.wmnet on all recursors |
[production] |
| 07:56 |
<jayme@cumin1001> |
START - Cookbook sre.dns.wipe-cache _etcd._tcp.ulsfo.wmnet on all recursors |
[production] |
| 07:56 |
<jayme@cumin1001> |
END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) _etcd._tcp.codfw.wmnet on all recursors |
[production] |
| 07:56 |
<jayme@cumin1001> |
START - Cookbook sre.dns.wipe-cache _etcd._tcp.codfw.wmnet on all recursors |
[production] |
| 07:52 |
<wm-bot> |
<jeanfred> Webservice restart |
[tools.wudele] |
| 07:52 |
<wm-bot> |
<valeriobozzolan> Run 'jobs restart itwiki-orphanizerbot' but get a timeout from api.svc.tools.eqiad1.wikimedia.cloud, multiple times. But worked. |
[tools.itwiki] |
| 07:44 |
<jmm@cumin2002> |
END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=1) for host debmonitor2003.codfw.wmnet |
[production] |
| 07:35 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host debmonitor2003.codfw.wmnet |
[production] |
| 07:32 |
<hashar> |
Backport & config deployment window completed. |
[production] |
| 07:22 |
<legoktm> |
$ sudo qdel 2153961 |
[tools.krinklebot] |
| 07:21 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host debmonitor1003.eqiad.wmnet |
[production] |
| 07:17 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host debmonitor1003.eqiad.wmnet |
[production] |
| 07:13 |
<kartik@deploy1002> |
Finished scap: Backport for [[gerrit:956807|Enable MinT translation service on MediaWiki - rollout #4 (T341445)]] (duration: 10m 17s) |
[production] |
| 07:09 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host apt1001.wikimedia.org |
[production] |