201-250 of 10000 results (42ms)
2026-07-24 §
09:19 <fnegri@cumin1003> START - Cookbook sre.mysql.multiinstance_reboot for 16 hosts [production]
09:16 <klausman@cumin1003> START - Cookbook sre.ganeti.reboot-vm for VM ml-etcd2003.codfw.wmnet [production]
09:16 <klausman@cumin1003> END (PASS) - Cookbook sre.ganeti.reboot-vm (exit_code=0) for VM ml-etcd1003.eqiad.wmnet [production]
09:15 <urbanecm@deploy1003> helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply [production]
09:15 <urbanecm@deploy1003> helmfile [eqiad] START helmfile.d/services/mw-experimental: apply [production]
09:13 <klausman@cumin1003> START - Cookbook sre.ganeti.reboot-vm for VM ml-etcd1003.eqiad.wmnet [production]
09:13 <klausman@cumin1003> END (PASS) - Cookbook sre.ganeti.reboot-vm (exit_code=0) for VM ml-etcd1002.eqiad.wmnet [production]
09:11 <klausman@cumin1003> START - Cookbook sre.ganeti.reboot-vm for VM ml-etcd1002.eqiad.wmnet [production]
09:11 <klausman@cumin1003> END (PASS) - Cookbook sre.ganeti.reboot-vm (exit_code=0) for VM ml-etcd1001.eqiad.wmnet [production]
09:07 <klausman@cumin1003> START - Cookbook sre.ganeti.reboot-vm for VM ml-etcd1001.eqiad.wmnet [production]
08:32 <trueg@deploy1003> helmfile [dse-k8s-codfw] DONE helmfile.d/services/wdqs: apply [production]
08:24 <trueg@deploy1003> helmfile [dse-k8s-codfw] START helmfile.d/services/wdqs: apply [production]
08:16 <trueg@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs: apply [production]
08:16 <trueg@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs: apply [production]
08:07 <brouberol@deploy1003> helmfile [eqiad] DONE helmfile.d/services/wikidata-query-gui: apply [production]
08:07 <brouberol@deploy1003> helmfile [eqiad] START helmfile.d/services/wikidata-query-gui: apply [production]
08:07 <trueg@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs: apply [production]
08:02 <trueg@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs: apply [production]
08:01 <trueg@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/services/wdqs: apply [production]
07:59 <trueg@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/services/wdqs: apply [production]
07:57 <brouberol@deploy1003> helmfile [eqiad] DONE helmfile.d/services/wikidata-query-gui: apply [production]
07:57 <brouberol@deploy1003> helmfile [eqiad] START helmfile.d/services/wikidata-query-gui: apply [production]
07:32 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-test-k8s: apply [production]
07:32 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-test-k8s: apply [production]
07:31 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-test-k8s: apply [production]
07:31 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-test-k8s: apply [production]
06:46 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on clouddb1025.eqiad.wmnet with reason: Cloning [production]
06:46 <marostegui@cumin1003> conftool action : set/pooled=no; selector: name=clouddb1025.eqiad.wmnet,service=s4 [production]
06:45 <marostegui@cumin1003> conftool action : set/pooled=no; selector: name=clouddb1025.eqiad.wmnet,service=s6 [production]
06:44 <marostegui@cumin1003> conftool action : set/pooled=no; selector: name=clouddb1019.eqiad.wmnet,service=s6 [production]
06:44 <marostegui@cumin1003> conftool action : set/pooled=no; selector: name=clouddb1019.eqiad.wmnet,service=s4 [production]
03:40 <ssastry@deploy1003> helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply [production]
03:40 <ssastry@deploy1003> helmfile [codfw] START helmfile.d/services/mw-parsoid: apply [production]
03:40 <ssastry@deploy1003> helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply [production]
03:40 <ssastry@deploy1003> helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply [production]
03:37 <ssastry@deploy1003> helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply [production]
03:37 <ssastry@deploy1003> helmfile [codfw] START helmfile.d/services/mw-parsoid: apply [production]
03:37 <ssastry@deploy1003> helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply [production]
03:36 <ssastry@deploy1003> helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply [production]
02:49 <pt1979@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on mr1-eqsin,mr1-eqsin IPv6 with reason: connection issue [production]
02:38 <pt1979@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on cr[2-3]-eqsin.mgmt,ps1-[603-604]-eqsin with reason: connection issue [production]
02:08 <mwpresync@deploy1003> Finished scap build-images: Publishing wmf/next image (duration: 06m 27s) [production]
02:01 <mwpresync@deploy1003> Started scap build-images: Publishing wmf/next image [production]
2026-07-23 §
23:50 <larssandergreen> civicrm upgraded from bf93abac to 98ef4abd [fundraising]
23:27 <pt1979@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mr1-eqsin.oob,mr1-eqsin.oob IPv6 with reason: switch refresh [production]
22:21 <eevans@cumin1003> END (PASS) - Cookbook sre.cassandra.roll-restart (exit_code=0) for nodes matching A:cassandra-dev: Setting storage compatibility to NONE - eevans@cumin1003 [production]
22:01 <eevans@cumin1003> START - Cookbook sre.cassandra.roll-restart for nodes matching A:cassandra-dev: Setting storage compatibility to NONE - eevans@cumin1003 [production]
21:29 <bking@cumin2003> START - Cookbook sre.wdqs.data-transfer (T430880, restore data on newly-reimaged host) xfer wdqs-all from wdqs1011.eqiad.wmnet -> wdqs1014.eqiad.wmnet, repooling source-only afterwards [production]
21:28 <bking@deploy1003> Finished deploy [wdqs/wdqs@e8fb00c]: T430880 (duration: 00m 46s) [production]
21:28 <bking@deploy1003> Started deploy [wdqs/wdqs@e8fb00c]: T430880 [production]