8401-8450 of 10000 results (104ms)
2023-03-29 ยง
16:30 <btullis@cumin1001> END (PASS) - Cookbook sre.wikireplicas.add-wiki (exit_code=0) [production]
16:29 <btullis@cumin1001> Added views for new wiki: anpwiki T332458 [production]
16:05 <btullis@cumin1001> START - Cookbook sre.wikireplicas.add-wiki [production]
16:00 <hnowlan@deploy2002> helmfile [eqiad] DONE helmfile.d/services/thumbor: apply [production]
16:00 <pt1979@cumin2002> START - Cookbook sre.hosts.provision for host db1207.mgmt.eqiad.wmnet with reboot policy FORCED [production]
15:59 <pt1979@cumin2002> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host db1207.mgmt.eqiad.wmnet with reboot policy FORCED [production]
15:58 <hnowlan@deploy2002> helmfile [eqiad] START helmfile.d/services/thumbor: apply [production]
15:51 <btullis@cumin1001> END (PASS) - Cookbook sre.wikireplicas.add-wiki (exit_code=0) [production]
15:51 <btullis@cumin1001> Added views for new wiki: gucwiki T326235 [production]
15:50 <hnowlan@deploy2002> helmfile [codfw] DONE helmfile.d/services/thumbor: apply [production]
15:50 <hnowlan@deploy2002> helmfile [codfw] START helmfile.d/services/thumbor: apply [production]
15:47 <sukhe@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on lvs4010.ulsfo.wmnet with reason: host reimage [production]
15:44 <sukhe@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on lvs4010.ulsfo.wmnet with reason: host reimage [production]
15:29 <elukey@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'sync'. [production]
15:29 <elukey@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/admin 'sync'. [production]
15:28 <elukey@deploy2002> helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'sync'. [production]
15:27 <elukey@deploy2002> helmfile [ml-serve-eqiad] START helmfile.d/admin 'sync'. [production]
15:27 <elukey@deploy2002> helmfile [ml-serve-codfw] DONE helmfile.d/admin 'sync'. [production]
15:27 <sukhe@cumin2002> START - Cookbook sre.hosts.reimage for host lvs4010.ulsfo.wmnet with OS bullseye [production]
15:27 <elukey@deploy2002> helmfile [ml-serve-codfw] START helmfile.d/admin 'sync'. [production]
15:26 <btullis@cumin1001> START - Cookbook sre.wikireplicas.add-wiki [production]
15:24 <gmodena@deploy1002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mediawiki-page-content-change-enrichment: apply [production]
15:24 <gmodena@deploy1002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mediawiki-page-content-change-enrichment: apply [production]
15:23 <gmodena@deploy1002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mediawiki-page-content-change-enrichment: apply [production]
15:23 <gmodena@deploy1002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mediawiki-page-content-change-enrichment: apply [production]
15:07 <hnowlan@deploy2002> helmfile [codfw] DONE helmfile.d/services/thumbor: apply [production]
15:06 <elukey@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on kafka-main2001.codfw.wmnet with reason: Stop kafka, dist-upgrade [production]
15:06 <elukey@cumin1001> START - Cookbook sre.hosts.downtime for 1:00:00 on kafka-main2001.codfw.wmnet with reason: Stop kafka, dist-upgrade [production]
15:03 <gmodena@deploy1002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mediawiki-page-content-change-enrichment: apply [production]
15:03 <gmodena@deploy1002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mediawiki-page-content-change-enrichment: apply [production]
15:01 <jgleeson> SmashPig upgraded from 758a34c1 to 240c80a2 [production]
15:00 <ebernhardson@deploy2002> Finished deploy [airflow-dags/search@4a7a6cc]: prefix hive properties with spark.hive. (duration: 00m 13s) [production]
15:00 <ebernhardson@deploy2002> Started deploy [airflow-dags/search@4a7a6cc]: prefix hive properties with spark.hive. [production]
14:59 <cmooney@cumin1001> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudcephmon2005-dev [production]
14:58 <cmooney@cumin1001> START - Cookbook sre.network.configure-switch-interfaces for host cloudcephmon2005-dev [production]
14:57 <cmooney@cumin1001> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudcephmon2005-dev [production]
14:57 <cmooney@cumin1001> START - Cookbook sre.network.configure-switch-interfaces for host cloudcephmon2005-dev [production]
14:53 <gmodena@deploy1002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mediawiki-page-content-change-enrichment: apply [production]
14:53 <gmodena@deploy1002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mediawiki-page-content-change-enrichment: apply [production]
14:49 <XioNoX> Remove custom BGP graceful-shutdown on all core routers - T320230 [production]
14:47 <hnowlan@deploy2002> helmfile [codfw] START helmfile.d/services/thumbor: apply [production]
14:35 <pt1979@cumin2002> START - Cookbook sre.hosts.provision for host db1207.mgmt.eqiad.wmnet with reboot policy FORCED [production]
14:34 <pt1979@cumin2002> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host db1207.mgmt.eqiad.wmnet with reboot policy FORCED [production]
14:30 <pt1979@cumin2002> START - Cookbook sre.hosts.provision for host db1207.mgmt.eqiad.wmnet with reboot policy FORCED [production]
14:20 <bking@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/rdf-streaming-updater: apply [production]
14:20 <bking@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/rdf-streaming-updater: apply [production]
14:19 <bking@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/rdf-streaming-updater: apply [production]
14:19 <bking@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/rdf-streaming-updater: apply [production]
14:19 <bking@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/rdf-streaming-updater: apply [production]
14:18 <bking@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/rdf-streaming-updater: apply [production]