5751-5800 of 10000 results (101ms)
2024-03-27 ยง
16:28 <denisse@cumin2002> END (PASS) - Cookbook sre.puppet.migrate-host (exit_code=0) for host alert2001.wikimedia.org [production]
16:22 <denisse@cumin2002> START - Cookbook sre.puppet.migrate-host for host alert2001.wikimedia.org [production]
16:21 <denisse@cumin2002> END (FAIL) - Cookbook sre.puppet.migrate-host (exit_code=99) for host alert2001.wikimedia.org [production]
16:21 <denisse@cumin2002> START - Cookbook sre.puppet.migrate-host for host alert2001.wikimedia.org [production]
16:12 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 22:00:00 on db[2115,2215].codfw.wmnet with reason: Downtime for analysis [production]
16:12 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 22:00:00 on db[2115,2215].codfw.wmnet with reason: Downtime for analysis [production]
16:10 <jayme@deploy1002> helmfile [eqiad] DONE helmfile.d/services/changeprop-jobqueue: apply [production]
16:09 <jayme@deploy1002> helmfile [eqiad] START helmfile.d/services/changeprop-jobqueue: apply [production]
16:08 <jayme@deploy1002> helmfile [codfw] DONE helmfile.d/services/changeprop-jobqueue: apply [production]
16:07 <jayme@deploy1002> helmfile [codfw] START helmfile.d/services/changeprop-jobqueue: apply [production]
16:06 <jayme@deploy1002> helmfile [staging] DONE helmfile.d/services/changeprop-jobqueue: apply [production]
16:05 <jayme@deploy1002> helmfile [staging] START helmfile.d/services/changeprop-jobqueue: apply [production]
15:55 <inflatador> bking@cumin2002 running puppet against A:wdqs-main to apply nginx changes T360993 [production]
15:53 <jayme@deploy1002> helmfile [codfw] DONE helmfile.d/services/changeprop: apply [production]
15:53 <jayme@deploy1002> helmfile [codfw] START helmfile.d/services/changeprop: apply [production]
15:51 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12 days, 0:00:00 on elastic2038.codfw.wmnet with reason: T358882 [production]
15:51 <bking@cumin2002> START - Cookbook sre.hosts.downtime for 12 days, 0:00:00 on elastic2038.codfw.wmnet with reason: T358882 [production]
15:51 <arnaudb@cumin1002> END (ERROR) - Cookbook sre.mysql.clone (exit_code=97) Will create a clone of db2115.codfw.wmnet onto db2215.codfw.wmnet [production]
15:51 <jayme@deploy1002> helmfile [eqiad] DONE helmfile.d/services/changeprop: apply [production]
15:51 <claime> 50% of backend RESTbase traffic to mw-api-int - T358213 [production]
15:50 <jayme@deploy1002> helmfile [eqiad] START helmfile.d/services/changeprop: apply [production]
15:50 <jayme@deploy1002> helmfile [eqiad] DONE helmfile.d/services/changeprop: apply [production]
15:50 <jayme@deploy1002> helmfile [eqiad] START helmfile.d/services/changeprop: apply [production]
15:43 <jayme@deploy1002> helmfile [staging] DONE helmfile.d/services/changeprop: apply [production]
15:43 <jayme@deploy1002> helmfile [staging] START helmfile.d/services/changeprop: apply [production]
15:35 <jnuche@deploy1002> Finished deploy [releng/jenkins-deploy@1a343bf] (releasing): deploying fix for T361084 to all targets (duration: 01m 03s) [production]
15:33 <jnuche@deploy1002> Started deploy [releng/jenkins-deploy@1a343bf] (releasing): deploying fix for T361084 to all targets [production]
15:33 <jnuche@deploy1002> Finished deploy [releng/jenkins-deploy@1a343bf] (releasing): deploying fix for T361084 to all targets (duration: 00m 19s) [production]
15:33 <jnuche@deploy1002> Started deploy [releng/jenkins-deploy@1a343bf] (releasing): deploying fix for T361084 to all targets [production]
15:23 <brouberol@cumin2002> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts an-tool1009.eqiad.wmnet [production]
15:23 <brouberol@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
15:23 <brouberol@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: an-tool1009.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - brouberol@cumin2002" [production]
15:23 <logmsgbot> andrewtavis-wmde@deploy1002 Finished deploy [airflow-dags/wmde@36dee63]: (no justification provided) (duration: 00m 08s) [production]
15:23 <logmsgbot> andrewtavis-wmde@deploy1002 Started deploy [airflow-dags/wmde@36dee63]: (no justification provided) [production]
15:21 <brouberol@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: an-tool1009.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - brouberol@cumin2002" [production]
15:19 <brouberol@cumin2002> START - Cookbook sre.dns.netbox [production]
15:17 <claime> enabling and running puppet on P:restbase - T358213 [production]
15:14 <brouberol@cumin2002> START - Cookbook sre.hosts.decommission for hosts an-tool1009.eqiad.wmnet [production]
15:14 <claime> enabling and running puppet on restbase1035.eqiad.wmnet - T358213 [production]
15:12 <jnuche@deploy1002> Finished deploy [releng/jenkins-deploy@1a343bf] (releasing): testing fix for T361084 (duration: 00m 20s) [production]
15:12 <jnuche@deploy1002> Started deploy [releng/jenkins-deploy@1a343bf] (releasing): testing fix for T361084 [production]
15:11 <claime> enabling and running puppet on restbase2021.codfw.wmnet - T358213 [production]
15:08 <claime> Disabling puppet on P:restbase - T358213 [production]
14:54 <cgoubert@deploy1002> helmfile [codfw] DONE helmfile.d/services/proton: apply [production]
14:52 <cgoubert@deploy1002> helmfile [codfw] START helmfile.d/services/proton: apply [production]
14:45 <effie> Day 8: Pool codfw for user traffic - T357547 [production]
14:42 <jiji@cumin1002> END (PASS) - Cookbook sre.discovery.datacenter (exit_code=0) status all services in all: None - None [production]
14:42 <jiji@cumin1002> START - Cookbook sre.discovery.datacenter status all services in all: None - None [production]
14:40 <jiji@cumin1002> END (PASS) - Cookbook sre.discovery.datacenter (exit_code=0) pool all active/active services in codfw: Pool active/active services on codfw - T357547 [production]
14:21 <jiji@cumin1002> START - Cookbook sre.discovery.datacenter pool all active/active services in codfw: Pool active/active services on codfw - T357547 [production]