1001-1050 of 10000 results (78ms)
2023-08-23 ยง
15:40 <jiji@deploy1002> helmfile [codfw] DONE helmfile.d/services/tegola-vector-tiles: apply [production]
15:40 <eevans@deploy1002> helmfile [eqiad] START helmfile.d/services/sessionstore: apply [production]
15:39 <eevans@deploy1002> helmfile [codfw] DONE helmfile.d/services/sessionstore: apply [production]
15:39 <eevans@deploy1002> helmfile [codfw] START helmfile.d/services/sessionstore: apply [production]
15:38 <jiji@deploy1002> helmfile [codfw] START helmfile.d/services/tegola-vector-tiles: apply [production]
15:37 <eevans@deploy1002> helmfile [staging] DONE helmfile.d/services/sessionstore: apply [production]
15:37 <eevans@deploy1002> helmfile [staging] START helmfile.d/services/sessionstore: apply [production]
15:35 <eevans@deploy1002> helmfile [eqiad] DONE helmfile.d/services/echostore: apply [production]
15:35 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host testvm2004.codfw.wmnet with OS bookworm [production]
15:34 <eevans@deploy1002> helmfile [eqiad] START helmfile.d/services/echostore: apply [production]
15:33 <ayounsi@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
15:33 <ayounsi@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: esams sandbox - ayounsi@cumin1001" [production]
15:33 <eevans@deploy1002> helmfile [codfw] DONE helmfile.d/services/echostore: apply [production]
15:32 <btullis@cumin1001> END (PASS) - Cookbook sre.druid.reboot-workers (exit_code=0) for Druid analytics cluster: Reboot Druid nodes [production]
15:31 <eevans@deploy1002> helmfile [codfw] START helmfile.d/services/echostore: apply [production]
15:31 <ayounsi@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: esams sandbox - ayounsi@cumin1001" [production]
15:30 <eevans@deploy1002> helmfile [staging] DONE helmfile.d/services/echostore: apply [production]
15:29 <eevans@deploy1002> helmfile [staging] START helmfile.d/services/echostore: apply [production]
15:29 <ayounsi@cumin1001> START - Cookbook sre.dns.netbox [production]
15:21 <brennen@deploy1002> Finished deploy [phabricator/deployment@82e8e76]: update phabricator to phorge (T333885) (duration: 00m 38s) [production]
15:21 <brennen@deploy1002> Started deploy [phabricator/deployment@82e8e76]: update phabricator to phorge (T333885) [production]
15:19 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on testvm2004.codfw.wmnet with reason: host reimage [production]
15:16 <jmm@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on testvm2004.codfw.wmnet with reason: host reimage [production]
15:12 <btullis@cumin1001> END (PASS) - Cookbook sre.hadoop.reboot-workers (exit_code=0) for Hadoop test cluster [production]
15:11 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cuminunpriv1001.eqiad.wmnet [production]
15:11 <brennen@deploy1002> Finished deploy [phabricator/deployment@82e8e76]: update phabricator to phorge (T333885) (duration: 00m 34s) [production]
15:10 <brennen@deploy1002> Started deploy [phabricator/deployment@82e8e76]: update phabricator to phorge (T333885) [production]
15:07 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host cuminunpriv1001.eqiad.wmnet [production]
15:04 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host bast4004.wikimedia.org [production]
15:03 <jelto@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on phab1004.eqiad.wmnet with reason: Switch Phabricator to Phorge [production]
15:02 <jelto@cumin1001> START - Cookbook sre.hosts.downtime for 1:00:00 on phab1004.eqiad.wmnet with reason: Switch Phabricator to Phorge [production]
15:00 <btullis@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host an-launcher1002.eqiad.wmnet [production]
14:59 <jmm@cumin2002> START - Cookbook sre.hosts.reimage for host testvm2004.codfw.wmnet with OS bookworm [production]
14:59 <akosiaris@cumin1001> conftool action : set/pooled=true; selector: dnsdisc=kartotherian,name=codfw [production]
14:59 <akosiaris> pool kartotherian in codfw for testing T344324 [production]
14:58 <akosiaris@deploy1002> helmfile [codfw] DONE helmfile.d/services/tegola-vector-tiles: apply [production]
14:58 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host bast4004.wikimedia.org [production]
14:58 <akosiaris@deploy1002> helmfile [codfw] START helmfile.d/services/tegola-vector-tiles: apply [production]
14:57 <stevemunene@cumin1001> START - Cookbook sre.hosts.reimage for host an-worker1117.eqiad.wmnet with OS bullseye [production]
14:57 <akosiaris> deploy codfw tegola-vector-tiles with high CPU limits to rule out a hunch. T344324 [production]
14:56 <akosiaris@deploy1002> helmfile [codfw] START helmfile.d/services/tegola-vector-tiles: apply [production]
14:50 <btullis@cumin1001> START - Cookbook sre.hosts.reboot-single for host an-launcher1002.eqiad.wmnet [production]
14:48 <btullis@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host datahubsearch1003.eqiad.wmnet [production]
14:44 <btullis@cumin1001> START - Cookbook sre.hosts.reboot-single for host datahubsearch1003.eqiad.wmnet [production]
14:41 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.reboot-vm (exit_code=0) for VM testvm2002.codfw.wmnet [production]
14:37 <jmm@cumin2002> START - Cookbook sre.ganeti.reboot-vm for VM testvm2002.codfw.wmnet [production]
14:36 <btullis@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host datahubsearch1002.eqiad.wmnet [production]
14:34 <akosiaris@cumin1001> conftool action : set/pooled=false; selector: dnsdisc=kartotherian,name=codfw [production]
14:34 <akosiaris> depool again kartotherian in codfw for testing T344324 [production]
14:34 <vgutierrez@cumin1001> END (PASS) - Cookbook sre.cdn.roll-upgrade-haproxy (exit_code=0) rolling upgrade of HAProxy on P{cp4052.*,cp5032.*} and A:cp [production]