201-250 of 10000 results (73ms)
2023-06-26 §
06:27 <kartik@deploy1002> helmfile [eqiad] DONE helmfile.d/services/cxserver: apply [production]
06:26 <kartik@deploy1002> helmfile [eqiad] START helmfile.d/services/cxserver: apply [production]
06:20 <marostegui@cumin1001> dbctl commit (dc=all): 'Remove db1118 from dbctl T326683', diff saved to https://phabricator.wikimedia.org/P49477 and previous config saved to /var/cache/conftool/dbconfig/20230626-062036-marostegui.json [production]
06:15 <kartik@deploy1002> helmfile [codfw] DONE helmfile.d/services/cxserver: apply [production]
06:14 <kartik@deploy1002> helmfile [codfw] START helmfile.d/services/cxserver: apply [production]
06:11 <kartik@deploy1002> helmfile [staging] DONE helmfile.d/services/cxserver: apply [production]
06:10 <kartik@deploy1002> helmfile [staging] START helmfile.d/services/cxserver: apply [production]
2023-06-25 §
01:45 <andrew@deploy1002> Started deploy [horizon/deploy@9d02cd6]: installing (but not registering) magnum-ui [production]
01:35 <andrew@deploy1002> Finished deploy [horizon/deploy@9d02cd6]: installing (but not registering) magnum-ui (duration: 04m 05s) [production]
01:31 <andrew@deploy1002> Started deploy [horizon/deploy@9d02cd6]: installing (but not registering) magnum-ui [production]
01:30 <andrew@deploy1002> deploy aborted: asdf (duration: 00m 01s) [production]
01:30 <andrew@deploy1002> Started deploy [horizon/deploy@9d02cd6]: asdf [production]
2023-06-23 §
16:33 <btullis@deploy1002> helmfile [staging] DONE helmfile.d/services/datahub: sync on main [production]
16:23 <btullis@deploy1002> helmfile [staging] START helmfile.d/services/datahub: apply on main [production]
16:02 <btullis@deploy1002> helmfile [staging] DONE helmfile.d/services/datahub: sync on main [production]
15:51 <btullis@deploy1002> helmfile [staging] START helmfile.d/services/datahub: apply on main [production]
14:27 <urbanecm@deploy1002> Finished scap: Backport for [[gerrit:932280|Section images: Placeholder should serialize to empty string (T340170)]] (duration: 06m 56s) [production]
14:26 <eevans@cumin1001> END (PASS) - Cookbook sre.discovery.service-route (exit_code=0) pool sessionstore in codfw: maintenance [production]
14:21 <eevans@cumin1001> START - Cookbook sre.discovery.service-route pool sessionstore in codfw: maintenance [production]
14:20 <urbanecm@deploy1002> Started scap: Backport for [[gerrit:932280|Section images: Placeholder should serialize to empty string (T340170)]] [production]
14:12 <vgutierrez@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on parse1002.eqiad.wmnet with reason: HW issues [production]
14:12 <vgutierrez@cumin1001> START - Cookbook sre.hosts.downtime for 7 days, 0:00:00 on parse1002.eqiad.wmnet with reason: HW issues [production]
13:35 <Emperor> update private wiki container ACLs in eqiad-swift [production]
13:30 <Emperor> update private wiki container ACLs in codfw-swift [production]
13:29 <godog> add 200G to prometheus/k8s in eqiad [production]
12:40 <elukey> move varnishkafka drmrs instances to pki [production]
12:10 <Emperor> updating ACLs on wikipedia-office containers T340189 T338765 [production]
11:24 <btullis@deploy1002> helmfile [staging] DONE helmfile.d/services/datahub: sync on main [production]
11:13 <btullis@deploy1002> helmfile [staging] START helmfile.d/services/datahub: apply on main [production]
11:12 <btullis@deploy1002> helmfile [staging] DONE helmfile.d/services/datahub: sync on main [production]
11:02 <btullis@deploy1002> helmfile [staging] START helmfile.d/services/datahub: apply on main [production]
10:27 <btullis@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host an-worker1110.eqiad.wmnet [production]
10:20 <btullis@cumin1001> START - Cookbook sre.hosts.reboot-single for host an-worker1110.eqiad.wmnet [production]
10:12 <moritzm> installing vim security updates [production]
09:26 <moritzm> uploaded openjdk-8 8u372-ga-1~deb10u1 to component/jdk8 (forward port of Java 8 for Buster) [production]
09:20 <btullis@cumin1001> END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=99) for host an-worker1110.eqiad.wmnet [production]
08:48 <elukey@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:30:00 on ml-cache1001.eqiad.wmnet with reason: Working on pki [production]
08:48 <elukey@cumin1001> START - Cookbook sre.hosts.downtime for 0:30:00 on ml-cache1001.eqiad.wmnet with reason: Working on pki [production]
08:37 <btullis@cumin1001> START - Cookbook sre.hosts.reboot-single for host an-worker1110.eqiad.wmnet [production]
05:52 <ayounsi@cumin1001> END (PASS) - Cookbook sre.network.peering (exit_code=0) with action 'configure' for AS: 14860 [production]
05:49 <ayounsi@cumin1001> START - Cookbook sre.network.peering with action 'configure' for AS: 14860 [production]
04:57 <marostegui@cumin1001> dbctl commit (dc=all): 'Depool db1118', diff saved to https://phabricator.wikimedia.org/P49472 and previous config saved to /var/cache/conftool/dbconfig/20230623-045758-root.json [production]
01:19 <bking@cumin1001> END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) [production]
01:15 <bking@cumin1001> START - Cookbook sre.wdqs.data-transfer [production]
2023-06-22 §
21:00 <bking@cumin1001> END (FAIL) - Cookbook sre.wdqs.data-transfer (exit_code=99) [production]
19:41 <dzahn@cumin1001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host phab-test1001.eqiad.wmnet [production]
19:41 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host phab-test1001.eqiad.wmnet with OS buster [production]
19:29 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on phab-test1001.eqiad.wmnet with reason: host reimage [production]
19:26 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on phab-test1001.eqiad.wmnet with reason: host reimage [production]
19:25 <bking@cumin1001> START - Cookbook sre.wdqs.data-transfer [production]