1001-1050 of 10000 results (66ms)
2023-01-24 ยง
16:22 <jiji@deploy1002> helmfile [codfw] DONE helmfile.d/services/tegola-vector-tiles: apply [production]
16:22 <jiji@deploy1002> helmfile [codfw] START helmfile.d/services/tegola-vector-tiles: apply [production]
16:12 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mc2042.codfw.wmnet with reason: host reimage [production]
16:10 <hnowlan@deploy1002> helmfile [staging] DONE helmfile.d/services/changeprop: apply [production]
16:10 <hnowlan@deploy1002> helmfile [staging] START helmfile.d/services/changeprop: apply [production]
16:09 <jiji@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mc2042.codfw.wmnet with reason: host reimage [production]
15:54 <btullis@deploy1002> helmfile [staging] DONE helmfile.d/services/datahub: sync on main [production]
15:53 <jiji@cumin1001> START - Cookbook sre.hosts.reimage for host mc2042.codfw.wmnet with OS bullseye [production]
15:43 <btullis@deploy1002> helmfile [staging] START helmfile.d/services/datahub: apply on main [production]
15:31 <btullis@deploy1002> helmfile [staging] DONE helmfile.d/services/datahub: sync on main [production]
15:30 <btullis@deploy1002> helmfile [staging] START helmfile.d/services/datahub: sync on main [production]
15:26 <btullis@deploy1002> helmfile [staging] DONE helmfile.d/services/datahub: sync on main [production]
15:17 <jgiannelos@deploy1002> Finished deploy [kartotherian/deploy@5c58f8f] (codfw): Disable traffic mirroring from codfw to eqiad (duration: 01m 40s) [production]
15:15 <jgiannelos@deploy1002> Started deploy [kartotherian/deploy@5c58f8f] (codfw): Disable traffic mirroring from codfw to eqiad [production]
15:12 <jgiannelos@deploy1002> Finished deploy [kartotherian/deploy@15e6aa7] (codfw): Revert "codfw: Disable traffic mirroring" (duration: 00m 33s) [production]
15:11 <jgiannelos@deploy1002> Started deploy [kartotherian/deploy@15e6aa7] (codfw): Revert "codfw: Disable traffic mirroring" [production]
14:58 <btullis@deploy1002> helmfile [staging] START helmfile.d/services/datahub: apply on main [production]
14:58 <btullis@deploy1002> helmfile [staging] DONE helmfile.d/services/datahub: sync on main [production]
14:57 <btullis@deploy1002> helmfile [staging] START helmfile.d/services/datahub: sync on main [production]
14:55 <jclark@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - jclark@cumin1001" [production]
14:52 <btullis@deploy1002> helmfile [staging] DONE helmfile.d/services/datahub: sync on main [production]
14:52 <btullis@deploy1002> helmfile [staging] START helmfile.d/services/datahub: sync on main [production]
14:51 <btullis@deploy1002> helmfile [staging] DONE helmfile.d/services/datahub: sync on main [production]
14:41 <btullis@deploy1002> helmfile [staging] START helmfile.d/services/datahub: apply on main [production]
14:41 <jclark@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on druid1010.eqiad.wmnet with reason: host reimage [production]
14:39 <jiji@cumin1001> conftool action : set/pooled=false; selector: dnsdisc=kartotherian,name=eqiad [production]
14:38 <jclark@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on druid1010.eqiad.wmnet with reason: host reimage [production]
14:36 <volans@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
14:36 <volans@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Force update after switch upgrade - volans@cumin1001" [production]
14:35 <isaranto@deploy1002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-editquality-reverted' for release 'main' . [production]
14:35 <isaranto@deploy1002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-editquality-goodfaith' for release 'main' . [production]
14:35 <isaranto@deploy1002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-editquality-damaging' for release 'main' . [production]
14:35 <isaranto@deploy1002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-drafttopic' for release 'main' . [production]
14:35 <isaranto@deploy1002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-draftquality' for release 'main' . [production]
14:35 <volans@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Force update after switch upgrade - volans@cumin1001" [production]
14:35 <isaranto@deploy1002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-articletopic' for release 'main' . [production]
14:35 <isaranto@deploy1002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-articlequality' for release 'main' . [production]
14:34 <jiji@cumin1001> conftool action : set/pooled=false; selector: dnsdisc=kartotherian,name=eqiad [production]
14:33 <volans@cumin1001> START - Cookbook sre.dns.netbox [production]
14:29 <jiji@cumin1001> conftool action : set/pooled=true; selector: dnsdisc=kartotherian,name=codfw [production]
14:29 <effie> switch maps (kartotherian) from eqiad to codfw (attempt #2) [production]
14:28 <btullis@deploy1002> helmfile [staging] DONE helmfile.d/services/datahub: sync on main [production]
14:28 <btullis@deploy1002> helmfile [staging] START helmfile.d/services/datahub: apply on main [production]
14:25 <TheresNoTime> close UTC afternoon backport window [production]
14:24 <btullis@deploy1002> helmfile [staging] DONE helmfile.d/services/datahub: sync on main [production]
14:20 <XioNoX> repool ulsfo (maintenance over) [production]
14:20 <jclark@cumin1001> START - Cookbook sre.hosts.reimage for host druid1010.eqiad.wmnet with OS bullseye [production]
14:17 <samtar@deploy1002> Finished scap: Backport for [[gerrit:868127|Increase PC writes from parsoid API to 10% (T320534)]] (duration: 07m 41s) [production]
14:14 <btullis@deploy1002> helmfile [staging] START helmfile.d/services/datahub: apply on main [production]
14:11 <samtar@deploy1002> daniel and samtar: Backport for [[gerrit:868127|Increase PC writes from parsoid API to 10% (T320534)]] synced to the testservers: mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet [production]