5601-5650 of 10000 results (86ms)
2022-12-07 ยง
12:15 <oblivian@deploy1002> helmfile [codfw] DONE helmfile.d/services/shellbox-syntaxhighlight: apply [production]
12:14 <volans@cumin1001> END (FAIL) - Cookbook sre.hosts.downtime (exit_code=99) for 4:00:00 on cloudcumin2001.codfw.wmnet with reason: First installation [production]
12:14 <oblivian@deploy1002> helmfile [codfw] START helmfile.d/services/shellbox-syntaxhighlight: apply [production]
12:14 <oblivian@deploy1002> helmfile [eqiad] DONE helmfile.d/services/shellbox-syntaxhighlight: apply [production]
12:13 <oblivian@deploy1002> helmfile [eqiad] START helmfile.d/services/shellbox-syntaxhighlight: apply [production]
12:12 <oblivian@deploy1002> helmfile [staging] DONE helmfile.d/services/shellbox-syntaxhighlight: apply [production]
12:12 <oblivian@deploy1002> helmfile [staging] START helmfile.d/services/shellbox-syntaxhighlight: apply [production]
12:11 <oblivian@deploy1002> helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply [production]
12:11 <oblivian@deploy1002> helmfile [eqiad] START helmfile.d/services/shellbox-media: apply [production]
12:10 <volans@cumin1001> START - Cookbook sre.hosts.downtime for 4:00:00 on cloudcumin2001.codfw.wmnet with reason: First installation [production]
12:10 <oblivian@deploy1002> helmfile [codfw] DONE helmfile.d/services/shellbox-media: apply [production]
12:10 <oblivian@deploy1002> helmfile [codfw] START helmfile.d/services/shellbox-media: apply [production]
12:09 <oblivian@deploy1002> helmfile [staging] DONE helmfile.d/services/shellbox-media: apply [production]
12:09 <oblivian@deploy1002> helmfile [staging] START helmfile.d/services/shellbox-media: apply [production]
11:59 <oblivian@deploy1002> helmfile [eqiad] DONE helmfile.d/services/shellbox-constraints: apply [production]
11:59 <oblivian@deploy1002> helmfile [eqiad] START helmfile.d/services/shellbox-constraints: apply [production]
11:59 <moritzm> imported rsyslog 8.2102.0-2+deb11u1~buster1 to component/rsyslog-openssl T324623 [production]
11:57 <moritzm> imported librelp 1.10.0-1~buster1 to component/rsyslog-openssl T324623 [production]
11:55 <oblivian@deploy1002> helmfile [codfw] DONE helmfile.d/services/shellbox-constraints: apply [production]
11:55 <sukhe> running authdns-update for Gerrit: 865605 [production]
11:55 <oblivian@deploy1002> helmfile [codfw] START helmfile.d/services/shellbox-constraints: apply [production]
11:55 <oblivian@deploy1002> helmfile [staging] DONE helmfile.d/services/shellbox-constraints: apply [production]
11:54 <oblivian@deploy1002> helmfile [staging] START helmfile.d/services/shellbox-constraints: apply [production]
11:51 <hashar@deploy1002> Finished deploy [integration/docroot@2e0d44b]: Spelling, coobooks -> cookbooks (duration: 00m 14s) [production]
11:50 <hashar@deploy1002> Started deploy [integration/docroot@2e0d44b]: Spelling, coobooks -> cookbooks [production]
11:40 <isaranto@deploy1002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-drafttopic' for release 'main' . [production]
11:33 <isaranto@deploy1002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'revscoring-drafttopic' for release 'main' . [production]
11:30 <elukey@deploy1002> helmfile [ml-staging-codfw] DONE helmfile.d/admin 'sync'. [production]
11:29 <elukey@deploy1002> helmfile [ml-staging-codfw] START helmfile.d/admin 'sync'. [production]
11:11 <volans@cumin2002> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host cloudcumin2001.codfw.wmnet [production]
11:06 <volans@cumin2002> END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) cloudcumin2001.codfw.wmnet on all recursors [production]
11:06 <volans@cumin2002> START - Cookbook sre.dns.wipe-cache cloudcumin2001.codfw.wmnet on all recursors [production]
11:06 <volans@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
11:06 <volans@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM cloudcumin2001.codfw.wmnet - volans@cumin2002" [production]
11:05 <volans@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM cloudcumin2001.codfw.wmnet - volans@cumin2002" [production]
11:03 <hnowlan@deploy1002> helmfile [codfw] DONE helmfile.d/services/thumbor: apply [production]
11:03 <hnowlan@deploy1002> helmfile [codfw] START helmfile.d/services/thumbor: apply [production]
11:01 <volans@cumin2002> START - Cookbook sre.dns.netbox [production]
11:01 <volans@cumin2002> START - Cookbook sre.ganeti.makevm for new host cloudcumin2001.codfw.wmnet [production]
10:58 <volans@cumin1001> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host cloudcumin1001.eqiad.wmnet [production]
10:53 <volans@cumin1001> END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) cloudcumin1001.eqiad.wmnet on all recursors [production]
10:53 <volans@cumin1001> START - Cookbook sre.dns.wipe-cache cloudcumin1001.eqiad.wmnet on all recursors [production]
10:53 <volans@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
10:53 <volans@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM cloudcumin1001.eqiad.wmnet - volans@cumin1001" [production]
10:52 <volans@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM cloudcumin1001.eqiad.wmnet - volans@cumin1001" [production]
10:50 <volans@cumin1001> START - Cookbook sre.dns.netbox [production]
10:50 <volans@cumin1001> START - Cookbook sre.ganeti.makevm for new host cloudcumin1001.eqiad.wmnet [production]
10:35 <jiji@deploy1002> Finished scap: Backport for [[gerrit:865119|ProductionServices: Use redis_misc servers for LockManager (3/6) (T267581)]] (duration: 10m 29s) [production]
10:27 <jiji@deploy1002> jiji and jiji: Backport for [[gerrit:865119|ProductionServices: Use redis_misc servers for LockManager (3/6) (T267581)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet [production]
10:26 <claime> rebooted contin1001.eqiad.wmnet [production]