9301-9350 of 10000 results (108ms)
2023-06-16 ยง
12:02 <hnowlan@deploy2002> helmfile [codfw] DONE helmfile.d/services/thumbor: apply [production]
12:00 <hnowlan@deploy2002> helmfile [codfw] START helmfile.d/services/thumbor: apply [production]
12:00 <hnowlan@deploy2002> helmfile [eqiad] DONE helmfile.d/admin 'apply'. [production]
11:59 <hnowlan@deploy2002> helmfile [eqiad] START helmfile.d/admin 'apply'. [production]
11:56 <hnowlan@deploy2002> helmfile [codfw] DONE helmfile.d/admin 'apply'. [production]
11:55 <hnowlan@deploy2002> helmfile [codfw] START helmfile.d/admin 'apply'. [production]
11:53 <aborrero@cumin1001> END (FAIL) - Cookbook sre.dns.netbox (exit_code=99) [production]
11:53 <hnowlan@deploy2002> helmfile [staging-codfw] DONE helmfile.d/admin 'apply'. [production]
11:53 <hnowlan@deploy2002> helmfile [staging-codfw] START helmfile.d/admin 'apply'. [production]
11:50 <aborrero@cumin1001> START - Cookbook sre.dns.netbox [production]
11:47 <hnowlan@deploy2002> helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. [production]
11:46 <hnowlan@deploy2002> helmfile [staging-eqiad] START helmfile.d/admin 'apply'. [production]
11:21 <btullis@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host an-test-worker1002.eqiad.wmnet [production]
11:15 <hnowlan@deploy1002> helmfile [staging] DONE helmfile.d/services/api-gateway: apply [production]
11:14 <hnowlan@deploy1002> helmfile [staging] START helmfile.d/services/api-gateway: apply [production]
11:14 <btullis@cumin1001> START - Cookbook sre.hosts.reboot-single for host an-test-worker1002.eqiad.wmnet [production]
10:38 <Amir1> root@cumin1001:/home/ladsgroup/software2/dbtools# cat s1.dblist | grep -v "#" | while read db; do cat tables_to_check.txt | while read table index; do echo "$db.$table"; db-compare $db $table $index db1135.eqiad.wmnet:3306 db1118 db1139:3311 || break 2; done ; done (T338354) [production]
09:03 <akosiaris@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
09:02 <akosiaris@cumin1001> START - Cookbook sre.dns.netbox [production]
08:47 <klausman@deploy1002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'articletopic-outlink' for release 'main' . [production]
08:41 <klausman@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'articletopic-outlink' for release 'main' . [production]
08:35 <klausman@deploy1002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'articletopic-outlink' for release 'main' . [production]
08:25 <akosiaris@cumin1001> conftool action : set/pooled=inactive; selector: name=parse1002.eqiad.wmnet [production]
08:19 <hashar@deploy1002> Finished scap: Backport for [[gerrit:930692|Revert "Structured tasks: Fix toolbar rewriting" (T339292 T338934)]] (duration: 21m 08s) [production]
08:00 <hashar@deploy1002> hashar: Backport for [[gerrit:930692|Revert "Structured tasks: Fix toolbar rewriting" (T339292 T338934)]] synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet [production]
07:58 <hashar@deploy1002> Started scap: Backport for [[gerrit:930692|Revert "Structured tasks: Fix toolbar rewriting" (T339292 T338934)]] [production]
07:50 <vgutierrez@cumin1001> END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for acmechief2001.codfw.wmnet [production]
07:50 <vgutierrez@cumin1001> START - Cookbook sre.hosts.remove-downtime for acmechief2001.codfw.wmnet [production]
01:50 <jhancock@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['an-worker1150'] [production]
01:50 <jhancock@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['an-worker1150'] [production]
01:36 <pt1979@cumin2002> END (PASS) - Cookbook sre.network.provision (exit_code=0) for device lsw1-b8-codfw.mgmt.codfw.wmnet [production]
01:36 <pt1979@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
01:36 <pt1979@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add management record for lsw1-b8-codfw - pt1979@cumin2002" [production]
01:35 <pt1979@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add management record for lsw1-b8-codfw - pt1979@cumin2002" [production]
01:31 <pt1979@cumin2002> START - Cookbook sre.dns.netbox [production]
01:31 <pt1979@cumin2002> START - Cookbook sre.network.provision for device lsw1-b8-codfw.mgmt.codfw.wmnet [production]
01:29 <pt1979@cumin2002> END (PASS) - Cookbook sre.network.provision (exit_code=0) for device lsw1-b7-codfw.mgmt.codfw.wmnet [production]
01:28 <pt1979@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
01:28 <pt1979@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add management record for lsw1-b7-codfw - pt1979@cumin2002" [production]
01:27 <pt1979@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add management record for lsw1-b7-codfw - pt1979@cumin2002" [production]
01:24 <pt1979@cumin2002> START - Cookbook sre.dns.netbox [production]
01:24 <pt1979@cumin2002> START - Cookbook sre.network.provision for device lsw1-b7-codfw.mgmt.codfw.wmnet [production]
01:22 <pt1979@cumin2002> END (PASS) - Cookbook sre.network.provision (exit_code=0) for device lsw1-b6-codfw.mgmt.codfw.wmnet [production]
01:21 <pt1979@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
01:21 <pt1979@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add management record for lsw1-b6-codfw - pt1979@cumin2002" [production]
01:20 <pt1979@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add management record for lsw1-b6-codfw - pt1979@cumin2002" [production]
01:17 <pt1979@cumin2002> START - Cookbook sre.dns.netbox [production]
01:17 <pt1979@cumin2002> START - Cookbook sre.network.provision for device lsw1-b6-codfw.mgmt.codfw.wmnet [production]
01:16 <pt1979@cumin2002> END (PASS) - Cookbook sre.network.provision (exit_code=0) for device lsw1-b5-codfw.mgmt.codfw.wmnet [production]
01:15 <pt1979@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]