651-700 of 10000 results (73ms)
2023-09-19 ยง
21:47 <jclark@cumin1001> START - Cookbook sre.network.configure-switch-interfaces for host db1232 [production]
21:46 <jclark@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
21:45 <jclark@cumin1001> START - Cookbook sre.dns.netbox [production]
21:41 <brennen> train 1.41.0-wmf.27 (T345888): blockers resolved; rolling to group0 [production]
21:37 <brennen@deploy2002> Finished scap: Backport for [[gerrit:959013|Disable client preferences by default (T345363)]] (duration: 40m 45s) [production]
21:37 <jclark@cumin1001> END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host db1232 [production]
21:37 <jclark@cumin1001> START - Cookbook sre.network.configure-switch-interfaces for host db1232 [production]
21:36 <jclark@cumin1001> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host db1233 [production]
21:36 <jclark@cumin1001> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host db1231 [production]
21:35 <jclark@cumin1001> START - Cookbook sre.network.configure-switch-interfaces for host db1233 [production]
21:34 <jclark@cumin1001> END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host db1232 [production]
21:34 <jclark@cumin1001> START - Cookbook sre.network.configure-switch-interfaces for host db1232 [production]
21:34 <jclark@cumin1001> START - Cookbook sre.network.configure-switch-interfaces for host db1231 [production]
21:33 <cmooney@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
21:32 <jclark@cumin1001> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host db1230 [production]
21:32 <jclark@cumin1001> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host db1229 [production]
21:32 <cmooney@cumin1001> START - Cookbook sre.dns.netbox [production]
21:32 <cmooney@cumin1001> END (FAIL) - Cookbook sre.dns.netbox (exit_code=99) [production]
21:31 <jclark@cumin1001> START - Cookbook sre.network.configure-switch-interfaces for host db1230 [production]
21:31 <jclark@cumin1001> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host db1226 [production]
21:31 <jclark@cumin1001> START - Cookbook sre.network.configure-switch-interfaces for host db1229 [production]
21:31 <jclark@cumin1001> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host db1227 [production]
21:30 <jclark@cumin1001> START - Cookbook sre.network.configure-switch-interfaces for host db1227 [production]
21:30 <jclark@cumin1001> START - Cookbook sre.network.configure-switch-interfaces for host db1226 [production]
21:29 <jclark@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
21:29 <jclark@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update mgmt db12[26-33] - jclark@cumin1001" [production]
21:29 <jclark@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update mgmt db12[26-33] - jclark@cumin1001" [production]
21:26 <jclark@cumin1001> START - Cookbook sre.dns.netbox [production]
21:25 <brennen@deploy2002> jdlrobson and brennen: Continuing with sync [production]
21:20 <jclark@cumin1001> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['cloudelastic1010'] [production]
21:17 <brennen@deploy2002> jdlrobson and brennen: Backport for [[gerrit:959013|Disable client preferences by default (T345363)]] synced to the testservers mwdebug1001.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, and mw-debug kubernetes deployment (accessible via k8s-experimental XWD option) [production]
21:16 <jclark@cumin1001> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['cloudelastic1008'] [production]
21:11 <jclark@cumin1001> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['cloudelastic1010'] [production]
21:11 <jclark@cumin1001> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['cloudelastic1009'] [production]
21:07 <jclark@cumin1001> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['cloudelastic1008'] [production]
21:07 <jclark@cumin1001> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['cloudelastic1007'] [production]
21:01 <jclark@cumin1001> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['cloudelastic1009'] [production]
21:01 <jclark@cumin1001> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host cloudelastic1010.mgmt.eqiad.wmnet with reboot policy FORCED [production]
20:59 <jclark@cumin1001> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['cloudelastic1007'] [production]
20:57 <jclark@cumin1001> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['pki1002'] [production]
20:57 <brennen@deploy2002> Started scap: Backport for [[gerrit:959013|Disable client preferences by default (T345363)]] [production]
20:55 <jclark@cumin1001> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['pki1002'] [production]
20:55 <brennen@deploy2002> Finished scap: Backport for [[gerrit:959010|Fixes cannot read properties of undefined (T342277)]] (duration: 37m 39s) [production]
20:51 <bearloga@deploy2002> Finished deploy [airflow-dags/analytics_product@b603e64]: (no justification provided) (duration: 00m 05s) [production]
20:51 <bearloga@deploy2002> Started deploy [airflow-dags/analytics_product@b603e64]: (no justification provided) [production]
20:50 <bearloga@deploy2002> Finished deploy [airflow-dags/analytics_product@b603e64]: (no justification provided) (duration: 00m 09s) [production]
20:50 <bearloga@deploy2002> Started deploy [airflow-dags/analytics_product@b603e64]: (no justification provided) [production]
20:42 <brennen@deploy2002> jdlrobson and brennen: Continuing with sync [production]
20:38 <brennen@deploy2002> jdlrobson and brennen: Backport for [[gerrit:959010|Fixes cannot read properties of undefined (T342277)]] synced to the testservers mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet, and mw-debug kubernetes deployment (accessible via k8s-experimental XWD option) [production]
20:37 <jclark@cumin1001> START - Cookbook sre.hosts.provision for host cloudelastic1010.mgmt.eqiad.wmnet with reboot policy FORCED [production]