2401-2450 of 10000 results (88ms)
2023-03-30 ยง
17:06 <bd808@deploy2002> helmfile [eqiad] START helmfile.d/services/developer-portal: apply [production]
17:05 <bd808@deploy2002> helmfile [staging] DONE helmfile.d/services/developer-portal: apply [production]
17:04 <bd808@deploy2002> helmfile [staging] START helmfile.d/services/developer-portal: apply [production]
17:04 <pt1979@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['db1215'] [production]
17:04 <pt1979@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['db1213'] [production]
17:01 <pt1979@cumin2002> START - Cookbook sre.hosts.reimage for host db1207.eqiad.wmnet with OS bullseye [production]
17:00 <cmjohnson@cumin1001> START - Cookbook sre.hosts.reimage for host thanos-fe1004.eqiad.wmnet with OS bullseye [production]
16:46 <pt1979@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['db1214'] [production]
16:42 <pt1979@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['db1213'] [production]
16:36 <pt1979@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['db1212'] [production]
16:32 <pt1979@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['db1211'] [production]
16:21 <cmooney@cumin2002> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt2003-dev [production]
16:20 <cmooney@cumin2002> START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt2003-dev [production]
16:20 <cmooney@cumin2002> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudcephmon2004-dev [production]
16:20 <cmooney@cumin2002> START - Cookbook sre.network.configure-switch-interfaces for host cloudcephmon2004-dev [production]
16:19 <cmooney@cumin2002> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudcephosd2003-dev [production]
16:19 <cmooney@cumin2002> START - Cookbook sre.network.configure-switch-interfaces for host cloudcephosd2003-dev [production]
16:10 <pt1979@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['db1212'] [production]
16:09 <pt1979@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['db1211'] [production]
16:09 <pt1979@cumin2002> END (ERROR) - Cookbook sre.hardware.upgrade-firmware (exit_code=97) upgrade firmware for hosts ['db1209'] [production]
16:09 <pt1979@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['db1209'] [production]
16:03 <pt1979@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['db1208'] [production]
16:01 <cmooney@cumin2002> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudcephosd2001-dev [production]
16:01 <cmooney@cumin2002> START - Cookbook sre.network.configure-switch-interfaces for host cloudcephosd2001-dev [production]
16:00 <pt1979@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['db1207'] [production]
15:45 <cstone> SmashPig upgraded from 240c80a2 to e86b0a66 [production]
15:44 <mutante> phabricator maintenance window / deployment ended (T329974) [production]
15:40 <pt1979@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['db1208'] [production]
15:36 <cmooney@cumin2002> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt2001-dev [production]
15:36 <brennen@deploy2002> Finished deploy [phabricator/deployment@9f0866e]: deploy to phab1004 for T333516 (duration: 00m 42s) [production]
15:36 <cmooney@cumin2002> START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt2001-dev [production]
15:35 <pt1979@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['db1207'] [production]
15:35 <brennen@deploy2002> Started deploy [phabricator/deployment@9f0866e]: deploy to phab1004 for T333516 [production]
15:34 <brennen@deploy2002> Finished deploy [phabricator/deployment@9f0866e]: test deploy to phab2002 for T333516 (duration: 00m 30s) [production]
15:34 <volans> upgraded spicerack to v6.4.1 on the cumin hosts [production]
15:34 <brennen@deploy2002> Started deploy [phabricator/deployment@9f0866e]: test deploy to phab2002 for T333516 [production]
15:34 <cmooney@cumin2002> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt2002-dev [production]
15:33 <mutante> phabricator maintenance / deploy window starting [production]
15:33 <cmooney@cumin2002> START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt2002-dev [production]
15:32 <cmooney@cumin2002> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudcephosd2002-dev [production]
15:32 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on phab1004.eqiad.wmnet with reason: maintenance [production]
15:32 <cmooney@cumin2002> START - Cookbook sre.network.configure-switch-interfaces for host cloudcephosd2002-dev [production]
15:32 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 1:00:00 on phab1004.eqiad.wmnet with reason: maintenance [production]
15:32 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on phab2002.codfw.wmnet with reason: maintenance [production]
15:32 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 1:00:00 on phab2002.codfw.wmnet with reason: maintenance [production]
15:30 <volans> uploaded spicerack_6.4.1 to apt.wikimedia.org bullseye-wikimedia [production]
15:14 <lucaswerkmeister-wmde:> Deployed security patch for T333569 [production]
15:08 <lucaswerkmeister-wmde:> Deployed security patch for T333569 [production]
14:59 <gmodena@deploy1002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mediawiki-page-content-change-enrichment: apply [production]
14:59 <gmodena@deploy1002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mediawiki-page-content-change-enrichment: apply [production]