601-650 of 10000 results (73ms)
2023-01-26 §
01:46 <sukhe@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp2028.codfw.wmnet with OS bullseye [production]
01:24 <ejegg> payments-wiki upgraded from 15395d05 to 08b8c3bc (upgraded from MW 1.35 to MW 1.39) [production]
01:23 <sukhe@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cp2028.codfw.wmnet with reason: host reimage [production]
01:20 <sukhe@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on cp2028.codfw.wmnet with reason: host reimage [production]
01:19 <eevans@cumin1001> END (PASS) - Cookbook sre.cassandra.roll-restart (exit_code=0) for nodes matching cassandra-dev2*: Enable internode encryption - eevans@cumin1001 [production]
01:14 <ejegg> disabled fundraising scheduled jobs for queue server reboot [production]
01:05 <sukhe@cumin2002> START - Cookbook sre.hosts.reimage for host cp2028.codfw.wmnet with OS bullseye [production]
01:03 <sukhe@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts cp2028.codfw.wmnet [production]
01:03 <sukhe@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cp2028.codfw.wmnet [production]
01:00 <eevans@cumin1001> START - Cookbook sre.cassandra.roll-restart for nodes matching cassandra-dev2*: Enable internode encryption - eevans@cumin1001 [production]
01:00 <ejegg> turned pending transaction resolvers back on after civi deploy [production]
00:51 <sukhe@cumin2002> START - Cookbook sre.hosts.reboot-single for host cp2028.codfw.wmnet [production]
00:50 <ejegg> civicrm upgraded from 3e6b21b6 to b5d6a790 [production]
00:50 <sukhe@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts cp2028.codfw.wmnet [production]
00:49 <sukhe> depool cp2028 for testing firmware update cookbook: T321309 [production]
00:49 <ejegg> disabled pending transaction resolvers for civi deploy [production]
00:48 <sukhe@puppetmaster1001> conftool action : set/pooled=no; selector: name=cp2028.codfw.wmnet,service=ats-be [production]
00:48 <sukhe@puppetmaster1001> conftool action : set/pooled=no; selector: name=cp2028.codfw.wmnet,service=cdn [production]
2023-01-25 §
23:57 <brett@cumin1001> conftool action : set/pooled=yes; selector: name=cp6004.drmrs.wmnet [production]
23:57 <brett@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp6004.drmrs.wmnet with OS bullseye [production]
23:36 <brett@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cp6004.drmrs.wmnet with reason: host reimage [production]
23:33 <brett@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on cp6004.drmrs.wmnet with reason: host reimage [production]
23:29 <zabe@deploy1002> Finished scap: (no justification provided) (duration: 07m 34s) [production]
23:21 <zabe@deploy1002> Started scap: (no justification provided) [production]
23:20 <zabe@deploy1002> Backport cancelled. [production]
23:14 <brett@cumin1001> START - Cookbook sre.hosts.reimage for host cp6004.drmrs.wmnet with OS bullseye [production]
23:13 <brett@cumin1001> conftool action : set/pooled=yes; selector: name=cp6012.drmrs.wmnet [production]
23:07 <brett@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp6012.drmrs.wmnet with OS bullseye [production]
22:43 <brett@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cp6012.drmrs.wmnet with reason: host reimage [production]
22:40 <brett@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on cp6012.drmrs.wmnet with reason: host reimage [production]
22:21 <brett@cumin1001> START - Cookbook sre.hosts.reimage for host cp6012.drmrs.wmnet with OS bullseye [production]
22:14 <brett@cumin1001> conftool action : set/pooled=yes; selector: name=cp6003.drmrs.wmnet [production]
21:49 <bking@deploy1002> helmfile [dse-k8s-eqiad] DONE helmfile.d/services/flink-app-example: apply [production]
21:49 <bking@deploy1002> helmfile [dse-k8s-eqiad] START helmfile.d/services/flink-app-example: apply [production]
21:44 <bking@deploy1002> helmfile [dse-k8s-eqiad] DONE helmfile.d/services/flink-app-example: apply [production]
21:44 <bking@deploy1002> helmfile [dse-k8s-eqiad] START helmfile.d/services/flink-app-example: apply [production]
21:34 <samtar@deploy1002> Finished scap: Backport for [[gerrit:883617|Define grid template row for .mw-body grid container to ensure the grid cell containing the content will expand in height when needed (T327714)]], [[gerrit:883616|Define grid template row for .mw-body grid container to ensure the grid cell containing the content will expand in height when needed (T327714)]] (duration: 09m 27s) [production]
21:26 <samtar@deploy1002> jdrewniak and samtar: Backport for [[gerrit:883617|Define grid template row for .mw-body grid container to ensure the grid cell containing the content will expand in height when needed (T327714)]], [[gerrit:883616|Define grid template row for .mw-body grid container to ensure the grid cell containing the content will expand in height when needed (T327714)]] synced to the testservers: mwdebug2002.cod [production]
21:25 <bking@deploy1002> helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. [production]
21:24 <bking@deploy1002> helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. [production]
21:24 <samtar@deploy1002> Started scap: Backport for [[gerrit:883617|Define grid template row for .mw-body grid container to ensure the grid cell containing the content will expand in height when needed (T327714)]], [[gerrit:883616|Define grid template row for .mw-body grid container to ensure the grid cell containing the content will expand in height when needed (T327714)]] [production]
21:06 <sukhe@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts cp2028.codfw.wmnet [production]
20:59 <sukhe@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts cp2028.codfw.wmnet [production]
20:59 <brett@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp6003.drmrs.wmnet with OS bullseye [production]
20:59 <sukhe@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=True) upgrade firmware for hosts cp2028.codfw.wmnet [production]
20:58 <sukhe@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts cp2028.codfw.wmnet [production]
20:56 <sukhe@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts cp2028.codfw.wmnet [production]
20:49 <sukhe@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts cp2028.codfw.wmnet [production]
20:49 <sukhe@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts cp2028.codfw.wmnet [production]
20:49 <sukhe@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts cp2028.codfw.wmnet [production]