1201-1250 of 10000 results (65ms)
2023-01-26 §
05:09 <brett@cumin1001> conftool action : set/pooled=yes; selector: name=cp6014.drmrs.wmnet [production]
05:07 <brett@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp6014.drmrs.wmnet with OS bullseye [production]
04:45 <brett@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cp6014.drmrs.wmnet with reason: host reimage [production]
04:42 <brett@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on cp6014.drmrs.wmnet with reason: host reimage [production]
04:24 <brett@cumin1001> START - Cookbook sre.hosts.reimage for host cp6014.drmrs.wmnet with OS bullseye [production]
04:22 <brett@cumin1001> conftool action : set/pooled=yes; selector: name=cp6005.drmrs.wmnet [production]
04:17 <brett@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp6005.drmrs.wmnet with OS bullseye [production]
03:52 <brett@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cp6005.drmrs.wmnet with reason: host reimage [production]
03:49 <brett@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on cp6005.drmrs.wmnet with reason: host reimage [production]
03:29 <brett@cumin1001> START - Cookbook sre.hosts.reimage for host cp6005.drmrs.wmnet with OS bullseye [production]
03:27 <brett@cumin1001> conftool action : set/pooled=yes; selector: name=cp6013.drmrs.wmnet [production]
03:27 <ejegg> payments-wiki upgraded from 08b8c3bc to 82d89841 [production]
03:26 <brett@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp6013.drmrs.wmnet with OS bullseye [production]
03:04 <brett@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cp6013.drmrs.wmnet with reason: host reimage [production]
03:01 <brett@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on cp6013.drmrs.wmnet with reason: host reimage [production]
02:41 <brett@cumin1001> START - Cookbook sre.hosts.reimage for host cp6013.drmrs.wmnet with OS bullseye [production]
02:30 <sukhe@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host cp2027.codfw.wmnet with OS bullseye [production]
02:17 <sukhe@cumin2002> START - Cookbook sre.hosts.reimage for host cp2027.codfw.wmnet with OS bullseye [production]
02:17 <sukhe@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host cp2027.codfw.wmnet with OS bullseye [production]
01:58 <ejegg> restarted fundraising scheduled jobs after queue server reboot [production]
01:53 <sukhe@cumin2002> START - Cookbook sre.hosts.reimage for host cp2027.codfw.wmnet with OS bullseye [production]
01:49 <sukhe@puppetmaster1001> conftool action : set/pooled=yes; selector: name=cp2028.codfw.wmnet,service=ats-be [production]
01:49 <sukhe@puppetmaster1001> conftool action : set/pooled=yes; selector: name=cp2028.codfw.wmnet,service=cdn [production]
01:48 <sukhe@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on cp2027.codfw.wmnet with reason: firmware test [production]
01:48 <sukhe@cumin2002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on cp2027.codfw.wmnet with reason: firmware test [production]
01:46 <sukhe@puppetmaster1001> conftool action : set/pooled=no; selector: name=cp2027.codfw.wmnet,service=ats-be [production]
01:46 <sukhe@puppetmaster1001> conftool action : set/pooled=no; selector: name=cp2027.codfw.wmnet,service=cdn [production]
01:46 <sukhe@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp2028.codfw.wmnet with OS bullseye [production]
01:24 <ejegg> payments-wiki upgraded from 15395d05 to 08b8c3bc (upgraded from MW 1.35 to MW 1.39) [production]
01:23 <sukhe@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cp2028.codfw.wmnet with reason: host reimage [production]
01:20 <sukhe@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on cp2028.codfw.wmnet with reason: host reimage [production]
01:19 <eevans@cumin1001> END (PASS) - Cookbook sre.cassandra.roll-restart (exit_code=0) for nodes matching cassandra-dev2*: Enable internode encryption - eevans@cumin1001 [production]
01:14 <ejegg> disabled fundraising scheduled jobs for queue server reboot [production]
01:05 <sukhe@cumin2002> START - Cookbook sre.hosts.reimage for host cp2028.codfw.wmnet with OS bullseye [production]
01:03 <sukhe@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts cp2028.codfw.wmnet [production]
01:03 <sukhe@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host cp2028.codfw.wmnet [production]
01:00 <eevans@cumin1001> START - Cookbook sre.cassandra.roll-restart for nodes matching cassandra-dev2*: Enable internode encryption - eevans@cumin1001 [production]
01:00 <ejegg> turned pending transaction resolvers back on after civi deploy [production]
00:51 <sukhe@cumin2002> START - Cookbook sre.hosts.reboot-single for host cp2028.codfw.wmnet [production]
00:50 <ejegg> civicrm upgraded from 3e6b21b6 to b5d6a790 [production]
00:50 <sukhe@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts cp2028.codfw.wmnet [production]
00:49 <sukhe> depool cp2028 for testing firmware update cookbook: T321309 [production]
00:49 <ejegg> disabled pending transaction resolvers for civi deploy [production]
00:48 <sukhe@puppetmaster1001> conftool action : set/pooled=no; selector: name=cp2028.codfw.wmnet,service=ats-be [production]
00:48 <sukhe@puppetmaster1001> conftool action : set/pooled=no; selector: name=cp2028.codfw.wmnet,service=cdn [production]
2023-01-25 §
23:57 <brett@cumin1001> conftool action : set/pooled=yes; selector: name=cp6004.drmrs.wmnet [production]
23:57 <brett@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cp6004.drmrs.wmnet with OS bullseye [production]
23:36 <brett@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cp6004.drmrs.wmnet with reason: host reimage [production]
23:33 <brett@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on cp6004.drmrs.wmnet with reason: host reimage [production]
23:29 <zabe@deploy1002> Finished scap: (no justification provided) (duration: 07m 34s) [production]