7101-7150 of 10000 results (71ms)
2024-08-16 §
07:40 <jayme@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host kafka-main2007.codfw.wmnet with OS bullseye [production]
07:23 <jayme@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on kafka-main2007.codfw.wmnet with reason: host reimage [production]
07:20 <jayme@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on kafka-main2007.codfw.wmnet with reason: host reimage [production]
07:01 <jayme@cumin1002> START - Cookbook sre.hosts.reimage for host kafka-main2007.codfw.wmnet with OS bullseye [production]
06:56 <marostegui@cumin1002> dbctl commit (dc=all): 'Repool db2136 - running 10.11', diff saved to https://phabricator.wikimedia.org/P67345 and previous config saved to /var/cache/conftool/dbconfig/20240816-065606-marostegui.json [production]
06:42 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2 days, 0:00:00 on db2152.codfw.wmnet with reason: Schema change [production]
06:42 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 2 days, 0:00:00 on db2152.codfw.wmnet with reason: Schema change [production]
05:40 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.osd.drain_node (exit_code=0) (T363344) [admin]
04:46 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.osd.drain_node (exit_code=0) (T363344) [admin]
04:45 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.drain_node (T363344) [admin]
04:42 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.osd.drain_node (exit_code=0) (T363344) [admin]
04:42 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.drain_node (T363344) [admin]
04:33 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.osd.drain_node (exit_code=0) (T363344) [admin]
04:32 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.drain_node (T363344) [admin]
04:26 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.osd.drain_node (exit_code=0) (T363344) [admin]
04:25 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.drain_node (T363344) [admin]
03:58 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.osd.drain_node (exit_code=0) (T363344) [admin]
03:57 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.drain_node (T363344) [admin]
03:53 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.osd.drain_node (exit_code=0) (T363344) [admin]
03:52 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.drain_node (T363344) [admin]
03:51 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.osd.drain_node (exit_code=0) (T363344) [admin]
03:50 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.drain_node (T363344) [admin]
03:45 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.drain_node (T363344) [admin]
01:17 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.osd.undrain_node (exit_code=0) [admin]
01:17 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.undrain_node [admin]
01:14 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.osd.undrain_node (exit_code=0) [admin]
01:14 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.undrain_node [admin]
01:12 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.osd.undrain_node (exit_code=0) [admin]
01:12 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.undrain_node [admin]
01:10 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.osd.undrain_node (exit_code=0) [admin]
01:10 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.undrain_node [admin]
01:10 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.osd.undrain_node (exit_code=0) [admin]
01:09 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.undrain_node [admin]
01:08 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.osd.undrain_node (exit_code=0) [admin]
01:08 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.undrain_node [admin]
01:08 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.osd.undrain_node (exit_code=0) [admin]
01:08 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.undrain_node [admin]
01:08 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.osd.undrain_node (exit_code=0) [admin]
01:08 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.undrain_node [admin]
01:08 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.osd.undrain_node (exit_code=0) [admin]
01:08 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.undrain_node [admin]
01:08 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.osd.undrain_node (exit_code=0) [admin]
01:08 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.undrain_node [admin]
2024-08-15 §
23:30 <jhancock@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudlb2004-dev.codfw.wmnet with OS bookworm [production]
23:10 <xSavitar> T372449 mwscript extensions/CentralAuth/maintenance/fixStuckGlobalRename.php --wiki=commonswiki --logwiki=metawiki 'Philip Federici' 'FilippoFederici' --ignorestatus [production]
22:42 <ryankemper@cumin2002> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) (T370754, transfer fresh wdqs-main journal to codfw host) xfer wikidata_main from wdqs1021.eqiad.wmnet -> wdqs2021.codfw.wmnet w/ force delete existing files, repooling neither afterwards [production]
22:10 <jhancock@cumin2002> START - Cookbook sre.hosts.reimage for host cloudlb2004-dev.codfw.wmnet with OS bookworm [production]
22:09 <jhancock@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['cloudlb2004-dev'] [production]
22:02 <jhancock@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['cloudlb2004-dev'] [production]
22:01 <jhancock@cumin2002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host cloudlb2004-dev.mgmt.codfw.wmnet with reboot policy FORCED [production]