6251-6300 of 10000 results (94ms)
2024-02-24 §
02:47 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1248 (T357189)', diff saved to https://phabricator.wikimedia.org/P57862 and previous config saved to /var/cache/conftool/dbconfig/20240224-024722-arnaudb.json [production]
01:47 <brett> Upload ncmonitor 0.0.3 to bookworm-wikimedia [production]
01:47 <arnaudb@cumin1002> dbctl commit (dc=all): 'Depooling db1248 (T357189)', diff saved to https://phabricator.wikimedia.org/P57861 and previous config saved to /var/cache/conftool/dbconfig/20240224-014734-arnaudb.json [production]
01:47 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1248.eqiad.wmnet with reason: Maintenance [production]
01:47 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1248.eqiad.wmnet with reason: Maintenance [production]
01:47 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1247 (T357189)', diff saved to https://phabricator.wikimedia.org/P57860 and previous config saved to /var/cache/conftool/dbconfig/20240224-014711-arnaudb.json [production]
01:32 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1247', diff saved to https://phabricator.wikimedia.org/P57859 and previous config saved to /var/cache/conftool/dbconfig/20240224-013205-arnaudb.json [production]
01:16 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1247', diff saved to https://phabricator.wikimedia.org/P57858 and previous config saved to /var/cache/conftool/dbconfig/20240224-011658-arnaudb.json [production]
01:01 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1247 (T357189)', diff saved to https://phabricator.wikimedia.org/P57857 and previous config saved to /var/cache/conftool/dbconfig/20240224-010152-arnaudb.json [production]
2024-02-23 §
23:59 <arnaudb@cumin1002> dbctl commit (dc=all): 'Depooling db1247 (T357189)', diff saved to https://phabricator.wikimedia.org/P57856 and previous config saved to /var/cache/conftool/dbconfig/20240223-235919-arnaudb.json [production]
23:59 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1247.eqiad.wmnet with reason: Maintenance [production]
23:59 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1247.eqiad.wmnet with reason: Maintenance [production]
23:14 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1245.eqiad.wmnet with reason: Maintenance [production]
23:14 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1245.eqiad.wmnet with reason: Maintenance [production]
23:14 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1244 (T357189)', diff saved to https://phabricator.wikimedia.org/P57855 and previous config saved to /var/cache/conftool/dbconfig/20240223-231440-arnaudb.json [production]
22:59 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1244', diff saved to https://phabricator.wikimedia.org/P57854 and previous config saved to /var/cache/conftool/dbconfig/20240223-225933-arnaudb.json [production]
22:44 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1244', diff saved to https://phabricator.wikimedia.org/P57853 and previous config saved to /var/cache/conftool/dbconfig/20240223-224427-arnaudb.json [production]
22:29 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1244 (T357189)', diff saved to https://phabricator.wikimedia.org/P57852 and previous config saved to /var/cache/conftool/dbconfig/20240223-222920-arnaudb.json [production]
21:49 <sbassett> Deployed updated security mitigation for T336027 [production]
21:42 <arnaudb@cumin1002> dbctl commit (dc=all): 'Depooling db1244 (T357189)', diff saved to https://phabricator.wikimedia.org/P57850 and previous config saved to /var/cache/conftool/dbconfig/20240223-214211-arnaudb.json [production]
21:42 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1244.eqiad.wmnet with reason: Maintenance [production]
21:41 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1244.eqiad.wmnet with reason: Maintenance [production]
21:41 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1243 (T357189)', diff saved to https://phabricator.wikimedia.org/P57848 and previous config saved to /var/cache/conftool/dbconfig/20240223-214149-arnaudb.json [production]
21:26 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1243', diff saved to https://phabricator.wikimedia.org/P57847 and previous config saved to /var/cache/conftool/dbconfig/20240223-212643-arnaudb.json [production]
21:11 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1243', diff saved to https://phabricator.wikimedia.org/P57846 and previous config saved to /var/cache/conftool/dbconfig/20240223-211136-arnaudb.json [production]
21:07 <jhancock@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2196.codfw.wmnet with reason: host reimage [production]
21:04 <jhancock@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on db2196.codfw.wmnet with reason: host reimage [production]
20:56 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1243 (T357189)', diff saved to https://phabricator.wikimedia.org/P57845 and previous config saved to /var/cache/conftool/dbconfig/20240223-205630-arnaudb.json [production]
20:48 <jhancock@cumin2002> START - Cookbook sre.hosts.reimage for host db2196.codfw.wmnet with OS bookworm [production]
20:42 <isaranto@deploy2002> helmfile [ml-staging-codfw] Ran 'sync' command on namespace 'experimental' for release 'main' . [production]
20:23 <RhinosF1> [relog due to stashbot errors] jhancock@cumin2002 ran cookbook SRE.hardware.upgrade-firmware for hosts db2201/db2204/db2197/db2198/db2202/db2203/db2205 and all END PASS [production]
20:10 <jhancock@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['db2205'] [production]
20:09 <jhancock@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['db2203'] [production]
20:09 <jhancock@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['db2202'] [production]
20:08 <jhancock@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['db2198'] [production]
20:08 <jhancock@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['db2197'] [production]
20:07 <jhancock@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['db2204'] [production]
20:07 <jhancock@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['db2201'] [production]
20:06 <jhancock@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['db2199'] [production]
20:03 <eileen> civicrm upgraded from c50fcae3 to a65551c7 [production]
20:00 <jhancock@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['db2205'] [production]
20:00 <jhancock@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['db2204'] [production]
19:59 <jhancock@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['db2203'] [production]
19:59 <jhancock@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['db2202'] [production]
19:59 <jhancock@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['db2201'] [production]
19:59 <jhancock@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['db2199'] [production]
19:59 <jhancock@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['db2198'] [production]
19:59 <jhancock@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['db2197'] [production]
19:58 <arnaudb@cumin1002> dbctl commit (dc=all): 'Depooling db1243 (T357189)', diff saved to https://phabricator.wikimedia.org/P57844 and previous config saved to /var/cache/conftool/dbconfig/20240223-195835-arnaudb.json [production]
19:58 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1243.eqiad.wmnet with reason: Maintenance [production]