2025-08-12
ยง
|
06:32 |
<vriley@cumin1002> |
START - Cookbook sre.hosts.provision for host cloudcephosd1044.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART and with Dell SCP reboot policy FORCED |
[production] |
06:26 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2155', diff saved to https://phabricator.wikimedia.org/P81044 and previous config saved to /var/cache/conftool/dbconfig/20250812-062638-fceratto.json |
[production] |
06:24 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2152', diff saved to https://phabricator.wikimedia.org/P81043 and previous config saved to /var/cache/conftool/dbconfig/20250812-062405-ladsgroup.json |
[production] |
06:11 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2155', diff saved to https://phabricator.wikimedia.org/P81042 and previous config saved to /var/cache/conftool/dbconfig/20250812-061130-fceratto.json |
[production] |
06:08 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2152 (T400854)', diff saved to https://phabricator.wikimedia.org/P81041 and previous config saved to /var/cache/conftool/dbconfig/20250812-060857-ladsgroup.json |
[production] |
06:07 |
<vriley@cumin1002> |
START - Cookbook sre.hosts.reimage for host cloudcephosd1044.eqiad.wmnet with OS bookworm |
[production] |
06:07 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Depooling db2152 (T400854)', diff saved to https://phabricator.wikimedia.org/P81040 and previous config saved to /var/cache/conftool/dbconfig/20250812-060705-ladsgroup.json |
[production] |
06:06 |
<ladsgroup@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2152.codfw.wmnet with reason: Maintenance |
[production] |
06:06 |
<ladsgroup@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on dbstore1009.eqiad.wmnet with reason: Maintenance |
[production] |
06:06 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1226 (T400854)', diff saved to https://phabricator.wikimedia.org/P81039 and previous config saved to /var/cache/conftool/dbconfig/20250812-060559-ladsgroup.json |
[production] |
05:56 |
<fceratto@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db2155 (T399249)', diff saved to https://phabricator.wikimedia.org/P81038 and previous config saved to /var/cache/conftool/dbconfig/20250812-055623-fceratto.json |
[production] |
05:50 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1226', diff saved to https://phabricator.wikimedia.org/P81037 and previous config saved to /var/cache/conftool/dbconfig/20250812-055052-ladsgroup.json |
[production] |
05:35 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1226', diff saved to https://phabricator.wikimedia.org/P81036 and previous config saved to /var/cache/conftool/dbconfig/20250812-053544-ladsgroup.json |
[production] |
05:20 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1226 (T400854)', diff saved to https://phabricator.wikimedia.org/P81035 and previous config saved to /var/cache/conftool/dbconfig/20250812-052037-ladsgroup.json |
[production] |
05:17 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Depooling db1226 (T400854)', diff saved to https://phabricator.wikimedia.org/P81034 and previous config saved to /var/cache/conftool/dbconfig/20250812-051757-ladsgroup.json |
[production] |
05:17 |
<ladsgroup@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1226.eqiad.wmnet with reason: Maintenance |
[production] |
05:17 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1214 (T400854)', diff saved to https://phabricator.wikimedia.org/P81033 and previous config saved to /var/cache/conftool/dbconfig/20250812-051735-ladsgroup.json |
[production] |
05:02 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1214', diff saved to https://phabricator.wikimedia.org/P81032 and previous config saved to /var/cache/conftool/dbconfig/20250812-050227-ladsgroup.json |
[production] |
04:47 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1214', diff saved to https://phabricator.wikimedia.org/P81031 and previous config saved to /var/cache/conftool/dbconfig/20250812-044719-ladsgroup.json |
[production] |
04:32 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1214 (T400854)', diff saved to https://phabricator.wikimedia.org/P81030 and previous config saved to /var/cache/conftool/dbconfig/20250812-043212-ladsgroup.json |
[production] |
04:29 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Depooling db1214 (T400854)', diff saved to https://phabricator.wikimedia.org/P81029 and previous config saved to /var/cache/conftool/dbconfig/20250812-042937-ladsgroup.json |
[production] |
04:29 |
<ladsgroup@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1214.eqiad.wmnet with reason: Maintenance |
[production] |
04:29 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1209 (T400854)', diff saved to https://phabricator.wikimedia.org/P81028 and previous config saved to /var/cache/conftool/dbconfig/20250812-042915-ladsgroup.json |
[production] |
04:14 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1209', diff saved to https://phabricator.wikimedia.org/P81027 and previous config saved to /var/cache/conftool/dbconfig/20250812-041408-ladsgroup.json |
[production] |
04:04 |
<mwpresync@deploy1003> |
Pruned MediaWiki: 1.45.0-wmf.11 (duration: 04m 19s) |
[production] |
03:59 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1209', diff saved to https://phabricator.wikimedia.org/P81026 and previous config saved to /var/cache/conftool/dbconfig/20250812-035900-ladsgroup.json |
[production] |
03:41 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Depooling db1209 (T400854)', diff saved to https://phabricator.wikimedia.org/P81024 and previous config saved to /var/cache/conftool/dbconfig/20250812-034107-ladsgroup.json |
[production] |
03:41 |
<ladsgroup@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1209.eqiad.wmnet with reason: Maintenance |
[production] |
03:40 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1203 (T400854)', diff saved to https://phabricator.wikimedia.org/P81023 and previous config saved to /var/cache/conftool/dbconfig/20250812-034045-ladsgroup.json |
[production] |
03:25 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1203', diff saved to https://phabricator.wikimedia.org/P81022 and previous config saved to /var/cache/conftool/dbconfig/20250812-032537-ladsgroup.json |
[production] |
03:10 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1203', diff saved to https://phabricator.wikimedia.org/P81021 and previous config saved to /var/cache/conftool/dbconfig/20250812-031029-ladsgroup.json |
[production] |
02:55 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1203 (T400854)', diff saved to https://phabricator.wikimedia.org/P81020 and previous config saved to /var/cache/conftool/dbconfig/20250812-025522-ladsgroup.json |
[production] |
02:52 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Depooling db1203 (T400854)', diff saved to https://phabricator.wikimedia.org/P81019 and previous config saved to /var/cache/conftool/dbconfig/20250812-025239-ladsgroup.json |
[production] |
02:52 |
<ladsgroup@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1203.eqiad.wmnet with reason: Maintenance |
[production] |
02:52 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1192 (T400854)', diff saved to https://phabricator.wikimedia.org/P81018 and previous config saved to /var/cache/conftool/dbconfig/20250812-025216-ladsgroup.json |
[production] |
02:37 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P81017 and previous config saved to /var/cache/conftool/dbconfig/20250812-023709-ladsgroup.json |
[production] |
02:22 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1192', diff saved to https://phabricator.wikimedia.org/P81016 and previous config saved to /var/cache/conftool/dbconfig/20250812-022201-ladsgroup.json |
[production] |
02:06 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1192 (T400854)', diff saved to https://phabricator.wikimedia.org/P81015 and previous config saved to /var/cache/conftool/dbconfig/20250812-020653-ladsgroup.json |
[production] |
02:04 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Depooling db1192 (T400854)', diff saved to https://phabricator.wikimedia.org/P81014 and previous config saved to /var/cache/conftool/dbconfig/20250812-020403-ladsgroup.json |
[production] |
02:03 |
<ladsgroup@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1192.eqiad.wmnet with reason: Maintenance |
[production] |
02:03 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1178 (T400854)', diff saved to https://phabricator.wikimedia.org/P81013 and previous config saved to /var/cache/conftool/dbconfig/20250812-020341-ladsgroup.json |
[production] |
01:48 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P81012 and previous config saved to /var/cache/conftool/dbconfig/20250812-014833-ladsgroup.json |
[production] |
01:33 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1178', diff saved to https://phabricator.wikimedia.org/P81011 and previous config saved to /var/cache/conftool/dbconfig/20250812-013325-ladsgroup.json |
[production] |
01:30 |
<ejegg> |
payments-wiki upgraded from 0a1084a8 to cd876775 |
[production] |
01:26 |
<vriley@cumin1002> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudcephosd1044.eqiad.wmnet with OS bookworm |
[production] |
01:18 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1178 (T400854)', diff saved to https://phabricator.wikimedia.org/P81010 and previous config saved to /var/cache/conftool/dbconfig/20250812-011817-ladsgroup.json |
[production] |
01:14 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Depooling db1178 (T400854)', diff saved to https://phabricator.wikimedia.org/P81009 and previous config saved to /var/cache/conftool/dbconfig/20250812-011427-ladsgroup.json |
[production] |
01:14 |
<ladsgroup@cumin1002> |
DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1178.eqiad.wmnet with reason: Maintenance |
[production] |
01:14 |
<ladsgroup@cumin1002> |
dbctl commit (dc=all): 'Repooling after maintenance db1177 (T400854)', diff saved to https://phabricator.wikimedia.org/P81008 and previous config saved to /var/cache/conftool/dbconfig/20250812-011403-ladsgroup.json |
[production] |
01:01 |
<eileen> |
civicrm upgraded from ebb98a9e to 13fb0ba8 |
[production] |