|
2022-05-04
ยง
|
| 12:46 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1158 (T307525)', diff saved to https://phabricator.wikimedia.org/P27450 and previous config saved to /var/cache/conftool/dbconfig/20220504-124650-ladsgroup.json |
[production] |
| 12:46 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on clouddb[1014,1018,1021].eqiad.wmnet,db1155.eqiad.wmnet with reason: Maintenance |
[production] |
| 12:46 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 12:00:00 on clouddb[1014,1018,1021].eqiad.wmnet,db1155.eqiad.wmnet with reason: Maintenance |
[production] |
| 12:46 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1158.eqiad.wmnet with reason: Maintenance |
[production] |
| 12:46 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db1158.eqiad.wmnet with reason: Maintenance |
[production] |
| 12:46 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1174 (T307525)', diff saved to https://phabricator.wikimedia.org/P27449 and previous config saved to /var/cache/conftool/dbconfig/20220504-124637-ladsgroup.json |
[production] |
| 12:45 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply |
[production] |
| 12:45 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 12:41 |
<mvernon@cumin1001> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ms-be2045.codfw.wmnet with OS bullseye |
[production] |
| 12:41 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply |
[production] |
| 12:41 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1161 (T307525)', diff saved to https://phabricator.wikimedia.org/P27448 and previous config saved to /var/cache/conftool/dbconfig/20220504-124140-ladsgroup.json |
[production] |
| 12:38 |
<hashar@deploy1002> |
Synchronized php-1.39.0-wmf.10/extensions/FlaggedRevs/: FlaggedRevsHooks: Update use of GoogleNewsSitemap constants - T307552 (duration: 00m 51s) |
[production] |
| 12:33 |
<jbond@cumin1001> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host sretest1001.eqiad.wmnet with OS buster |
[production] |
| 12:31 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1174', diff saved to https://phabricator.wikimedia.org/P27447 and previous config saved to /var/cache/conftool/dbconfig/20220504-123132-ladsgroup.json |
[production] |
| 12:28 |
<ayounsi@cumin2002> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 12:27 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1161 (T307525)', diff saved to https://phabricator.wikimedia.org/P27446 and previous config saved to /var/cache/conftool/dbconfig/20220504-122715-ladsgroup.json |
[production] |
| 12:27 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on clouddb[1016,1020-1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance |
[production] |
| 12:27 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 12:00:00 on clouddb[1016,1020-1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance |
[production] |
| 12:27 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1161.eqiad.wmnet with reason: Maintenance |
[production] |
| 12:27 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db1161.eqiad.wmnet with reason: Maintenance |
[production] |
| 12:26 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1161 (T307525)', diff saved to https://phabricator.wikimedia.org/P27445 and previous config saved to /var/cache/conftool/dbconfig/20220504-122615-ladsgroup.json |
[production] |
| 12:24 |
<ayounsi@cumin2002> |
START - Cookbook sre.dns.netbox |
[production] |
| 12:24 |
<ayounsi@cumin2002> |
START - Cookbook sre.ganeti.makevm for new host netbox-dev2002.codfw.wmnet |
[production] |
| 12:21 |
<jbond@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on sretest1001.eqiad.wmnet with reason: host reimage |
[production] |
| 12:17 |
<jbond@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on sretest1001.eqiad.wmnet with reason: host reimage |
[production] |
| 12:16 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1174', diff saved to https://phabricator.wikimedia.org/P27444 and previous config saved to /var/cache/conftool/dbconfig/20220504-121627-ladsgroup.json |
[production] |
| 12:15 |
<mvernon@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ms-be2045.codfw.wmnet with reason: host reimage |
[production] |
| 12:12 |
<mvernon@cumin1001> |
START - Cookbook sre.hosts.downtime for 2:00:00 on ms-be2045.codfw.wmnet with reason: host reimage |
[production] |
| 12:11 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1161', diff saved to https://phabricator.wikimedia.org/P27443 and previous config saved to /var/cache/conftool/dbconfig/20220504-121110-ladsgroup.json |
[production] |
| 12:07 |
<jbond@cumin1001> |
START - Cookbook sre.hosts.reimage for host sretest1001.eqiad.wmnet with OS buster |
[production] |
| 12:01 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1174 (T307525)', diff saved to https://phabricator.wikimedia.org/P27442 and previous config saved to /var/cache/conftool/dbconfig/20220504-120122-ladsgroup.json |
[production] |
| 11:56 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1161', diff saved to https://phabricator.wikimedia.org/P27441 and previous config saved to /var/cache/conftool/dbconfig/20220504-115605-ladsgroup.json |
[production] |
| 11:50 |
<mvernon@cumin1001> |
START - Cookbook sre.hosts.reimage for host ms-be2045.codfw.wmnet with OS bullseye |
[production] |
| 11:47 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1174 (T307525)', diff saved to https://phabricator.wikimedia.org/P27440 and previous config saved to /var/cache/conftool/dbconfig/20220504-114749-ladsgroup.json |
[production] |
| 11:47 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1174.eqiad.wmnet with reason: Maintenance |
[production] |
| 11:47 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db1174.eqiad.wmnet with reason: Maintenance |
[production] |
| 11:41 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1161 (T307525)', diff saved to https://phabricator.wikimedia.org/P27439 and previous config saved to /var/cache/conftool/dbconfig/20220504-114100-ladsgroup.json |
[production] |
| 11:34 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1161 (T307525)', diff saved to https://phabricator.wikimedia.org/P27438 and previous config saved to /var/cache/conftool/dbconfig/20220504-113443-ladsgroup.json |
[production] |
| 11:34 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on clouddb[1016,1020-1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance |
[production] |
| 11:34 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 12:00:00 on clouddb[1016,1020-1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance |
[production] |
| 11:34 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1161.eqiad.wmnet with reason: Maintenance |
[production] |
| 11:34 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db1161.eqiad.wmnet with reason: Maintenance |
[production] |
| 11:34 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1113:3315 (T307525)', diff saved to https://phabricator.wikimedia.org/P27437 and previous config saved to /var/cache/conftool/dbconfig/20220504-113429-ladsgroup.json |
[production] |
| 11:33 |
<kart_> |
Updated cxserver to 2022-05-04-034605-production (T304828, T304858, T201491) |
[production] |
| 11:32 |
<kartik@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/cxserver: apply |
[production] |
| 11:31 |
<kartik@deploy1002> |
helmfile [eqiad] START helmfile.d/services/cxserver: apply |
[production] |
| 11:26 |
<kartik@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/cxserver: apply |
[production] |
| 11:25 |
<kartik@deploy1002> |
helmfile [codfw] START helmfile.d/services/cxserver: apply |
[production] |
| 11:23 |
<btullis@cumin1001> |
END (PASS) - Cookbook sre.hadoop.roll-restart-masters (exit_code=0) restart masters for Hadoop test cluster: Restart of jvm daemons. |
[production] |
| 11:20 |
<kartik@deploy1002> |
helmfile [staging] DONE helmfile.d/services/cxserver: apply |
[production] |