|
2022-04-21
ยง
|
| 21:40 |
<dzahn@cumin2002> |
START - Cookbook sre.hosts.downtime for 1:00:00 on gitlab-runner1001.eqiad.wmnet with reason: reimage |
[production] |
| 21:40 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1113:3316', diff saved to https://phabricator.wikimedia.org/P26074 and previous config saved to /var/cache/conftool/dbconfig/20220421-214035-ladsgroup.json |
[production] |
| 21:40 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1129 (T298565)', diff saved to https://phabricator.wikimedia.org/P26073 and previous config saved to /var/cache/conftool/dbconfig/20220421-214027-ladsgroup.json |
[production] |
| 21:38 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1129 (T298565)', diff saved to https://phabricator.wikimedia.org/P26072 and previous config saved to /var/cache/conftool/dbconfig/20220421-213819-ladsgroup.json |
[production] |
| 21:38 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1129.eqiad.wmnet with reason: Maintenance |
[production] |
| 21:38 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db1129.eqiad.wmnet with reason: Maintenance |
[production] |
| 21:38 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1182 (T298565)', diff saved to https://phabricator.wikimedia.org/P26071 and previous config saved to /var/cache/conftool/dbconfig/20220421-213811-ladsgroup.json |
[production] |
| 21:35 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1157', diff saved to https://phabricator.wikimedia.org/P26070 and previous config saved to /var/cache/conftool/dbconfig/20220421-213529-ladsgroup.json |
[production] |
| 21:29 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1136', diff saved to https://phabricator.wikimedia.org/P26069 and previous config saved to /var/cache/conftool/dbconfig/20220421-212916-ladsgroup.json |
[production] |
| 21:25 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1113:3316', diff saved to https://phabricator.wikimedia.org/P26068 and previous config saved to /var/cache/conftool/dbconfig/20220421-212523-ladsgroup.json |
[production] |
| 21:23 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1182', diff saved to https://phabricator.wikimedia.org/P26067 and previous config saved to /var/cache/conftool/dbconfig/20220421-212306-ladsgroup.json |
[production] |
| 21:20 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1157 (T298565)', diff saved to https://phabricator.wikimedia.org/P26066 and previous config saved to /var/cache/conftool/dbconfig/20220421-212022-ladsgroup.json |
[production] |
| 21:19 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 21:19 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply |
[production] |
| 21:19 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 21:19 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply |
[production] |
| 21:14 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1136', diff saved to https://phabricator.wikimedia.org/P26065 and previous config saved to /var/cache/conftool/dbconfig/20220421-211411-ladsgroup.json |
[production] |
| 21:10 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1113:3316 (T306560)', diff saved to https://phabricator.wikimedia.org/P26064 and previous config saved to /var/cache/conftool/dbconfig/20220421-211018-ladsgroup.json |
[production] |
| 21:08 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1182', diff saved to https://phabricator.wikimedia.org/P26063 and previous config saved to /var/cache/conftool/dbconfig/20220421-210801-ladsgroup.json |
[production] |
| 21:07 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1113:3316 (T306560)', diff saved to https://phabricator.wikimedia.org/P26062 and previous config saved to /var/cache/conftool/dbconfig/20220421-210658-ladsgroup.json |
[production] |
| 21:07 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1113.eqiad.wmnet with reason: Maintenance |
[production] |
| 21:06 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db1113.eqiad.wmnet with reason: Maintenance |
[production] |
| 21:06 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1131 (T306560)', diff saved to https://phabricator.wikimedia.org/P26061 and previous config saved to /var/cache/conftool/dbconfig/20220421-210650-ladsgroup.json |
[production] |
| 21:04 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 21:04 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply |
[production] |
| 21:04 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 21:04 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply |
[production] |
| 21:04 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1157 (T298565)', diff saved to https://phabricator.wikimedia.org/P26060 and previous config saved to /var/cache/conftool/dbconfig/20220421-210414-ladsgroup.json |
[production] |
| 21:04 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1157.eqiad.wmnet with reason: Maintenance |
[production] |
| 21:04 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db1157.eqiad.wmnet with reason: Maintenance |
[production] |
| 20:59 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1136 (T298565)', diff saved to https://phabricator.wikimedia.org/P26059 and previous config saved to /var/cache/conftool/dbconfig/20220421-205906-ladsgroup.json |
[production] |
| 20:52 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1182 (T298565)', diff saved to https://phabricator.wikimedia.org/P26058 and previous config saved to /var/cache/conftool/dbconfig/20220421-205256-ladsgroup.json |
[production] |
| 20:51 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1131', diff saved to https://phabricator.wikimedia.org/P26057 and previous config saved to /var/cache/conftool/dbconfig/20220421-205145-ladsgroup.json |
[production] |
| 20:50 |
<cdanis> |
re-enabled puppet and repooled cp2029 |
[production] |
| 20:47 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1157 (T298565)', diff saved to https://phabricator.wikimedia.org/P26056 and previous config saved to /var/cache/conftool/dbconfig/20220421-204709-ladsgroup.json |
[production] |
| 20:45 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1136 (T298565)', diff saved to https://phabricator.wikimedia.org/P26055 and previous config saved to /var/cache/conftool/dbconfig/20220421-204532-ladsgroup.json |
[production] |
| 20:45 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1136.eqiad.wmnet with reason: Maintenance |
[production] |
| 20:45 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db1136.eqiad.wmnet with reason: Maintenance |
[production] |
| 20:45 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1136 (T298565)', diff saved to https://phabricator.wikimedia.org/P26054 and previous config saved to /var/cache/conftool/dbconfig/20220421-204508-ladsgroup.json |
[production] |
| 20:45 |
<dzahn@cumin2002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on gitlab-runner2001.codfw.wmnet with reason: reimage |
[production] |
| 20:45 |
<dzahn@cumin2002> |
START - Cookbook sre.hosts.downtime for 1:00:00 on gitlab-runner2001.codfw.wmnet with reason: reimage |
[production] |
| 20:41 |
<nokafor@deploy1002> |
Finished deploy [airflow-dags/analytics@bd28d80]: (no justification provided) (duration: 00m 07s) |
[production] |
| 20:41 |
<nokafor@deploy1002> |
Started deploy [airflow-dags/analytics@bd28d80]: (no justification provided) |
[production] |
| 20:39 |
<nokafor@deploy1002> |
Finished deploy [airflow-dags/analytics@bd28d80]: (no justification provided) (duration: 00m 27s) |
[production] |
| 20:39 |
<nokafor@deploy1002> |
Started deploy [airflow-dags/analytics@bd28d80]: (no justification provided) |
[production] |
| 20:36 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1131', diff saved to https://phabricator.wikimedia.org/P26053 and previous config saved to /var/cache/conftool/dbconfig/20220421-203640-ladsgroup.json |
[production] |
| 20:32 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1157', diff saved to https://phabricator.wikimedia.org/P26052 and previous config saved to /var/cache/conftool/dbconfig/20220421-203204-ladsgroup.json |
[production] |
| 20:30 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1136', diff saved to https://phabricator.wikimedia.org/P26051 and previous config saved to /var/cache/conftool/dbconfig/20220421-203003-ladsgroup.json |
[production] |
| 20:28 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1182 (T298565)', diff saved to https://phabricator.wikimedia.org/P26050 and previous config saved to /var/cache/conftool/dbconfig/20220421-202826-ladsgroup.json |
[production] |
| 20:28 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1182.eqiad.wmnet with reason: Maintenance |
[production] |