|
2022-04-26
ยง
|
| 20:08 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply |
[production] |
| 20:08 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply |
[production] |
| 20:08 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply |
[production] |
| 20:05 |
<urbanecm@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: 080b8fc573d9d682038e09a7a7ad875bce478c00: cirrus: Turn on retry_on_conflict quirk (duration: 00m 53s) |
[production] |
| 20:01 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1161', diff saved to https://phabricator.wikimedia.org/P26614 and previous config saved to /var/cache/conftool/dbconfig/20220426-200105-ladsgroup.json |
[production] |
| 19:54 |
<dzahn@cumin2002> |
conftool action : set/pooled=yes; selector: dc=codfw,name=mw2419.codfw.wmnet |
[production] |
| 19:53 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1164', diff saved to https://phabricator.wikimedia.org/P26612 and previous config saved to /var/cache/conftool/dbconfig/20220426-195349-ladsgroup.json |
[production] |
| 19:48 |
<mutante> |
mw2419 - set weight to 25 in conftool, scap pull, first time in production, jobrunner/videoscaler T290192 |
[production] |
| 19:46 |
<dzahn@cumin2002> |
conftool action : set/weight=25; selector: dc=codfw,name=mw2419.codfw.wmnet |
[production] |
| 19:46 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1161', diff saved to https://phabricator.wikimedia.org/P26611 and previous config saved to /var/cache/conftool/dbconfig/20220426-194600-ladsgroup.json |
[production] |
| 19:45 |
<dzahn@cumin2002> |
conftool action : set/pooled=no; selector: dc=codfw,name=mw2419.codfw.wmnet |
[production] |
| 19:42 |
<aqu@deploy1002> |
Finished deploy [analytics/refinery@96a3934] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@96a3934] (duration: 07m 19s) |
[production] |
| 19:38 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1164 (T306560)', diff saved to https://phabricator.wikimedia.org/P26610 and previous config saved to /var/cache/conftool/dbconfig/20220426-193844-ladsgroup.json |
[production] |
| 19:36 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1164 (T306560)', diff saved to https://phabricator.wikimedia.org/P26609 and previous config saved to /var/cache/conftool/dbconfig/20220426-193610-ladsgroup.json |
[production] |
| 19:36 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1164.eqiad.wmnet with reason: Maintenance |
[production] |
| 19:36 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db1164.eqiad.wmnet with reason: Maintenance |
[production] |
| 19:36 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1106 (T306560)', diff saved to https://phabricator.wikimedia.org/P26608 and previous config saved to /var/cache/conftool/dbconfig/20220426-193602-ladsgroup.json |
[production] |
| 19:34 |
<aqu@deploy1002> |
Started deploy [analytics/refinery@96a3934] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@96a3934] |
[production] |
| 19:34 |
<aqu@deploy1002> |
Finished deploy [analytics/refinery@96a3934] (thin): Regular analytics weekly train THIN [analytics/refinery@96a3934] (duration: 00m 07s) |
[production] |
| 19:34 |
<aqu@deploy1002> |
Started deploy [analytics/refinery@96a3934] (thin): Regular analytics weekly train THIN [analytics/refinery@96a3934] |
[production] |
| 19:30 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1161 (T298556)', diff saved to https://phabricator.wikimedia.org/P26607 and previous config saved to /var/cache/conftool/dbconfig/20220426-193055-ladsgroup.json |
[production] |
| 19:30 |
<aqu@deploy1002> |
Finished deploy [analytics/refinery@96a3934]: Regular analytics weekly train [analytics/refinery@96a3934] (duration: 24m 35s) |
[production] |
| 19:28 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1161 (T298556)', diff saved to https://phabricator.wikimedia.org/P26606 and previous config saved to /var/cache/conftool/dbconfig/20220426-192841-ladsgroup.json |
[production] |
| 19:28 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 20:00:00 on clouddb[1016,1020-1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance |
[production] |
| 19:28 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 20:00:00 on clouddb[1016,1020-1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance |
[production] |
| 19:28 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on db1161.eqiad.wmnet with reason: Maintenance |
[production] |
| 19:28 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 10:00:00 on db1161.eqiad.wmnet with reason: Maintenance |
[production] |
| 19:28 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1096:3315 (T298556)', diff saved to https://phabricator.wikimedia.org/P26605 and previous config saved to /var/cache/conftool/dbconfig/20220426-192828-ladsgroup.json |
[production] |
| 19:20 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1106', diff saved to https://phabricator.wikimedia.org/P26604 and previous config saved to /var/cache/conftool/dbconfig/20220426-192057-ladsgroup.json |
[production] |
| 19:13 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1096:3315', diff saved to https://phabricator.wikimedia.org/P26603 and previous config saved to /var/cache/conftool/dbconfig/20220426-191323-ladsgroup.json |
[production] |
| 19:06 |
<aqu@deploy1002> |
Started deploy [analytics/refinery@96a3934]: Regular analytics weekly train [analytics/refinery@96a3934] |
[production] |
| 19:05 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1106', diff saved to https://phabricator.wikimedia.org/P26602 and previous config saved to /var/cache/conftool/dbconfig/20220426-190552-ladsgroup.json |
[production] |
| 19:02 |
<aqu> |
About to deploy analytics/refinery: Weekly deployment train + Artifacts to 0.1.27 |
[production] |
| 18:58 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1096:3315', diff saved to https://phabricator.wikimedia.org/P26601 and previous config saved to /var/cache/conftool/dbconfig/20220426-185818-ladsgroup.json |
[production] |
| 18:50 |
<cmooney@cumin1001> |
END (PASS) - Cookbook sre.deploy.python-code (exit_code=0) homer to cumin2002.codfw.wmnet,cumin1001.eqiad.wmnet with reason: Release v0.4.1a - cmooney@cumin1001 |
[production] |
| 18:50 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1106 (T306560)', diff saved to https://phabricator.wikimedia.org/P26599 and previous config saved to /var/cache/conftool/dbconfig/20220426-185047-ladsgroup.json |
[production] |
| 18:49 |
<cmooney@cumin1001> |
START - Cookbook sre.deploy.python-code homer to cumin2002.codfw.wmnet,cumin1001.eqiad.wmnet with reason: Release v0.4.1a - cmooney@cumin1001 |
[production] |
| 18:48 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1106 (T306560)', diff saved to https://phabricator.wikimedia.org/P26598 and previous config saved to /var/cache/conftool/dbconfig/20220426-184815-ladsgroup.json |
[production] |
| 18:48 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on clouddb[1013,1017,1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance |
[production] |
| 18:48 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 12:00:00 on clouddb[1013,1017,1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance |
[production] |
| 18:48 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1106.eqiad.wmnet with reason: Maintenance |
[production] |
| 18:48 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db1106.eqiad.wmnet with reason: Maintenance |
[production] |
| 18:47 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1133.eqiad.wmnet with reason: Maintenance |
[production] |
| 18:47 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 6:00:00 on db1133.eqiad.wmnet with reason: Maintenance |
[production] |
| 18:47 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1132 (T306560)', diff saved to https://phabricator.wikimedia.org/P26597 and previous config saved to /var/cache/conftool/dbconfig/20220426-184729-ladsgroup.json |
[production] |
| 18:43 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db1096:3315 (T298556)', diff saved to https://phabricator.wikimedia.org/P26596 and previous config saved to /var/cache/conftool/dbconfig/20220426-184313-ladsgroup.json |
[production] |
| 18:41 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Depooling db1096:3315 (T298556)', diff saved to https://phabricator.wikimedia.org/P26595 and previous config saved to /var/cache/conftool/dbconfig/20220426-184058-ladsgroup.json |
[production] |
| 18:40 |
<ladsgroup@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on db1096.eqiad.wmnet with reason: Maintenance |
[production] |
| 18:40 |
<ladsgroup@cumin1001> |
START - Cookbook sre.hosts.downtime for 10:00:00 on db1096.eqiad.wmnet with reason: Maintenance |
[production] |
| 18:37 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: apply |
[production] |