|
2021-09-03
§
|
| 10:04 |
<hnowlan@deploy1002> |
Started deploy [analytics/aqs/deploy@d273fde] (aqs-next): deploying aqs to inactive aqs-next hosts |
[production] |
| 10:02 |
<hnowlan@deploy1002> |
Finished deploy [analytics/aqs/deploy@d273fde] (aqs-next): deploying aqs to inactive aqs-next hosts (duration: 01m 25s) |
[production] |
| 10:01 |
<hnowlan@deploy1002> |
Started deploy [analytics/aqs/deploy@d273fde] (aqs-next): deploying aqs to inactive aqs-next hosts |
[production] |
| 10:00 |
<hnowlan@deploy1002> |
Finished deploy [analytics/aqs/deploy@d273fde] (aqs-next): deploying aqs to inactive aqs-next hosts (duration: 01m 53s) |
[production] |
| 09:58 |
<hnowlan@deploy1002> |
Started deploy [analytics/aqs/deploy@d273fde] (aqs-next): deploying aqs to inactive aqs-next hosts |
[production] |
| 09:57 |
<hnowlan@deploy1002> |
Finished deploy [analytics/aqs/deploy@d273fde] (aqs-next): deploying aqs to inactive aqs-next hosts (duration: 00m 09s) |
[production] |
| 09:57 |
<hnowlan@deploy1002> |
Started deploy [analytics/aqs/deploy@d273fde] (aqs-next): deploying aqs to inactive aqs-next hosts |
[production] |
| 09:32 |
<joal@deploy1002> |
Finished deploy [analytics/refinery@4ff8979] (thin): Analytics hotfix deploy THIN [analytics/refinery@4ff8979] (duration: 00m 07s) |
[production] |
| 09:32 |
<joal@deploy1002> |
Started deploy [analytics/refinery@4ff8979] (thin): Analytics hotfix deploy THIN [analytics/refinery@4ff8979] |
[production] |
| 09:26 |
<joal@deploy1002> |
Finished deploy [analytics/refinery@4ff8979]: Analytics hotfix deploy [analytics/refinery@4ff8979] (duration: 17m 36s) |
[production] |
| 09:25 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.decommission for hosts mc[1025-1026].eqiad.wmnet |
[production] |
| 09:15 |
<jelto@deploy1002> |
helmfile [codfw] DONE helmfile.d/admin 'apply'. |
[production] |
| 09:14 |
<jiji@cumin1001> |
END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts mc1022.eqiad.wmnet |
[production] |
| 09:13 |
<jelto@deploy1002> |
helmfile [codfw] START helmfile.d/admin 'apply'. |
[production] |
| 09:09 |
<akosiaris@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mathoid' for release 'production' . |
[production] |
| 09:09 |
<jelto@deploy1002> |
helmfile [eqiad] DONE helmfile.d/admin 'apply'. |
[production] |
| 09:09 |
<joal@deploy1002> |
Started deploy [analytics/refinery@4ff8979]: Analytics hotfix deploy [analytics/refinery@4ff8979] |
[production] |
| 09:08 |
<jelto@deploy1002> |
helmfile [eqiad] START helmfile.d/admin 'apply'. |
[production] |
| 09:06 |
<akosiaris@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mathoid' for release 'production' . |
[production] |
| 09:03 |
<jelto@deploy1002> |
helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. |
[production] |
| 09:03 |
<jelto@deploy1002> |
helmfile [staging-eqiad] START helmfile.d/admin 'apply'. |
[production] |
| 08:53 |
<jelto@deploy1002> |
helmfile [staging-codfw] DONE helmfile.d/admin 'apply'. |
[production] |
| 08:52 |
<jelto@deploy1002> |
helmfile [staging-codfw] START helmfile.d/admin 'apply'. |
[production] |
| 08:45 |
<ema> |
cp-eqsin: clean apt cache to free up some space T290305 |
[production] |
| 08:45 |
<jiji@cumin1001> |
START - Cookbook sre.hosts.decommission for hosts mc1022.eqiad.wmnet |
[production] |
| 08:23 |
<akosiaris@deploy1002> |
helmfile [staging] Ran 'sync' command on namespace 'mathoid' for release 'staging' . |
[production] |
| 07:43 |
<legoktm> |
uploaded pygments 2.10.0+dfsg-1~wmf1 to apt.wm.o in component/pygments |
[production] |
| 07:42 |
<marostegui> |
Remove flaggedrevs_stats2 and flaggedrevs_stats from severak s3 wikis - T289050 |
[production] |
| 07:10 |
<godog> |
more weight to ms-be20[62-65] - T288458 |
[production] |
| 07:01 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.dns.netbox (exit_code=0) |
[production] |
| 06:57 |
<marostegui@cumin1001> |
START - Cookbook sre.dns.netbox |
[production] |
| 06:45 |
<elukey> |
run `apt-get clean` on cp5012 to free some space (94% of the root partition used) |
[production] |
| 06:12 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db2138:3314 (re)pooling @ 100%: Slowly repool after reimage T288803', diff saved to https://phabricator.wikimedia.org/P17203 and previous config saved to /var/cache/conftool/dbconfig/20210903-061204-root.json |
[production] |
| 06:11 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db2138:3312 (re)pooling @ 100%: Slowly repool after reimage T288803', diff saved to https://phabricator.wikimedia.org/P17202 and previous config saved to /var/cache/conftool/dbconfig/20210903-061138-root.json |
[production] |
| 05:57 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db2138:3314 (re)pooling @ 75%: Slowly repool after reimage T288803', diff saved to https://phabricator.wikimedia.org/P17201 and previous config saved to /var/cache/conftool/dbconfig/20210903-055700-root.json |
[production] |
| 05:56 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db2138:3312 (re)pooling @ 75%: Slowly repool after reimage T288803', diff saved to https://phabricator.wikimedia.org/P17200 and previous config saved to /var/cache/conftool/dbconfig/20210903-055635-root.json |
[production] |
| 05:41 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db2138:3314 (re)pooling @ 50%: Slowly repool after reimage T288803', diff saved to https://phabricator.wikimedia.org/P17199 and previous config saved to /var/cache/conftool/dbconfig/20210903-054157-root.json |
[production] |
| 05:41 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db2138:3312 (re)pooling @ 50%: Slowly repool after reimage T288803', diff saved to https://phabricator.wikimedia.org/P17198 and previous config saved to /var/cache/conftool/dbconfig/20210903-054131-root.json |
[production] |
| 05:30 |
<marostegui@cumin1001> |
END (FAIL) - Cookbook sre.hosts.decommission (exit_code=1) for hosts pc2007.codfw.wmnet |
[production] |
| 05:26 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db2138:3314 (re)pooling @ 25%: Slowly repool after reimage T288803', diff saved to https://phabricator.wikimedia.org/P17196 and previous config saved to /var/cache/conftool/dbconfig/20210903-052653-root.json |
[production] |
| 05:26 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db2138:3312 (re)pooling @ 25%: Slowly repool after reimage T288803', diff saved to https://phabricator.wikimedia.org/P17195 and previous config saved to /var/cache/conftool/dbconfig/20210903-052628-root.json |
[production] |
| 05:20 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.decommission for hosts pc2007.codfw.wmnet |
[production] |
| 05:11 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db2138:3314 (re)pooling @ 10%: Slowly repool after reimage T288803', diff saved to https://phabricator.wikimedia.org/P17194 and previous config saved to /var/cache/conftool/dbconfig/20210903-051149-root.json |
[production] |
| 05:11 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db2138:3312 (re)pooling @ 10%: Slowly repool after reimage T288803', diff saved to https://phabricator.wikimedia.org/P17193 and previous config saved to /var/cache/conftool/dbconfig/20210903-051124-root.json |
[production] |
| 05:04 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db2138 for upgrade', diff saved to https://phabricator.wikimedia.org/P17192 and previous config saved to /var/cache/conftool/dbconfig/20210903-050423-marostegui.json |
[production] |
| 00:31 |
<tgr@deploy1002> |
Synchronized php-1.37.0-wmf.21/extensions/GrowthExperiments/maintenance/fixLinkRecommendationData.php: Backport: [[gerrit:716491|fixLinkRecommendationData: Try harder to avoid >10K result sets (T284531)]] (duration: 00m 58s) |
[production] |
| 00:25 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 00:23 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |