|
2021-04-08
§
|
| 09:29 |
<akosiaris@deploy1002> |
helmfile [ml-serve-codfw] START helmfile.d/admin 'sync'. |
[production] |
| 09:27 |
<zpapierski@deploy1002> |
Finished deploy [wikimedia/discovery/analytics@d098717]: T273847 export queries to relforge dag deployment - sensor name fix (duration: 01m 48s) |
[production] |
| 09:26 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1166 (re)pooling @ 25%: Repool after schema change', diff saved to https://phabricator.wikimedia.org/P15243 and previous config saved to /var/cache/conftool/dbconfig/20210408-092608-root.json |
[production] |
| 09:25 |
<zpapierski@deploy1002> |
Started deploy [wikimedia/discovery/analytics@d098717]: T273847 export queries to relforge dag deployment - sensor name fix |
[production] |
| 09:24 |
<moritzm> |
installing libzstd security updates on buster |
[production] |
| 09:20 |
<ema> |
cp5001: varnish-frontend-restart to test exp policy settings starting from a empty cache T275809 |
[production] |
| 09:14 |
<akosiaris@deploy1002> |
helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'sync'. |
[production] |
| 09:14 |
<akosiaris@deploy1002> |
helmfile [ml-serve-eqiad] START helmfile.d/admin 'sync'. |
[production] |
| 09:09 |
<moritzm> |
installing underscore security updates on stretch |
[production] |
| 08:56 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1166 for schema change', diff saved to https://phabricator.wikimedia.org/P15242 and previous config saved to /var/cache/conftool/dbconfig/20210408-085630-marostegui.json |
[production] |
| 08:56 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1175 (re)pooling @ 100%: Repool after schema change', diff saved to https://phabricator.wikimedia.org/P15241 and previous config saved to /var/cache/conftool/dbconfig/20210408-085610-root.json |
[production] |
| 08:44 |
<akosiaris@deploy1002> |
helmfile [eqiad] DONE helmfile.d/admin 'sync'. |
[production] |
| 08:43 |
<akosiaris@deploy1002> |
helmfile [eqiad] START helmfile.d/admin 'sync'. |
[production] |
| 08:41 |
<akosiaris@deploy1002> |
helmfile [codfw] DONE helmfile.d/admin 'sync'. |
[production] |
| 08:41 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1175 (re)pooling @ 75%: Repool after schema change', diff saved to https://phabricator.wikimedia.org/P15240 and previous config saved to /var/cache/conftool/dbconfig/20210408-084107-root.json |
[production] |
| 08:40 |
<akosiaris@deploy1002> |
helmfile [codfw] START helmfile.d/admin 'sync'. |
[production] |
| 08:39 |
<akosiaris@deploy1002> |
helmfile [staging-codfw] DONE helmfile.d/admin 'sync'. |
[production] |
| 08:38 |
<akosiaris@deploy1002> |
helmfile [staging-codfw] START helmfile.d/admin 'sync'. |
[production] |
| 08:38 |
<akosiaris@deploy1002> |
helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. |
[production] |
| 08:37 |
<akosiaris@deploy1002> |
helmfile [staging-eqiad] START helmfile.d/admin 'apply'. |
[production] |
| 08:33 |
<moritzm> |
installing remaining curl security updates for buster |
[production] |
| 08:26 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1175 (re)pooling @ 50%: Repool after schema change', diff saved to https://phabricator.wikimedia.org/P15239 and previous config saved to /var/cache/conftool/dbconfig/20210408-082603-root.json |
[production] |
| 08:24 |
<marostegui> |
Stop MySQL on all db1117 sections to upgrade kernel |
[production] |
| 08:17 |
<moritzm> |
imported postgis 3.1.1+dfsg-1~wmf1 to component/postgis for buster-wikimedia T277064 |
[production] |
| 08:15 |
<filippo@cumin1001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be2028.codfw.wmnet |
[production] |
| 08:11 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1175 (re)pooling @ 25%: Repool after schema change', diff saved to https://phabricator.wikimedia.org/P15238 and previous config saved to /var/cache/conftool/dbconfig/20210408-081059-root.json |
[production] |
| 08:03 |
<filippo@cumin1001> |
START - Cookbook sre.hosts.reboot-single for host ms-be2028.codfw.wmnet |
[production] |
| 07:54 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es1023 (re)pooling @ 100%: Repool es1023', diff saved to https://phabricator.wikimedia.org/P15237 and previous config saved to /var/cache/conftool/dbconfig/20210408-075457-root.json |
[production] |
| 07:50 |
<filippo@cumin1001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be1020.eqiad.wmnet |
[production] |
| 07:49 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Remove weight from es5 master', diff saved to https://phabricator.wikimedia.org/P15236 and previous config saved to /var/cache/conftool/dbconfig/20210408-074911-marostegui.json |
[production] |
| 07:45 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1175 for schema change', diff saved to https://phabricator.wikimedia.org/P15235 and previous config saved to /var/cache/conftool/dbconfig/20210408-074524-marostegui.json |
[production] |
| 07:42 |
<filippo@cumin1001> |
START - Cookbook sre.hosts.reboot-single for host ms-be1020.eqiad.wmnet |
[production] |
| 07:39 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es1023 (re)pooling @ 75%: Repool es1023', diff saved to https://phabricator.wikimedia.org/P15234 and previous config saved to /var/cache/conftool/dbconfig/20210408-073953-root.json |
[production] |
| 07:24 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es1023 (re)pooling @ 50%: Repool es1023', diff saved to https://phabricator.wikimedia.org/P15233 and previous config saved to /var/cache/conftool/dbconfig/20210408-072450-root.json |
[production] |
| 07:09 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'es1023 (re)pooling @ 25%: Repool es1023', diff saved to https://phabricator.wikimedia.org/P15232 and previous config saved to /var/cache/conftool/dbconfig/20210408-070946-root.json |
[production] |
| 06:56 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool es1023 to upgrade kernel and mysql, remove weight from es1021, to leave it as it was yesterday T279281', diff saved to https://phabricator.wikimedia.org/P15231 and previous config saved to /var/cache/conftool/dbconfig/20210408-065627-marostegui.json |
[production] |
| 06:44 |
<elukey@deploy1002> |
Finished deploy [analytics/refinery@1dbbd3d] (hadoop-test): (no justification provided) (duration: 02m 20s) |
[production] |
| 06:41 |
<elukey@deploy1002> |
Started deploy [analytics/refinery@1dbbd3d] (hadoop-test): (no justification provided) |
[production] |
| 06:33 |
<marostegui> |
Stop MySQL on db1111 to clone db1177 T275633 |
[production] |
| 06:33 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1111 to clone db1177 T275633', diff saved to https://phabricator.wikimedia.org/P15229 and previous config saved to /var/cache/conftool/dbconfig/20210408-063331-marostegui.json |
[production] |
| 06:01 |
<kart_> |
Updated cxserver to 2021-04-07-062518-production (T278141, T263139, T271711, T201491, T240525, T207662) |
[production] |
| 05:58 |
<kartik@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'cxserver' for release 'production' . |
[production] |
| 05:54 |
<kartik@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'cxserver' for release 'production' . |
[production] |
| 05:43 |
<kartik@deploy1002> |
helmfile [staging] Ran 'sync' command on namespace 'cxserver' for release 'staging' . |
[production] |
| 02:50 |
<AaronSchulz> |
Restarted importMissingLocalNames.php (mwmaint 1002, wiki=metawiki,batch-size=1000) |
[production] |