|
2021-10-19
ยง
|
| 12:04 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1105:3311 (re)pooling @ 5%: After upgrade', diff saved to https://phabricator.wikimedia.org/P17535 and previous config saved to /var/cache/conftool/dbconfig/20211019-120409-root.json |
[production] |
| 12:03 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1101:3318 (re)pooling @ 100%: After upgrade', diff saved to https://phabricator.wikimedia.org/P17534 and previous config saved to /var/cache/conftool/dbconfig/20211019-120348-root.json |
[production] |
| 12:01 |
<urbanecm@deploy1002> |
Synchronized php-1.38.0-wmf.5/extensions/WikibaseMediaInfo/: ec0125770775c1a1a54c3b592d86d287fd9e3ad6: Escape captions when writing stored data into js state (T293556) (duration: 00m 55s) |
[production] |
| 12:01 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1105:3312 (re)pooling @ 5%: After upgrade', diff saved to https://phabricator.wikimedia.org/P17533 and previous config saved to /var/cache/conftool/dbconfig/20211019-120132-root.json |
[production] |
| 12:00 |
<urbanecm@deploy1002> |
Synchronized php-1.38.0-wmf.4/extensions/WikibaseMediaInfo/: 79808a90a95dd5dac2b532b87fb7ec1a490ea0f0: Escape captions when writing stored data into js state (T293556) (duration: 00m 56s) |
[production] |
| 12:00 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1101:3317 (re)pooling @ 100%: After upgrade', diff saved to https://phabricator.wikimedia.org/P17532 and previous config saved to /var/cache/conftool/dbconfig/20211019-120024-root.json |
[production] |
| 11:58 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 11:56 |
<XioNoX> |
push anycast tuning to Tele2, Init7, DT transit links - T288843 |
[production] |
| 11:55 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 11:48 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1101:3318 (re)pooling @ 75%: After upgrade', diff saved to https://phabricator.wikimedia.org/P17531 and previous config saved to /var/cache/conftool/dbconfig/20211019-114844-root.json |
[production] |
| 11:46 |
<marostegui> |
Upgrade db1105 (s1,s2) |
[production] |
| 11:46 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1105 (s1,s2) for upgrade', diff saved to https://phabricator.wikimedia.org/P17530 and previous config saved to /var/cache/conftool/dbconfig/20211019-114649-marostegui.json |
[production] |
| 11:45 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1101:3317 (re)pooling @ 75%: After upgrade', diff saved to https://phabricator.wikimedia.org/P17529 and previous config saved to /var/cache/conftool/dbconfig/20211019-114520-root.json |
[production] |
| 11:33 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1101:3318 (re)pooling @ 50%: After upgrade', diff saved to https://phabricator.wikimedia.org/P17527 and previous config saved to /var/cache/conftool/dbconfig/20211019-113340-root.json |
[production] |
| 11:30 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1101:3317 (re)pooling @ 50%: After upgrade', diff saved to https://phabricator.wikimedia.org/P17526 and previous config saved to /var/cache/conftool/dbconfig/20211019-113017-root.json |
[production] |
| 11:18 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1101:3318 (re)pooling @ 25%: After upgrade', diff saved to https://phabricator.wikimedia.org/P17525 and previous config saved to /var/cache/conftool/dbconfig/20211019-111837-root.json |
[production] |
| 11:15 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1101:3317 (re)pooling @ 25%: After upgrade', diff saved to https://phabricator.wikimedia.org/P17524 and previous config saved to /var/cache/conftool/dbconfig/20211019-111513-root.json |
[production] |
| 11:11 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 11:09 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 11:08 |
<urbanecm@deploy1002> |
Synchronized wmf-config/InitialiseSettings.php: 7c31b04e50101a60db7ae8acae64bc031f5e1007: DPL: Explicitly note it is not possible to enable DPL on any more wikis (duration: 00m 55s) |
[production] |
| 11:03 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1101:3318 (re)pooling @ 10%: After upgrade', diff saved to https://phabricator.wikimedia.org/P17523 and previous config saved to /var/cache/conftool/dbconfig/20211019-110333-root.json |
[production] |
| 11:02 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2025.codfw.wmnet |
[production] |
| 11:00 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1101:3317 (re)pooling @ 10%: After upgrade', diff saved to https://phabricator.wikimedia.org/P17522 and previous config saved to /var/cache/conftool/dbconfig/20211019-110009-root.json |
[production] |
| 10:56 |
<marostegui> |
Upgrade clouddb1021 |
[production] |
| 10:53 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host ganeti2025.codfw.wmnet |
[production] |
| 10:51 |
<moritzm> |
failover master in ganeti-test to ganeti2026 |
[production] |
| 10:50 |
<godog> |
bounce superset on an-tool1005 to pick up statsd changes - T247963 |
[production] |
| 10:49 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2112.codfw.wmnet with OS stretch |
[production] |
| 10:48 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1101:3318 (re)pooling @ 5%: After upgrade', diff saved to https://phabricator.wikimedia.org/P17521 and previous config saved to /var/cache/conftool/dbconfig/20211019-104829-root.json |
[production] |
| 10:45 |
<godog> |
bounce navtiming on webperf1001 to pick up statsd changes - T247963 |
[production] |
| 10:45 |
<godog> |
bounce superset on an-tool1010 to pick up statsd changes - T247963 |
[production] |
| 10:45 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1101:3317 (re)pooling @ 5%: After upgrade', diff saved to https://phabricator.wikimedia.org/P17520 and previous config saved to /var/cache/conftool/dbconfig/20211019-104506-root.json |
[production] |
| 10:38 |
<oblivian@deploy1002> |
Synchronized w/static.php: Config: [[gerrit:730182|static.php: Add support for /static/current rewrites (take 2) (T285232)]] (duration: 00m 55s) |
[production] |
| 10:38 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti2026.codfw.wmnet |
[production] |
| 10:37 |
<marostegui> |
Upgrade db1101 (s7,s8) |
[production] |
| 10:36 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1101 (s7,s8) for upgrade', diff saved to https://phabricator.wikimedia.org/P17519 and previous config saved to /var/cache/conftool/dbconfig/20211019-103634-marostegui.json |
[production] |
| 10:34 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 10:31 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 10:29 |
<elukey@deploy1002> |
helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'sync'. |
[production] |
| 10:28 |
<elukey@deploy1002> |
helmfile [ml-serve-eqiad] START helmfile.d/admin 'sync'. |
[production] |
| 10:27 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reboot-single for host ganeti2026.codfw.wmnet |
[production] |
| 10:22 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 10:22 |
<oblivian@deploy1002> |
Synchronized tests/WmfConfigServicesTest.php: Config: [[gerrit:731918|ProductionServices: use graphite2003 for statsd (T247963)]] (duration: 00m 54s) |
[production] |
| 10:22 |
<godog> |
flip mw statsd traffic with https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/731918 - T247963 |
[production] |
| 10:21 |
<oblivian@deploy1002> |
Synchronized wmf-config/ProductionServices.php: Config: [[gerrit:731918|ProductionServices: use graphite2003 for statsd (T247963)]] (duration: 00m 54s) |
[production] |
| 10:20 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] Ran 'sync' command on namespace 'mwdebug' for release 'pinkunicorn' . |
[production] |
| 10:18 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.reimage for host db2112.codfw.wmnet with OS stretch |
[production] |
| 10:16 |
<marostegui@cumin1001> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host db2112.codfw.wmnet with OS buster |
[production] |
| 09:52 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.reimage for host db2112.codfw.wmnet with OS buster |
[production] |
| 09:50 |
<marostegui@cumin1001> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host db2112.codfw.wmnet with OS buster |
[production] |