|
2023-02-09
ยง
|
| 10:31 |
<joal@deploy1002> |
Finished deploy [airflow-dags/analytics@2ab6564]: Analytics deploy for 3 druid jobs and webrequest_actor jobs (duration: 00m 17s) |
[production] |
| 10:31 |
<joal@deploy1002> |
Started deploy [airflow-dags/analytics@2ab6564]: Analytics deploy for 3 druid jobs and webrequest_actor jobs |
[production] |
| 10:27 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1107 (re)pooling @ 75%: Repooling', diff saved to https://phabricator.wikimedia.org/P43968 and previous config saved to /var/cache/conftool/dbconfig/20230209-102713-root.json |
[production] |
| 10:27 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1132 (re)pooling @ 75%: Repooling', diff saved to https://phabricator.wikimedia.org/P43967 and previous config saved to /var/cache/conftool/dbconfig/20230209-102709-root.json |
[production] |
| 10:22 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2137:3315', diff saved to https://phabricator.wikimedia.org/P43966 and previous config saved to /var/cache/conftool/dbconfig/20230209-102206-marostegui.json |
[production] |
| 10:12 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1107 (re)pooling @ 50%: Repooling', diff saved to https://phabricator.wikimedia.org/P43965 and previous config saved to /var/cache/conftool/dbconfig/20230209-101209-root.json |
[production] |
| 10:12 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1132 (re)pooling @ 50%: Repooling', diff saved to https://phabricator.wikimedia.org/P43964 and previous config saved to /var/cache/conftool/dbconfig/20230209-101204-root.json |
[production] |
| 10:07 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2137:3315', diff saved to https://phabricator.wikimedia.org/P43963 and previous config saved to /var/cache/conftool/dbconfig/20230209-100700-marostegui.json |
[production] |
| 10:01 |
<kostajh> |
UTC morning deploys really done |
[production] |
| 09:59 |
<kharlan@deploy1002> |
Finished scap: Backport for [[gerrit:887848|ComputedUserImpactLookup: Reduce logspam for page view rate limiting (T328945)]] (duration: 09m 06s) |
[production] |
| 09:57 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1107 (re)pooling @ 25%: Repooling', diff saved to https://phabricator.wikimedia.org/P43962 and previous config saved to /var/cache/conftool/dbconfig/20230209-095704-root.json |
[production] |
| 09:57 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1132 (re)pooling @ 25%: Repooling', diff saved to https://phabricator.wikimedia.org/P43961 and previous config saved to /var/cache/conftool/dbconfig/20230209-095659-root.json |
[production] |
| 09:51 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2137:3315 (T328817)', diff saved to https://phabricator.wikimedia.org/P43960 and previous config saved to /var/cache/conftool/dbconfig/20230209-095153-marostegui.json |
[production] |
| 09:51 |
<kharlan@deploy1002> |
kharlan: Backport for [[gerrit:887848|ComputedUserImpactLookup: Reduce logspam for page view rate limiting (T328945)]] synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet |
[production] |
| 09:49 |
<kharlan@deploy1002> |
Started scap: Backport for [[gerrit:887848|ComputedUserImpactLookup: Reduce logspam for page view rate limiting (T328945)]] |
[production] |
| 09:48 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db2137:3315 (T328817)', diff saved to https://phabricator.wikimedia.org/P43959 and previous config saved to /var/cache/conftool/dbconfig/20230209-094816-marostegui.json |
[production] |
| 09:48 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2137.codfw.wmnet with reason: Maintenance |
[production] |
| 09:47 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 12:00:00 on db2137.codfw.wmnet with reason: Maintenance |
[production] |
| 09:47 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2128 (T328817)', diff saved to https://phabricator.wikimedia.org/P43958 and previous config saved to /var/cache/conftool/dbconfig/20230209-094755-marostegui.json |
[production] |
| 09:42 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1107 (re)pooling @ 10%: Repooling', diff saved to https://phabricator.wikimedia.org/P43957 and previous config saved to /var/cache/conftool/dbconfig/20230209-094159-root.json |
[production] |
| 09:41 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1132 (re)pooling @ 10%: Repooling', diff saved to https://phabricator.wikimedia.org/P43956 and previous config saved to /var/cache/conftool/dbconfig/20230209-094154-root.json |
[production] |
| 09:32 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2128', diff saved to https://phabricator.wikimedia.org/P43955 and previous config saved to /var/cache/conftool/dbconfig/20230209-093248-marostegui.json |
[production] |
| 09:32 |
<godog> |
roll-restart opensearch-dashboards to apply memory limit - T327161 |
[production] |
| 09:29 |
<kharlan@deploy1002> |
Finished scap: Backport for [[gerrit:887849|Add StatusValue::hasMessagesExcept() (T272081)]] (duration: 09m 20s) |
[production] |
| 09:26 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1107 (re)pooling @ 5%: Repooling', diff saved to https://phabricator.wikimedia.org/P43954 and previous config saved to /var/cache/conftool/dbconfig/20230209-092654-root.json |
[production] |
| 09:26 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1132 (re)pooling @ 5%: Repooling', diff saved to https://phabricator.wikimedia.org/P43953 and previous config saved to /var/cache/conftool/dbconfig/20230209-092650-root.json |
[production] |
| 09:22 |
<kharlan@deploy1002> |
kharlan: Backport for [[gerrit:887849|Add StatusValue::hasMessagesExcept() (T272081)]] synced to the testservers: mwdebug1001.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet |
[production] |
| 09:20 |
<kharlan@deploy1002> |
Started scap: Backport for [[gerrit:887849|Add StatusValue::hasMessagesExcept() (T272081)]] |
[production] |
| 09:17 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2128', diff saved to https://phabricator.wikimedia.org/P43952 and previous config saved to /var/cache/conftool/dbconfig/20230209-091742-marostegui.json |
[production] |
| 09:13 |
<moritzm> |
installing openssl security updates on Bullseye |
[production] |
| 09:11 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1107 (re)pooling @ 1%: Repooling', diff saved to https://phabricator.wikimedia.org/P43951 and previous config saved to /var/cache/conftool/dbconfig/20230209-091149-root.json |
[production] |
| 09:11 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1132 (re)pooling @ 1%: Repooling', diff saved to https://phabricator.wikimedia.org/P43950 and previous config saved to /var/cache/conftool/dbconfig/20230209-091145-root.json |
[production] |
| 09:10 |
<marostegui> |
Install 10.4.28 on db1107 T329011 |
[production] |
| 09:10 |
<marostegui> |
Install 10.6.12 on db1132 T329011 |
[production] |
| 09:09 |
<vgutierrez> |
pool cp4044 with ESI testing enabled |
[production] |
| 09:08 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1107 db1132', diff saved to https://phabricator.wikimedia.org/P43949 and previous config saved to /var/cache/conftool/dbconfig/20230209-090846-root.json |
[production] |
| 09:02 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2128 (T328817)', diff saved to https://phabricator.wikimedia.org/P43948 and previous config saved to /var/cache/conftool/dbconfig/20230209-090236-marostegui.json |
[production] |
| 09:00 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depooling db2128 (T328817)', diff saved to https://phabricator.wikimedia.org/P43947 and previous config saved to /var/cache/conftool/dbconfig/20230209-090018-marostegui.json |
[production] |
| 09:00 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db2094.codfw.wmnet with reason: Maintenance |
[production] |
| 09:00 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db2094.codfw.wmnet with reason: Maintenance |
[production] |
| 09:00 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2128.codfw.wmnet with reason: Maintenance |
[production] |
| 08:59 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 12:00:00 on db2128.codfw.wmnet with reason: Maintenance |
[production] |
| 08:59 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2123 (T328817)', diff saved to https://phabricator.wikimedia.org/P43946 and previous config saved to /var/cache/conftool/dbconfig/20230209-085952-marostegui.json |
[production] |
| 08:57 |
<vgutierrez> |
depool cp4044 - T308799 |
[production] |
| 08:44 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2123', diff saved to https://phabricator.wikimedia.org/P43945 and previous config saved to /var/cache/conftool/dbconfig/20230209-084446-marostegui.json |
[production] |
| 08:42 |
<apergos> |
UTC morning backport and config training window complete |
[production] |
| 08:32 |
<kartik@deploy1002> |
Finished scap: Backport for [[gerrit:887847|CX: Provide the appropriate arguments to ve.ui.CXSurface constructor (T329154)]] (duration: 13m 10s) |
[production] |
| 08:29 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2123', diff saved to https://phabricator.wikimedia.org/P43944 and previous config saved to /var/cache/conftool/dbconfig/20230209-082940-marostegui.json |
[production] |
| 08:20 |
<kartik@deploy1002> |
kartik: Backport for [[gerrit:887847|CX: Provide the appropriate arguments to ve.ui.CXSurface constructor (T329154)]] synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug1001.eqiad.wmnet, mwdebug2001.codfw.wmnet |
[production] |
| 08:19 |
<kartik@deploy1002> |
Started scap: Backport for [[gerrit:887847|CX: Provide the appropriate arguments to ve.ui.CXSurface constructor (T329154)]] |
[production] |