|
2023-04-19
ยง
|
| 09:01 |
<elukey@deploy2002> |
helmfile [codfw] DONE helmfile.d/services/eventgate-logging-external: sync |
[production] |
| 09:00 |
<elukey@deploy2002> |
helmfile [codfw] START helmfile.d/services/eventgate-logging-external: sync |
[production] |
| 09:00 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1113:3316 (re)pooling @ 5%: Repooling', diff saved to https://phabricator.wikimedia.org/P47236 and previous config saved to /var/cache/conftool/dbconfig/20230419-090050-root.json |
[production] |
| 09:00 |
<elukey@deploy2002> |
helmfile [eqiad] DONE helmfile.d/services/eventgate-logging-external: sync |
[production] |
| 08:59 |
<elukey@deploy2002> |
helmfile [eqiad] START helmfile.d/services/eventgate-logging-external: sync |
[production] |
| 08:59 |
<gmodena@deploy1002> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mediawiki-page-content-change-enrichment: apply |
[production] |
| 08:59 |
<gmodena@deploy1002> |
helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mediawiki-page-content-change-enrichment: apply |
[production] |
| 08:57 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1110 (re)pooling @ 10%: Repooling', diff saved to https://phabricator.wikimedia.org/P47235 and previous config saved to /var/cache/conftool/dbconfig/20230419-085748-root.json |
[production] |
| 08:52 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1219 (re)pooling @ 10%: Pooling', diff saved to https://phabricator.wikimedia.org/P47234 and previous config saved to /var/cache/conftool/dbconfig/20230419-085257-root.json |
[production] |
| 08:52 |
<jmm@cumin2002> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host sretest1002.eqiad.wmnet with OS bookworm |
[production] |
| 08:52 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1113:3315 (re)pooling @ 5%: Repooling', diff saved to https://phabricator.wikimedia.org/P47233 and previous config saved to /var/cache/conftool/dbconfig/20230419-085222-root.json |
[production] |
| 08:45 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1113:3316 (re)pooling @ 4%: Repooling', diff saved to https://phabricator.wikimedia.org/P47232 and previous config saved to /var/cache/conftool/dbconfig/20230419-084545-root.json |
[production] |
| 08:45 |
<gmodena@deploy1002> |
helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mediawiki-page-content-change-enrichment: apply |
[production] |
| 08:45 |
<gmodena@deploy1002> |
helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mediawiki-page-content-change-enrichment: apply |
[production] |
| 08:43 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on sretest1002.eqiad.wmnet with reason: host reimage |
[production] |
| 08:42 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1110 (re)pooling @ 5%: Repooling', diff saved to https://phabricator.wikimedia.org/P47231 and previous config saved to /var/cache/conftool/dbconfig/20230419-084243-root.json |
[production] |
| 08:40 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.downtime for 2:00:00 on sretest1002.eqiad.wmnet with reason: host reimage |
[production] |
| 08:37 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1219 (re)pooling @ 9%: Pooling', diff saved to https://phabricator.wikimedia.org/P47230 and previous config saved to /var/cache/conftool/dbconfig/20230419-083753-root.json |
[production] |
| 08:37 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1113:3315 (re)pooling @ 4%: Repooling', diff saved to https://phabricator.wikimedia.org/P47229 and previous config saved to /var/cache/conftool/dbconfig/20230419-083717-root.json |
[production] |
| 08:35 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:01:00 on db2185.codfw.wmnet,db[1115,1215].eqiad.wmnet with reason: Test |
[production] |
| 08:34 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.downtime for 0:01:00 on db2185.codfw.wmnet,db[1115,1215].eqiad.wmnet with reason: Test |
[production] |
| 08:30 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1113:3316 (re)pooling @ 3%: Repooling', diff saved to https://phabricator.wikimedia.org/P47228 and previous config saved to /var/cache/conftool/dbconfig/20230419-083040-root.json |
[production] |
| 08:27 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1110 (re)pooling @ 1%: Repooling', diff saved to https://phabricator.wikimedia.org/P47227 and previous config saved to /var/cache/conftool/dbconfig/20230419-082738-root.json |
[production] |
| 08:24 |
<jnuche@deploy2002> |
Synchronized php: group1 wikis to 1.41.0-wmf.5 refs T330211 (duration: 05m 43s) |
[production] |
| 08:23 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1212 (re)pooling @ 100%: Pooling', diff saved to https://phabricator.wikimedia.org/P47226 and previous config saved to /var/cache/conftool/dbconfig/20230419-082345-root.json |
[production] |
| 08:23 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.reimage for host sretest1002.eqiad.wmnet with OS bookworm |
[production] |
| 08:22 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1219 (re)pooling @ 8%: Pooling', diff saved to https://phabricator.wikimedia.org/P47225 and previous config saved to /var/cache/conftool/dbconfig/20230419-082247-root.json |
[production] |
| 08:22 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1113:3315 (re)pooling @ 3%: Repooling', diff saved to https://phabricator.wikimedia.org/P47224 and previous config saved to /var/cache/conftool/dbconfig/20230419-082213-root.json |
[production] |
| 08:18 |
<jnuche@deploy2002> |
rebuilt and synchronized wikiversions files: group1 wikis to 1.41.0-wmf.5 refs T330211 |
[production] |
| 08:15 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1113:3316 (re)pooling @ 2%: Repooling', diff saved to https://phabricator.wikimedia.org/P47223 and previous config saved to /var/cache/conftool/dbconfig/20230419-081535-root.json |
[production] |
| 08:08 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1212 (re)pooling @ 75%: Pooling', diff saved to https://phabricator.wikimedia.org/P47222 and previous config saved to /var/cache/conftool/dbconfig/20230419-080841-root.json |
[production] |
| 08:07 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1219 (re)pooling @ 7%: Pooling', diff saved to https://phabricator.wikimedia.org/P47221 and previous config saved to /var/cache/conftool/dbconfig/20230419-080742-root.json |
[production] |
| 08:07 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1113:3315 (re)pooling @ 2%: Repooling', diff saved to https://phabricator.wikimedia.org/P47220 and previous config saved to /var/cache/conftool/dbconfig/20230419-080708-root.json |
[production] |
| 08:00 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1113:3316 (re)pooling @ 1%: Repooling', diff saved to https://phabricator.wikimedia.org/P47219 and previous config saved to /var/cache/conftool/dbconfig/20230419-080030-root.json |
[production] |
| 07:53 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1212 (re)pooling @ 50%: Pooling', diff saved to https://phabricator.wikimedia.org/P47218 and previous config saved to /var/cache/conftool/dbconfig/20230419-075336-root.json |
[production] |
| 07:52 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1219 (re)pooling @ 6%: Pooling', diff saved to https://phabricator.wikimedia.org/P47217 and previous config saved to /var/cache/conftool/dbconfig/20230419-075237-root.json |
[production] |
| 07:52 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1113:3315 (re)pooling @ 1%: Repooling', diff saved to https://phabricator.wikimedia.org/P47216 and previous config saved to /var/cache/conftool/dbconfig/20230419-075203-root.json |
[production] |
| 07:38 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1212 (re)pooling @ 25%: Pooling', diff saved to https://phabricator.wikimedia.org/P47215 and previous config saved to /var/cache/conftool/dbconfig/20230419-073831-root.json |
[production] |
| 07:37 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1219 (re)pooling @ 5%: Pooling', diff saved to https://phabricator.wikimedia.org/P47214 and previous config saved to /var/cache/conftool/dbconfig/20230419-073732-root.json |
[production] |
| 07:23 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1212 (re)pooling @ 10%: Pooling', diff saved to https://phabricator.wikimedia.org/P47213 and previous config saved to /var/cache/conftool/dbconfig/20230419-072326-root.json |
[production] |
| 07:22 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1219 (re)pooling @ 4%: Pooling', diff saved to https://phabricator.wikimedia.org/P47212 and previous config saved to /var/cache/conftool/dbconfig/20230419-072228-root.json |
[production] |
| 07:15 |
<XioNoX> |
update TLS cert on pfw - T334676 |
[production] |
| 07:13 |
<kartik@deploy2002> |
Finished scap: Backport for [[gerrit:909607|Enable Content/Section translation on 6 Wikipedias (T327102)]] (duration: 09m 33s) |
[production] |
| 07:10 |
<XioNoX> |
push pfw policies - T334983 |
[production] |
| 07:09 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2178 (T333332)', diff saved to https://phabricator.wikimedia.org/P47211 and previous config saved to /var/cache/conftool/dbconfig/20230419-070920-ladsgroup.json |
[production] |
| 07:08 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1212 (re)pooling @ 5%: Pooling', diff saved to https://phabricator.wikimedia.org/P47210 and previous config saved to /var/cache/conftool/dbconfig/20230419-070822-root.json |
[production] |
| 07:07 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1219 (re)pooling @ 3%: Pooling', diff saved to https://phabricator.wikimedia.org/P47209 and previous config saved to /var/cache/conftool/dbconfig/20230419-070723-root.json |
[production] |
| 07:05 |
<kartik@deploy2002> |
kartik: Backport for [[gerrit:909607|Enable Content/Section translation on 6 Wikipedias (T327102)]] synced to the testservers: mwdebug1001.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet |
[production] |
| 07:03 |
<kartik@deploy2002> |
Started scap: Backport for [[gerrit:909607|Enable Content/Section translation on 6 Wikipedias (T327102)]] |
[production] |
| 06:54 |
<ladsgroup@cumin1001> |
dbctl commit (dc=all): 'Repooling after maintenance db2178', diff saved to https://phabricator.wikimedia.org/P47208 and previous config saved to /var/cache/conftool/dbconfig/20230419-065413-ladsgroup.json |
[production] |