|
2022-01-28
ยง
|
| 15:50 |
<hnowlan@cumin1001> |
END (PASS) - Cookbook sre.postgresql.postgres-init (exit_code=0) |
[production] |
| 15:47 |
<hnowlan@cumin1001> |
START - Cookbook sre.hosts.reimage for host restbase1022.eqiad.wmnet with OS buster |
[production] |
| 15:47 |
<hnowlan@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=restbase1021.eqiad.wmnet |
[production] |
| 15:47 |
<hnowlan@cumin1001> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host restbase1021.eqiad.wmnet with OS buster |
[production] |
| 15:41 |
<vgutierrez> |
pool cp4031 using envoy as TLS termination layer - T271421 |
[production] |
| 15:14 |
<Amir1> |
start of cleaning lint errors caused by content model changes (T298343) |
[production] |
| 14:50 |
<hnowlan@cumin1001> |
START - Cookbook sre.hosts.reimage for host restbase1021.eqiad.wmnet with OS buster |
[production] |
| 14:50 |
<hnowlan@puppetmaster1001> |
conftool action : set/pooled=yes; selector: name=restbase1020.eqiad.wmnet |
[production] |
| 14:47 |
<vgutierrez> |
update varnish to version 6.0.10-1wm1 on cp4036 - T300264 |
[production] |
| 14:47 |
<Amir1> |
optimizing dewiki.flaggedtemplates in db2113 |
[production] |
| 14:27 |
<vgutierrez> |
update varnish to version 6.0.10-1wm1 on cp4034 - T300264 |
[production] |
| 13:20 |
<hnowlan@cumin1001> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host restbase1020.eqiad.wmnet with OS buster |
[production] |
| 13:01 |
<moritzm> |
installing uriparser security updates |
[production] |
| 12:32 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1168 (re)pooling @ 100%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P19562 and previous config saved to /var/cache/conftool/dbconfig/20220128-123210-root.json |
[production] |
| 12:30 |
<moritzm> |
installing libseccomp bugfix updates from bullseye point release |
[production] |
| 12:28 |
<hnowlan@cumin1001> |
START - Cookbook sre.hosts.reimage for host restbase1020.eqiad.wmnet with OS buster |
[production] |
| 12:20 |
<vgutierrez> |
upload varnish 6.0.10-1wm1 to apt.wm.o (buster component/varnish6) - T300264 |
[production] |
| 12:17 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1168 (re)pooling @ 75%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P19561 and previous config saved to /var/cache/conftool/dbconfig/20220128-121706-root.json |
[production] |
| 12:02 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1168 (re)pooling @ 60%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P19560 and previous config saved to /var/cache/conftool/dbconfig/20220128-120201-root.json |
[production] |
| 11:46 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1168 (re)pooling @ 50%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P19559 and previous config saved to /var/cache/conftool/dbconfig/20220128-114658-root.json |
[production] |
| 11:35 |
<hnowlan@cumin1001> |
START - Cookbook sre.postgresql.postgres-init |
[production] |
| 11:31 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1168 (re)pooling @ 40%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P19558 and previous config saved to /var/cache/conftool/dbconfig/20220128-113154-root.json |
[production] |
| 11:16 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1168 (re)pooling @ 25%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P19557 and previous config saved to /var/cache/conftool/dbconfig/20220128-111650-root.json |
[production] |
| 11:01 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1168 (re)pooling @ 20%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P19556 and previous config saved to /var/cache/conftool/dbconfig/20220128-110147-root.json |
[production] |
| 10:46 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1168 (re)pooling @ 10%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P19555 and previous config saved to /var/cache/conftool/dbconfig/20220128-104643-root.json |
[production] |
| 10:31 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1168 (re)pooling @ 5%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P19554 and previous config saved to /var/cache/conftool/dbconfig/20220128-103140-root.json |
[production] |
| 10:29 |
<mdipietro@cumin1001> |
END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudcontrol2001-dev.wikimedia.org with OS bullseye |
[production] |
| 10:25 |
<moritzm> |
draining ganeti1010 for eventual reimage |
[production] |
| 10:23 |
<jmm@cumin2002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4 days, 0:00:00 on ganeti1019.eqiad.wmnet with reason: Remove from Ganeti cluster for reimage |
[production] |
| 10:23 |
<jmm@cumin2002> |
START - Cookbook sre.hosts.downtime for 4 days, 0:00:00 on ganeti1019.eqiad.wmnet with reason: Remove from Ganeti cluster for reimage |
[production] |
| 10:18 |
<marostegui@cumin1001> |
END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1168.eqiad.wmnet with OS bullseye |
[production] |
| 10:16 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1168 (re)pooling @ 1%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P19553 and previous config saved to /var/cache/conftool/dbconfig/20220128-101636-root.json |
[production] |
| 09:50 |
<marostegui@cumin1001> |
START - Cookbook sre.hosts.reimage for host db1168.eqiad.wmnet with OS bullseye |
[production] |
| 09:46 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'Depool db1168 T299479', diff saved to https://phabricator.wikimedia.org/P19552 and previous config saved to /var/cache/conftool/dbconfig/20220128-094636-marostegui.json |
[production] |
| 09:46 |
<moritzm> |
installing brltty bugfix updates from bullseye point release |
[production] |
| 09:44 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1096:3316 (re)pooling @ 100%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P19551 and previous config saved to /var/cache/conftool/dbconfig/20220128-094430-root.json |
[production] |
| 09:44 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1096:3315 (re)pooling @ 100%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P19550 and previous config saved to /var/cache/conftool/dbconfig/20220128-094422-root.json |
[production] |
| 09:29 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1096:3316 (re)pooling @ 75%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P19549 and previous config saved to /var/cache/conftool/dbconfig/20220128-092927-root.json |
[production] |
| 09:29 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1096:3315 (re)pooling @ 75%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P19548 and previous config saved to /var/cache/conftool/dbconfig/20220128-092918-root.json |
[production] |
| 09:25 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] DONE helmfile.d/services/mwdebug: sync on pinkunicorn |
[production] |
| 09:24 |
<mwdebug-deploy@deploy1002> |
helmfile [codfw] START helmfile.d/services/mwdebug: apply on pinkunicorn |
[production] |
| 09:24 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] DONE helmfile.d/services/mwdebug: sync on pinkunicorn |
[production] |
| 09:23 |
<hashar@deploy1002> |
Synchronized wmf-config/CommonSettings.php: GrowthExperiments: Disable mobile quality gate - T298122 T300336 (duration: 00m 50s) |
[production] |
| 09:22 |
<mwdebug-deploy@deploy1002> |
helmfile [eqiad] START helmfile.d/services/mwdebug: apply on pinkunicorn |
[production] |
| 09:17 |
<godog> |
pool prometheus2005 and depool prometheus2003 - T296199 |
[production] |
| 09:14 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1096:3316 (re)pooling @ 60%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P19547 and previous config saved to /var/cache/conftool/dbconfig/20220128-091423-root.json |
[production] |
| 09:14 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1096:3315 (re)pooling @ 60%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P19546 and previous config saved to /var/cache/conftool/dbconfig/20220128-091415-root.json |
[production] |
| 08:59 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1096:3316 (re)pooling @ 50%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P19545 and previous config saved to /var/cache/conftool/dbconfig/20220128-085919-root.json |
[production] |
| 08:59 |
<marostegui@cumin1001> |
dbctl commit (dc=all): 'db1096:3315 (re)pooling @ 50%: repooling after reimage', diff saved to https://phabricator.wikimedia.org/P19544 and previous config saved to /var/cache/conftool/dbconfig/20220128-085911-root.json |
[production] |
| 08:44 |
<jayme@cumin1001> |
END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host kubemaster1001.eqiad.wmnet |
[production] |