|
2024-09-19
ยง
|
| 18:01 |
<aokoth@cumin1002> |
END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on vrts1001.eqiad.wmnet with reason: Migration |
[production] |
| 18:01 |
<aokoth@cumin1002> |
START - Cookbook sre.hosts.downtime for 7 days, 0:00:00 on vrts1001.eqiad.wmnet with reason: Migration |
[production] |
| 17:53 |
<swfrench@cumin2002> |
conftool action : set/pooled=true; selector: dnsdisc=mw-web-ro,name=codfw [reason: Reverting pre-switchover capacity validation - T371273] |
[production] |
| 17:51 |
<brett@cumin2002> |
conftool action : set/pooled=yes; selector: name=cp5024.eqsin.wmnet |
[production] |
| 17:49 |
<swfrench@cumin2002> |
conftool action : set/pooled=true; selector: dnsdisc=mw-api-ext-ro,name=codfw [reason: Reverting pre-switchover capacity validation - T371273] |
[production] |
| 17:47 |
<dancy@deploy1003> |
Installation of scap version "4.104.0" completed for 211 hosts |
[production] |
| 17:46 |
<swfrench@cumin2002> |
conftool action : set/pooled=true; selector: dnsdisc=mw-api-int-ro,name=codfw [reason: Reverting pre-switchover capacity validation - T371273] |
[production] |
| 17:43 |
<dancy@deploy1003> |
Installing scap version "4.104.0" for 211 hosts |
[production] |
| 17:38 |
<brett@cumin2002> |
conftool action : set/pooled=no; selector: name=cp5024.eqsin.wmnet |
[production] |
| 17:26 |
<Dreamy_Jazz> |
Finished running script for T375203 on `group0` |
[production] |
| 17:21 |
<mforns@deploy1003> |
helmfile [staging] DONE helmfile.d/services/commons-impact-analytics: apply |
[production] |
| 17:21 |
<mforns@deploy1003> |
helmfile [staging] START helmfile.d/services/commons-impact-analytics: apply |
[production] |
| 17:17 |
<swfrench@cumin2002> |
conftool action : set/pooled=false; selector: dnsdisc=mw-web-ro,name=codfw [reason: Pre-switchover capacity validation - T371273] |
[production] |
| 17:11 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'es2040 (re)pooling @ 100%: T373105', diff saved to https://phabricator.wikimedia.org/P69376 and previous config saved to /var/cache/conftool/dbconfig/20240919-171122-arnaudb.json |
[production] |
| 17:11 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2220 (re)pooling @ 100%: T373105', diff saved to https://phabricator.wikimedia.org/P69375 and previous config saved to /var/cache/conftool/dbconfig/20240919-171117-arnaudb.json |
[production] |
| 17:11 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2219 (re)pooling @ 100%: T373105', diff saved to https://phabricator.wikimedia.org/P69374 and previous config saved to /var/cache/conftool/dbconfig/20240919-171112-arnaudb.json |
[production] |
| 17:11 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2195 (re)pooling @ 100%: T373105', diff saved to https://phabricator.wikimedia.org/P69373 and previous config saved to /var/cache/conftool/dbconfig/20240919-171107-arnaudb.json |
[production] |
| 17:11 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2182 (re)pooling @ 100%: T373105', diff saved to https://phabricator.wikimedia.org/P69372 and previous config saved to /var/cache/conftool/dbconfig/20240919-171101-arnaudb.json |
[production] |
| 17:10 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2181 (re)pooling @ 100%: T373105', diff saved to https://phabricator.wikimedia.org/P69371 and previous config saved to /var/cache/conftool/dbconfig/20240919-171057-arnaudb.json |
[production] |
| 17:10 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2174 (re)pooling @ 100%: T373105', diff saved to https://phabricator.wikimedia.org/P69370 and previous config saved to /var/cache/conftool/dbconfig/20240919-171052-arnaudb.json |
[production] |
| 17:10 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2173 (re)pooling @ 100%: T373105', diff saved to https://phabricator.wikimedia.org/P69369 and previous config saved to /var/cache/conftool/dbconfig/20240919-171047-arnaudb.json |
[production] |
| 17:10 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2152 (re)pooling @ 100%: T373105', diff saved to https://phabricator.wikimedia.org/P69368 and previous config saved to /var/cache/conftool/dbconfig/20240919-171042-arnaudb.json |
[production] |
| 17:10 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2131 (re)pooling @ 100%: T373105', diff saved to https://phabricator.wikimedia.org/P69367 and previous config saved to /var/cache/conftool/dbconfig/20240919-171037-arnaudb.json |
[production] |
| 17:08 |
<swfrench@cumin2002> |
conftool action : set/pooled=false; selector: dnsdisc=mw-api-ext-ro,name=codfw [reason: Pre-switchover capacity validation - T371273] |
[production] |
| 17:02 |
<swfrench@cumin2002> |
conftool action : set/pooled=false; selector: dnsdisc=mw-api-int-ro,name=codfw [reason: Pre-switchover capacity validation - T371273] |
[production] |
| 16:56 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'es2040 (re)pooling @ 75%: T373105', diff saved to https://phabricator.wikimedia.org/P69366 and previous config saved to /var/cache/conftool/dbconfig/20240919-165617-arnaudb.json |
[production] |
| 16:56 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2220 (re)pooling @ 75%: T373105', diff saved to https://phabricator.wikimedia.org/P69365 and previous config saved to /var/cache/conftool/dbconfig/20240919-165611-arnaudb.json |
[production] |
| 16:56 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2219 (re)pooling @ 75%: T373105', diff saved to https://phabricator.wikimedia.org/P69364 and previous config saved to /var/cache/conftool/dbconfig/20240919-165606-arnaudb.json |
[production] |
| 16:56 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2195 (re)pooling @ 75%: T373105', diff saved to https://phabricator.wikimedia.org/P69363 and previous config saved to /var/cache/conftool/dbconfig/20240919-165602-arnaudb.json |
[production] |
| 16:55 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2182 (re)pooling @ 75%: T373105', diff saved to https://phabricator.wikimedia.org/P69362 and previous config saved to /var/cache/conftool/dbconfig/20240919-165556-arnaudb.json |
[production] |
| 16:55 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2181 (re)pooling @ 75%: T373105', diff saved to https://phabricator.wikimedia.org/P69361 and previous config saved to /var/cache/conftool/dbconfig/20240919-165551-arnaudb.json |
[production] |
| 16:55 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2174 (re)pooling @ 75%: T373105', diff saved to https://phabricator.wikimedia.org/P69360 and previous config saved to /var/cache/conftool/dbconfig/20240919-165546-arnaudb.json |
[production] |
| 16:55 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2173 (re)pooling @ 75%: T373105', diff saved to https://phabricator.wikimedia.org/P69359 and previous config saved to /var/cache/conftool/dbconfig/20240919-165542-arnaudb.json |
[production] |
| 16:55 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2152 (re)pooling @ 75%: T373105', diff saved to https://phabricator.wikimedia.org/P69358 and previous config saved to /var/cache/conftool/dbconfig/20240919-165537-arnaudb.json |
[production] |
| 16:55 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2131 (re)pooling @ 75%: T373105', diff saved to https://phabricator.wikimedia.org/P69357 and previous config saved to /var/cache/conftool/dbconfig/20240919-165531-arnaudb.json |
[production] |
| 16:52 |
<sukhe> |
ulsfo was depooled between 15:55 and 16:12 for sre.dns.admin test, current state is pooled |
[production] |
| 16:50 |
<swfrench-wmf> |
scaling up mw-web in eqiad for pre-switchover testing - T371273 |
[production] |
| 16:50 |
<swfrench@deploy1003> |
helmfile [eqiad] DONE helmfile.d/services/mw-web: apply |
[production] |
| 16:50 |
<swfrench@deploy1003> |
helmfile [eqiad] START helmfile.d/services/mw-web: apply |
[production] |
| 16:48 |
<swfrench-wmf> |
scaling up mw-api-ext in eqiad for pre-switchover testing - T371273 |
[production] |
| 16:48 |
<vgutierrez> |
updated to purged 0.24 in codfw, ulsfo and eqsin - T334078 |
[production] |
| 16:47 |
<swfrench@deploy1003> |
helmfile [eqiad] DONE helmfile.d/services/mw-api-ext: apply |
[production] |
| 16:47 |
<swfrench@deploy1003> |
helmfile [eqiad] START helmfile.d/services/mw-api-ext: apply |
[production] |
| 16:44 |
<vgutierrez> |
uploaded purged 0.24 to apt.wm.o (bullseye-wikimedia) |
[production] |
| 16:41 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'es2040 (re)pooling @ 50%: T373105', diff saved to https://phabricator.wikimedia.org/P69356 and previous config saved to /var/cache/conftool/dbconfig/20240919-164111-arnaudb.json |
[production] |
| 16:41 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2220 (re)pooling @ 50%: T373105', diff saved to https://phabricator.wikimedia.org/P69355 and previous config saved to /var/cache/conftool/dbconfig/20240919-164106-arnaudb.json |
[production] |
| 16:41 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2219 (re)pooling @ 50%: T373105', diff saved to https://phabricator.wikimedia.org/P69354 and previous config saved to /var/cache/conftool/dbconfig/20240919-164101-arnaudb.json |
[production] |
| 16:40 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2195 (re)pooling @ 50%: T373105', diff saved to https://phabricator.wikimedia.org/P69353 and previous config saved to /var/cache/conftool/dbconfig/20240919-164056-arnaudb.json |
[production] |
| 16:40 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2182 (re)pooling @ 50%: T373105', diff saved to https://phabricator.wikimedia.org/P69352 and previous config saved to /var/cache/conftool/dbconfig/20240919-164051-arnaudb.json |
[production] |
| 16:40 |
<arnaudb@cumin1002> |
dbctl commit (dc=all): 'db2181 (re)pooling @ 50%: T373105', diff saved to https://phabricator.wikimedia.org/P69351 and previous config saved to /var/cache/conftool/dbconfig/20240919-164046-arnaudb.json |
[production] |