6851-6900 of 10000 results (108ms)
2023-11-27 §
06:33 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on 6 hosts with reason: Switch [production]
06:16 <kart_> Update cxserver to 2023-11-20-052250-production (T341458, T349118) [production]
06:12 <kartik@deploy2002> helmfile [codfw] DONE helmfile.d/services/cxserver: apply [production]
06:12 <kartik@deploy2002> helmfile [codfw] START helmfile.d/services/cxserver: apply [production]
06:06 <kartik@deploy2002> helmfile [eqiad] DONE helmfile.d/services/cxserver: apply [production]
06:05 <kartik@deploy2002> helmfile [eqiad] START helmfile.d/services/cxserver: apply [production]
05:43 <kartik@deploy2002> helmfile [staging] DONE helmfile.d/services/cxserver: apply [production]
05:43 <kartik@deploy2002> helmfile [staging] START helmfile.d/services/cxserver: apply [production]
2023-11-24 §
15:01 <jayme@deploy2002> helmfile [staging] DONE helmfile.d/services/api-gateway: apply [production]
15:00 <jayme@deploy2002> helmfile [staging] START helmfile.d/services/api-gateway: apply [production]
14:58 <jayme@deploy2002> helmfile [staging] DONE helmfile.d/services/rest-gateway: apply [production]
14:58 <jayme@deploy2002> helmfile [staging] START helmfile.d/services/rest-gateway: apply [production]
13:41 <jayme@deploy2002> helmfile [codfw] DONE helmfile.d/services/api-gateway: apply [production]
13:41 <jayme@deploy2002> helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply [production]
13:41 <jayme@deploy2002> helmfile [codfw] START helmfile.d/services/api-gateway: apply [production]
13:40 <jayme@deploy2002> helmfile [codfw] START helmfile.d/services/rest-gateway: apply [production]
13:33 <arnaudb@cumin1001> dbctl commit (dc=all): 'db2191 (re)pooling @ 100%: Post clone repooling', diff saved to https://phabricator.wikimedia.org/P53911 and previous config saved to /var/cache/conftool/dbconfig/20231124-133300-arnaudb.json [production]
13:24 <jayme@deploy2002> helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply [production]
13:24 <jayme@deploy2002> helmfile [eqiad] DONE helmfile.d/services/api-gateway: apply [production]
13:24 <jayme@deploy2002> helmfile [eqiad] START helmfile.d/services/rest-gateway: apply [production]
13:23 <jayme@deploy2002> helmfile [eqiad] START helmfile.d/services/api-gateway: apply [production]
13:19 <brouberol@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. [production]
13:19 <jayme@deploy2002> helmfile [staging] DONE helmfile.d/services/api-gateway: apply [production]
13:18 <brouberol@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. [production]
13:18 <jayme@deploy2002> helmfile [staging] START helmfile.d/services/api-gateway: apply [production]
13:18 <jayme@deploy2002> helmfile [staging] DONE helmfile.d/services/rest-gateway: apply [production]
13:17 <arnaudb@cumin1001> dbctl commit (dc=all): 'db2191 (re)pooling @ 90%: Post clone repooling', diff saved to https://phabricator.wikimedia.org/P53910 and previous config saved to /var/cache/conftool/dbconfig/20231124-131755-arnaudb.json [production]
13:17 <jayme@deploy2002> helmfile [staging] START helmfile.d/services/rest-gateway: apply [production]
13:02 <arnaudb@cumin1001> dbctl commit (dc=all): 'db2191 (re)pooling @ 80%: Post clone repooling', diff saved to https://phabricator.wikimedia.org/P53909 and previous config saved to /var/cache/conftool/dbconfig/20231124-130250-arnaudb.json [production]
12:47 <arnaudb@cumin1001> dbctl commit (dc=all): 'db2191 (re)pooling @ 70%: Post clone repooling', diff saved to https://phabricator.wikimedia.org/P53908 and previous config saved to /var/cache/conftool/dbconfig/20231124-124745-arnaudb.json [production]
12:32 <arnaudb@cumin1001> dbctl commit (dc=all): 'db2191 (re)pooling @ 60%: Post clone repooling', diff saved to https://phabricator.wikimedia.org/P53907 and previous config saved to /var/cache/conftool/dbconfig/20231124-123240-arnaudb.json [production]
12:17 <arnaudb@cumin1001> dbctl commit (dc=all): 'db2191 (re)pooling @ 50%: Post clone repooling', diff saved to https://phabricator.wikimedia.org/P53906 and previous config saved to /var/cache/conftool/dbconfig/20231124-121735-arnaudb.json [production]
12:02 <arnaudb@cumin1001> dbctl commit (dc=all): 'db2191 (re)pooling @ 40%: Post clone repooling', diff saved to https://phabricator.wikimedia.org/P53905 and previous config saved to /var/cache/conftool/dbconfig/20231124-120230-arnaudb.json [production]
11:47 <arnaudb@cumin1001> dbctl commit (dc=all): 'db2191 (re)pooling @ 30%: Post clone repooling', diff saved to https://phabricator.wikimedia.org/P53904 and previous config saved to /var/cache/conftool/dbconfig/20231124-114725-arnaudb.json [production]
11:32 <arnaudb@cumin1001> dbctl commit (dc=all): 'db2191 (re)pooling @ 20%: Post clone repooling', diff saved to https://phabricator.wikimedia.org/P53903 and previous config saved to /var/cache/conftool/dbconfig/20231124-113220-arnaudb.json [production]
11:17 <arnaudb@cumin1001> dbctl commit (dc=all): 'db2191 (re)pooling @ 10%: Post clone repooling', diff saved to https://phabricator.wikimedia.org/P53902 and previous config saved to /var/cache/conftool/dbconfig/20231124-111715-arnaudb.json [production]
11:11 <arnaudb@cumin1001> dbctl commit (dc=all): 'set es2032 back as es1 master for T344589', diff saved to https://phabricator.wikimedia.org/P53901 and previous config saved to /var/cache/conftool/dbconfig/20231124-111109-arnaudb.json [production]
11:09 <arnaudb@cumin1001> dbctl commit (dc=all): 'es2032 (re)pooling @ 100%: Post warmup repooling', diff saved to https://phabricator.wikimedia.org/P53900 and previous config saved to /var/cache/conftool/dbconfig/20231124-110948-arnaudb.json [production]
11:02 <hnowlan@deploy2002> helmfile [codfw] DONE helmfile.d/services/changeprop-jobqueue: apply [production]
11:02 <hnowlan@deploy2002> helmfile [codfw] START helmfile.d/services/changeprop-jobqueue: apply [production]
11:00 <jayme@deploy2002> helmfile [staging] DONE helmfile.d/services/rest-gateway: apply [production]
10:55 <hnowlan@deploy2002> helmfile [eqiad] DONE helmfile.d/services/changeprop-jobqueue: apply [production]
10:54 <arnaudb@cumin1001> dbctl commit (dc=all): 'es2032 (re)pooling @ 90%: Post warmup repooling', diff saved to https://phabricator.wikimedia.org/P53899 and previous config saved to /var/cache/conftool/dbconfig/20231124-105443-arnaudb.json [production]
10:54 <hnowlan@deploy2002> helmfile [eqiad] START helmfile.d/services/changeprop-jobqueue: apply [production]
10:53 <hnowlan@deploy2002> helmfile [staging] DONE helmfile.d/services/changeprop-jobqueue: apply [production]
10:53 <hnowlan@deploy2002> helmfile [staging] START helmfile.d/services/changeprop-jobqueue: apply [production]
10:47 <arnaudb@cumin1001> dbctl commit (dc=all): 'db2195 (re)pooling @ 100%: Post warmup repooling', diff saved to https://phabricator.wikimedia.org/P53898 and previous config saved to /var/cache/conftool/dbconfig/20231124-104733-arnaudb.json [production]
10:46 <jayme@deploy2002> helmfile [staging] START helmfile.d/services/rest-gateway: apply [production]
10:46 <arnaudb@cumin1001> dbctl commit (dc=all): 'db2131 (re)pooling @ 100%: Post warmup repooling', diff saved to https://phabricator.wikimedia.org/P53897 and previous config saved to /var/cache/conftool/dbconfig/20231124-104635-arnaudb.json [production]
10:40 <arnaudb@cumin1001> dbctl commit (dc=all): 'db2190 (re)pooling @ 100%: Post warmup repooling', diff saved to https://phabricator.wikimedia.org/P53896 and previous config saved to /var/cache/conftool/dbconfig/20231124-104023-arnaudb.json [production]