1751-1800 of 10000 results (54ms)
2023-11-27 §
06:40 <marostegui> Failover m2 from db1119 to db1195 - T351863 [production]
06:33 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on 6 hosts with reason: Switch [production]
06:33 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on 6 hosts with reason: Switch [production]
06:16 <kart_> Update cxserver to 2023-11-20-052250-production (T341458, T349118) [production]
06:12 <kartik@deploy2002> helmfile [codfw] DONE helmfile.d/services/cxserver: apply [production]
06:12 <kartik@deploy2002> helmfile [codfw] START helmfile.d/services/cxserver: apply [production]
06:06 <kartik@deploy2002> helmfile [eqiad] DONE helmfile.d/services/cxserver: apply [production]
06:05 <kartik@deploy2002> helmfile [eqiad] START helmfile.d/services/cxserver: apply [production]
05:43 <kartik@deploy2002> helmfile [staging] DONE helmfile.d/services/cxserver: apply [production]
05:43 <kartik@deploy2002> helmfile [staging] START helmfile.d/services/cxserver: apply [production]
2023-11-26 §
14:50 <valhallasw> kubectl delete pod grrrrit-567d49c997-m5qzl (pod "grrrrit-567d49c997-m5qzl" deleted); kubectl get pods shows new pod grrrrit-567d49c997-xdvm2 (running 34s) [tools.wikibugs]
14:48 <valhallasw> restarting wikibugs to deploy T350778 / r975426 [tools.wikibugs]
2023-11-25 §
20:05 <taavi> the grid-deprecation tool no longer lists tools that ran jobs in the past (but are currently not running anything) and that are disabled [tools.grid-deprecation]
12:38 <wm-bot> <lucaswerkmeister> deployed 171fc2ea54 (l10n updates: br) [tools.lexeme-forms]
2023-11-24 §
15:01 <jayme@deploy2002> helmfile [staging] DONE helmfile.d/services/api-gateway: apply [production]
15:00 <jayme@deploy2002> helmfile [staging] START helmfile.d/services/api-gateway: apply [production]
14:58 <jayme@deploy2002> helmfile [staging] DONE helmfile.d/services/rest-gateway: apply [production]
14:58 <jayme@deploy2002> helmfile [staging] START helmfile.d/services/rest-gateway: apply [production]
14:51 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.restart_openstack (exit_code=0) [admin]
14:50 <taavi@cloudcumin1001> START - Cookbook wmcs.openstack.restart_openstack [admin]
13:41 <jayme@deploy2002> helmfile [codfw] DONE helmfile.d/services/api-gateway: apply [production]
13:41 <jayme@deploy2002> helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply [production]
13:41 <jayme@deploy2002> helmfile [codfw] START helmfile.d/services/api-gateway: apply [production]
13:40 <jayme@deploy2002> helmfile [codfw] START helmfile.d/services/rest-gateway: apply [production]
13:33 <arnaudb@cumin1001> dbctl commit (dc=all): 'db2191 (re)pooling @ 100%: Post clone repooling', diff saved to https://phabricator.wikimedia.org/P53911 and previous config saved to /var/cache/conftool/dbconfig/20231124-133300-arnaudb.json [production]
13:24 <jayme@deploy2002> helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply [production]
13:24 <jayme@deploy2002> helmfile [eqiad] DONE helmfile.d/services/api-gateway: apply [production]
13:24 <jayme@deploy2002> helmfile [eqiad] START helmfile.d/services/rest-gateway: apply [production]
13:23 <jayme@deploy2002> helmfile [eqiad] START helmfile.d/services/api-gateway: apply [production]
13:19 <brouberol@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. [production]
13:19 <jayme@deploy2002> helmfile [staging] DONE helmfile.d/services/api-gateway: apply [production]
13:18 <brouberol@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. [production]
13:18 <jayme@deploy2002> helmfile [staging] START helmfile.d/services/api-gateway: apply [production]
13:18 <jayme@deploy2002> helmfile [staging] DONE helmfile.d/services/rest-gateway: apply [production]
13:17 <arnaudb@cumin1001> dbctl commit (dc=all): 'db2191 (re)pooling @ 90%: Post clone repooling', diff saved to https://phabricator.wikimedia.org/P53910 and previous config saved to /var/cache/conftool/dbconfig/20231124-131755-arnaudb.json [production]
13:17 <jayme@deploy2002> helmfile [staging] START helmfile.d/services/rest-gateway: apply [production]
13:02 <arnaudb@cumin1001> dbctl commit (dc=all): 'db2191 (re)pooling @ 80%: Post clone repooling', diff saved to https://phabricator.wikimedia.org/P53909 and previous config saved to /var/cache/conftool/dbconfig/20231124-130250-arnaudb.json [production]
12:47 <arnaudb@cumin1001> dbctl commit (dc=all): 'db2191 (re)pooling @ 70%: Post clone repooling', diff saved to https://phabricator.wikimedia.org/P53908 and previous config saved to /var/cache/conftool/dbconfig/20231124-124745-arnaudb.json [production]
12:34 <joal> Rerun webrequest refine text for 2023-11-23T17 [analytics]
12:32 <arnaudb@cumin1001> dbctl commit (dc=all): 'db2191 (re)pooling @ 60%: Post clone repooling', diff saved to https://phabricator.wikimedia.org/P53907 and previous config saved to /var/cache/conftool/dbconfig/20231124-123240-arnaudb.json [production]
12:17 <arnaudb@cumin1001> dbctl commit (dc=all): 'db2191 (re)pooling @ 50%: Post clone repooling', diff saved to https://phabricator.wikimedia.org/P53906 and previous config saved to /var/cache/conftool/dbconfig/20231124-121735-arnaudb.json [production]
12:02 <arnaudb@cumin1001> dbctl commit (dc=all): 'db2191 (re)pooling @ 40%: Post clone repooling', diff saved to https://phabricator.wikimedia.org/P53905 and previous config saved to /var/cache/conftool/dbconfig/20231124-120230-arnaudb.json [production]
12:01 <taavi@cloudcumin1001> END (FAIL) - Cookbook wmcs.openstack.restart_openstack (exit_code=99) [admin]
12:01 <taavi@cloudcumin1001> START - Cookbook wmcs.openstack.restart_openstack [admin]
12:01 <taavi@cloudcumin1001> END (FAIL) - Cookbook wmcs.openstack.restart_openstack (exit_code=99) [admin]
12:00 <taavi@cloudcumin1001> START - Cookbook wmcs.openstack.restart_openstack [admin]
12:00 <taavi@cloudcumin1001> END (FAIL) - Cookbook wmcs.openstack.restart_openstack (exit_code=99) [admin]
12:00 <taavi@cloudcumin1001> START - Cookbook wmcs.openstack.restart_openstack [admin]
11:53 <taavi@cloudcumin1001> END (FAIL) - Cookbook wmcs.openstack.restart_openstack (exit_code=99) [admin]
11:53 <taavi@cloudcumin1001> START - Cookbook wmcs.openstack.restart_openstack [admin]