2351-2400 of 10000 results (91ms)
2024-02-22 ยง
15:13 <akosiaris@cumin1002> conftool action : set/weight=1; selector: service=parsoid-php,name=kubernetes.* [production]
15:12 <akosiaris@cumin1002> conftool action : set/weight=110; selector: service=parsoid-php,name=(pars.*|mw.*) [production]
15:12 <akosiaris> Bump weight of old parsoid hosts from 10 to 110. This is a noop right now but will makes calculations later spelled out in T357392 possible. [production]
14:55 <akosiaris@deploy2002> helmfile [codfw] DONE helmfile.d/services/mw-parsoid: apply [production]
14:55 <akosiaris@deploy2002> helmfile [codfw] START helmfile.d/services/mw-parsoid: apply [production]
14:55 <akosiaris@deploy2002> helmfile [eqiad] DONE helmfile.d/services/mw-parsoid: apply [production]
14:55 <akosiaris@deploy2002> helmfile [eqiad] START helmfile.d/services/mw-parsoid: apply [production]
14:51 <marostegui@cumin1002> dbctl commit (dc=all): 'db2149 (re)pooling @ 1%: After recloning', diff saved to https://phabricator.wikimedia.org/P57726 and previous config saved to /var/cache/conftool/dbconfig/20240222-145120-root.json [production]
14:46 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1203', diff saved to https://phabricator.wikimedia.org/P57725 and previous config saved to /var/cache/conftool/dbconfig/20240222-144648-arnaudb.json [production]
14:45 <cgoubert@deploy2002> Finished scap: Backport for [[gerrit:1004135|Enable $wgLocalHTTPProxy on group1 wikis (T298265)]] (duration: 17m 46s) [production]
14:44 <jclark@cumin1002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host an-redacteddb1001.eqiad.wmnet with OS bullseye [production]
14:44 <jclark@cumin1002> START - Cookbook sre.hosts.reimage for host an-redacteddb1001.eqiad.wmnet with OS bullseye [production]
14:37 <cgoubert@deploy2002> cgoubert: Continuing with sync [production]
14:31 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1203 (T357189)', diff saved to https://phabricator.wikimedia.org/P57724 and previous config saved to /var/cache/conftool/dbconfig/20240222-143141-arnaudb.json [production]
14:29 <cgoubert@deploy2002> cgoubert: Backport for [[gerrit:1004135|Enable $wgLocalHTTPProxy on group1 wikis (T298265)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
14:29 <arnaudb@cumin1002> dbctl commit (dc=all): 'Depooling db1203 (T357189)', diff saved to https://phabricator.wikimedia.org/P57723 and previous config saved to /var/cache/conftool/dbconfig/20240222-142921-arnaudb.json [production]
14:29 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1203.eqiad.wmnet with reason: Maintenance [production]
14:29 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1203.eqiad.wmnet with reason: Maintenance [production]
14:29 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1193 (T357189)', diff saved to https://phabricator.wikimedia.org/P57722 and previous config saved to /var/cache/conftool/dbconfig/20240222-142859-arnaudb.json [production]
14:28 <cgoubert@deploy2002> Started scap: Backport for [[gerrit:1004135|Enable $wgLocalHTTPProxy on group1 wikis (T298265)]] [production]
14:15 <marostegui@cumin1002> dbctl commit (dc=all): 'es1028 (re)pooling @ 100%: After migration to 10.6', diff saved to https://phabricator.wikimedia.org/P57721 and previous config saved to /var/cache/conftool/dbconfig/20240222-141508-root.json [production]
14:13 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1193', diff saved to https://phabricator.wikimedia.org/P57720 and previous config saved to /var/cache/conftool/dbconfig/20240222-141353-arnaudb.json [production]
14:03 <btullis@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/superset-next: apply [production]
14:00 <marostegui@cumin1002> dbctl commit (dc=all): 'es1028 (re)pooling @ 75%: After migration to 10.6', diff saved to https://phabricator.wikimedia.org/P57719 and previous config saved to /var/cache/conftool/dbconfig/20240222-140003-root.json [production]
13:58 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1193', diff saved to https://phabricator.wikimedia.org/P57718 and previous config saved to /var/cache/conftool/dbconfig/20240222-135846-arnaudb.json [production]
13:53 <btullis@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/superset-next: apply [production]
13:52 <btullis@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/superset-next: apply [production]
13:52 <btullis@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/superset-next: apply [production]
13:52 <btullis@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/superset-next: apply [production]
13:52 <btullis@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/superset-next: apply [production]
13:52 <btullis@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/superset-next: apply [production]
13:52 <btullis@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/superset-next: apply [production]
13:51 <btullis@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/superset-next: apply [production]
13:51 <btullis@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/superset-next: apply [production]
13:51 <btullis@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/superset-next: apply [production]
13:51 <btullis@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/superset-next: apply [production]
13:46 <aborrero@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cloudvirt1034.eqiad.wmnet with OS bookworm [production]
13:46 <btullis@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/superset-next: apply [production]
13:45 <btullis@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/superset-next: apply [production]
13:45 <btullis@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/superset-next: apply [production]
13:45 <marostegui@cumin1002> dbctl commit (dc=all): 'es1028 (re)pooling @ 50%: After migration to 10.6', diff saved to https://phabricator.wikimedia.org/P57717 and previous config saved to /var/cache/conftool/dbconfig/20240222-134458-root.json [production]
13:44 <btullis@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/superset-next: apply [production]
13:43 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1193 (T357189)', diff saved to https://phabricator.wikimedia.org/P57716 and previous config saved to /var/cache/conftool/dbconfig/20240222-134340-arnaudb.json [production]
13:41 <arnaudb@cumin1002> dbctl commit (dc=all): 'Depooling db1193 (T357189)', diff saved to https://phabricator.wikimedia.org/P57715 and previous config saved to /var/cache/conftool/dbconfig/20240222-134120-arnaudb.json [production]
13:41 <btullis@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/superset-next: apply [production]
13:41 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1193.eqiad.wmnet with reason: Maintenance [production]
13:41 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1193.eqiad.wmnet with reason: Maintenance [production]
13:41 <arnaudb@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db1192 (T357189)', diff saved to https://phabricator.wikimedia.org/P57714 and previous config saved to /var/cache/conftool/dbconfig/20240222-134059-arnaudb.json [production]
13:40 <aborrero@cumin1002> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host cloudvirt1034 [production]
13:40 <aborrero@cumin1002> START - Cookbook sre.network.configure-switch-interfaces for host cloudvirt1034 [production]