9651-9700 of 10000 results (132ms)
2024-09-19 ยง
16:56 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2219 (re)pooling @ 75%: T373105', diff saved to https://phabricator.wikimedia.org/P69364 and previous config saved to /var/cache/conftool/dbconfig/20240919-165606-arnaudb.json [production]
16:56 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2195 (re)pooling @ 75%: T373105', diff saved to https://phabricator.wikimedia.org/P69363 and previous config saved to /var/cache/conftool/dbconfig/20240919-165602-arnaudb.json [production]
16:55 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2182 (re)pooling @ 75%: T373105', diff saved to https://phabricator.wikimedia.org/P69362 and previous config saved to /var/cache/conftool/dbconfig/20240919-165556-arnaudb.json [production]
16:55 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2181 (re)pooling @ 75%: T373105', diff saved to https://phabricator.wikimedia.org/P69361 and previous config saved to /var/cache/conftool/dbconfig/20240919-165551-arnaudb.json [production]
16:55 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2174 (re)pooling @ 75%: T373105', diff saved to https://phabricator.wikimedia.org/P69360 and previous config saved to /var/cache/conftool/dbconfig/20240919-165546-arnaudb.json [production]
16:55 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2173 (re)pooling @ 75%: T373105', diff saved to https://phabricator.wikimedia.org/P69359 and previous config saved to /var/cache/conftool/dbconfig/20240919-165542-arnaudb.json [production]
16:55 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2152 (re)pooling @ 75%: T373105', diff saved to https://phabricator.wikimedia.org/P69358 and previous config saved to /var/cache/conftool/dbconfig/20240919-165537-arnaudb.json [production]
16:55 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2131 (re)pooling @ 75%: T373105', diff saved to https://phabricator.wikimedia.org/P69357 and previous config saved to /var/cache/conftool/dbconfig/20240919-165531-arnaudb.json [production]
16:52 <sukhe> ulsfo was depooled between 15:55 and 16:12 for sre.dns.admin test, current state is pooled [production]
16:50 <swfrench-wmf> scaling up mw-web in eqiad for pre-switchover testing - T371273 [production]
16:50 <swfrench@deploy1003> helmfile [eqiad] DONE helmfile.d/services/mw-web: apply [production]
16:50 <swfrench@deploy1003> helmfile [eqiad] START helmfile.d/services/mw-web: apply [production]
16:48 <swfrench-wmf> scaling up mw-api-ext in eqiad for pre-switchover testing - T371273 [production]
16:48 <vgutierrez> updated to purged 0.24 in codfw, ulsfo and eqsin - T334078 [production]
16:47 <swfrench@deploy1003> helmfile [eqiad] DONE helmfile.d/services/mw-api-ext: apply [production]
16:47 <swfrench@deploy1003> helmfile [eqiad] START helmfile.d/services/mw-api-ext: apply [production]
16:44 <vgutierrez> uploaded purged 0.24 to apt.wm.o (bullseye-wikimedia) [production]
16:41 <arnaudb@cumin1002> dbctl commit (dc=all): 'es2040 (re)pooling @ 50%: T373105', diff saved to https://phabricator.wikimedia.org/P69356 and previous config saved to /var/cache/conftool/dbconfig/20240919-164111-arnaudb.json [production]
16:41 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2220 (re)pooling @ 50%: T373105', diff saved to https://phabricator.wikimedia.org/P69355 and previous config saved to /var/cache/conftool/dbconfig/20240919-164106-arnaudb.json [production]
16:41 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2219 (re)pooling @ 50%: T373105', diff saved to https://phabricator.wikimedia.org/P69354 and previous config saved to /var/cache/conftool/dbconfig/20240919-164101-arnaudb.json [production]
16:40 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2195 (re)pooling @ 50%: T373105', diff saved to https://phabricator.wikimedia.org/P69353 and previous config saved to /var/cache/conftool/dbconfig/20240919-164056-arnaudb.json [production]
16:40 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2182 (re)pooling @ 50%: T373105', diff saved to https://phabricator.wikimedia.org/P69352 and previous config saved to /var/cache/conftool/dbconfig/20240919-164051-arnaudb.json [production]
16:40 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2181 (re)pooling @ 50%: T373105', diff saved to https://phabricator.wikimedia.org/P69351 and previous config saved to /var/cache/conftool/dbconfig/20240919-164046-arnaudb.json [production]
16:40 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2174 (re)pooling @ 50%: T373105', diff saved to https://phabricator.wikimedia.org/P69350 and previous config saved to /var/cache/conftool/dbconfig/20240919-164041-arnaudb.json [production]
16:40 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2173 (re)pooling @ 50%: T373105', diff saved to https://phabricator.wikimedia.org/P69349 and previous config saved to /var/cache/conftool/dbconfig/20240919-164036-arnaudb.json [production]
16:40 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2152 (re)pooling @ 50%: T373105', diff saved to https://phabricator.wikimedia.org/P69348 and previous config saved to /var/cache/conftool/dbconfig/20240919-164031-arnaudb.json [production]
16:40 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2131 (re)pooling @ 50%: T373105', diff saved to https://phabricator.wikimedia.org/P69347 and previous config saved to /var/cache/conftool/dbconfig/20240919-164026-arnaudb.json [production]
16:38 <swfrench@cumin2002> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host parse2020.codfw.wmnet [production]
16:38 <swfrench@cumin2002> START - Cookbook sre.k8s.pool-depool-node pool for host parse2020.codfw.wmnet [production]
16:38 <topranks> disable LAG interface from asw-d-codfw to ssw1-dX-codfw [production]
16:38 <swfrench@cumin2002> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host parse2019.codfw.wmnet [production]
16:38 <swfrench@cumin2002> START - Cookbook sre.k8s.pool-depool-node pool for host parse2019.codfw.wmnet [production]
16:37 <swfrench@cumin2002> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host parse2018.codfw.wmnet [production]
16:37 <swfrench@cumin2002> START - Cookbook sre.k8s.pool-depool-node pool for host parse2018.codfw.wmnet [production]
16:37 <swfrench@cumin2002> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host mw2282.codfw.wmnet [production]
16:37 <swfrench@cumin2002> START - Cookbook sre.k8s.pool-depool-node pool for host mw2282.codfw.wmnet [production]
16:37 <swfrench@cumin2002> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host kubernetes2053.codfw.wmnet [production]
16:37 <swfrench@cumin2002> START - Cookbook sre.k8s.pool-depool-node pool for host kubernetes2053.codfw.wmnet [production]
16:37 <swfrench@cumin2002> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host kubernetes2052.codfw.wmnet [production]
16:36 <swfrench@cumin2002> START - Cookbook sre.k8s.pool-depool-node pool for host kubernetes2052.codfw.wmnet [production]
16:34 <effie> restarting confd on all servers on: codfw, ulsof, eqsin - T373105 [production]
16:33 <vgutierrez> testing purged 0.24 in cp2037 - T334078 [production]
16:27 <Emperor> restart swift-object-replicator on thanos-be2002 [production]
16:27 <sukhe@puppetmaster1001> conftool action : set/pooled=yes; selector: name=(cp2041|cp2042).codfw.wmnet [reason: T373105 is done] [production]
16:26 <sukhe@cumin1002> END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for cp2042.codfw.wmnet [production]
16:26 <sukhe@cumin1002> START - Cookbook sre.hosts.remove-downtime for cp2042.codfw.wmnet [production]
16:26 <sukhe@cumin1002> END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for cp2041.codfw.wmnet [production]
16:26 <sukhe@cumin1002> START - Cookbook sre.hosts.remove-downtime for cp2041.codfw.wmnet [production]
16:26 <arnaudb@cumin1002> dbctl commit (dc=all): 'es2040 (re)pooling @ 25%: T373105', diff saved to https://phabricator.wikimedia.org/P69346 and previous config saved to /var/cache/conftool/dbconfig/20240919-162606-arnaudb.json [production]
16:26 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2220 (re)pooling @ 25%: T373105', diff saved to https://phabricator.wikimedia.org/P69345 and previous config saved to /var/cache/conftool/dbconfig/20240919-162601-arnaudb.json [production]