51-100 of 10000 results (112ms)
2026-07-08 ยง
14:50 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. [production]
14:49 <mvernon@cumin2003> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host thanos-be1007.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL [production]
14:47 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. [production]
14:46 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. [production]
14:44 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. [production]
14:43 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. [production]
14:41 <moritzm> uninstalling dhcpcd-base from trixie hosts which still have it installed T414341 [production]
14:40 <sukhe> sudo cumin -b1 -s120 "P{lvs2011*} or P{lvs2012*}" "systemctl restart pybal.service" [production]
14:39 <mvernon@cumin2003> START - Cookbook sre.hosts.provision for host thanos-be1007.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART and with Dell SCP reboot policy GRACEFUL [production]
14:39 <mvernon@cumin2003> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host thanos-be1007.eqiad.wmnet with OS trixie [production]
14:37 <sukhe> restart pybal on lvs2013 to revert back to conf2004 [production]
14:35 <sukhe> restart pybal on lvs2014 to revert back to conf2004 [production]
14:34 <swfrench-wmf> switched codfw, eqsin, ulsfo etcd client SRV records back to codfw - T430909 [production]
14:32 <cwilliams@cumin1003> END (FAIL) - Cookbook sre.mysql.multiinstance_reboot (exit_code=99) for db-test1002.eqiad.wmnet [production]
14:32 <cwilliams@cumin1003> START - Cookbook sre.mysql.multiinstance_reboot for db-test1002.eqiad.wmnet [production]
14:31 <mvernon@cumin2003> START - Cookbook sre.hosts.reimage for host thanos-be1007.eqiad.wmnet with OS trixie [production]
14:31 <jforrester@deploy1003> helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply [production]
14:31 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. [production]
14:31 <jforrester@deploy1003> helmfile [eqiad] START helmfile.d/services/wikifunctions: apply [production]
14:31 <jforrester@deploy1003> helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply [production]
14:30 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. [production]
14:30 <jforrester@deploy1003> helmfile [codfw] START helmfile.d/services/wikifunctions: apply [production]
14:30 <jforrester@deploy1003> helmfile [staging] DONE helmfile.d/services/wikifunctions: apply [production]
14:30 <fceratto@cumin1003> END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2228: Pool test [production]
14:30 <jforrester@deploy1003> helmfile [staging] START helmfile.d/services/wikifunctions: apply [production]
14:29 <swfrench@dns1004> END - running authdns-update [production]
14:29 <moritzm> installing jackson-core security updates [production]
14:27 <swfrench@dns1004> START - running authdns-update [production]
14:22 <jforrester@deploy1003> helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply [production]
14:22 <jforrester@deploy1003> helmfile [eqiad] START helmfile.d/services/wikifunctions: apply [production]
14:22 <atsuko@cumin1003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cirrussearch1119.eqiad.wmnet with OS trixie [production]
14:22 <jforrester@deploy1003> helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply [production]
14:21 <jforrester@deploy1003> helmfile [codfw] START helmfile.d/services/wikifunctions: apply [production]
14:20 <jforrester@deploy1003> helmfile [staging] DONE helmfile.d/services/wikifunctions: apply [production]
14:20 <blake@deploy1003> helmfile [codfw] DONE helmfile.d/admin 'apply'. [production]
14:20 <jforrester@deploy1003> helmfile [staging] START helmfile.d/services/wikifunctions: apply [production]
14:19 <blake@deploy1003> helmfile [codfw] START helmfile.d/admin 'apply'. [production]
14:19 <moritzm> installing librabbitmq security updates [production]
14:19 <cwilliams@cumin1003> END (FAIL) - Cookbook sre.mysql.multiinstance_reboot (exit_code=99) for db-test1002.eqiad.wmnet [production]
14:18 <cwilliams@cumin1003> START - Cookbook sre.mysql.multiinstance_reboot for db-test1002.eqiad.wmnet [production]
14:16 <jforrester@deploy1003> helmfile [eqiad] DONE helmfile.d/services/wikifunctions: apply [production]
14:16 <jforrester@deploy1003> helmfile [eqiad] START helmfile.d/services/wikifunctions: apply [production]
14:16 <jforrester@deploy1003> helmfile [codfw] DONE helmfile.d/services/wikifunctions: apply [production]
14:15 <jforrester@deploy1003> helmfile [codfw] START helmfile.d/services/wikifunctions: apply [production]
14:15 <jforrester@deploy1003> helmfile [staging] DONE helmfile.d/services/wikifunctions: apply [production]
14:14 <fceratto@cumin1003> START - Cookbook sre.mysql.pool pool db2228: Pool test [production]
14:14 <sukhe@cumin1003> END (PASS) - Cookbook sre.dns.roll-restart (exit_code=0) rolling restart_daemons on A:dnsbox and (A:dnsbox) [production]
14:14 <jforrester@deploy1003> helmfile [staging] START helmfile.d/services/wikifunctions: apply [production]
14:13 <cwilliams@cumin1003> END (PASS) - Cookbook sre.mysql.multiinstance_reboot (exit_code=0) for db-test[1002-1003].eqiad.wmnet [production]
14:10 <elukey@cumin1003> END (PASS) - Cookbook sre.hosts.bmc-user-mgmt (exit_code=0) for host sretest[2001,2003-2004,2006,2009-2010].codfw.wmnet,sretest[1005-1006].eqiad.wmnet [production]