651-700 of 10000 results (105ms)
2024-09-13 ยง
10:29 <xSavitar> T374684 Ran mwscript extensions/CentralAuth/maintenance/fixStuckGlobalRename.php --wiki=metawiki --logwiki=metawiki 'Iosonopony' 'L.Sala' [production]
10:29 <xSavitar> T374684 Ran mwscript extensions/CentralAuth/maintenance/fixStuckGlobalRename.php --wiki=commonswiki --logwiki=metawiki 'IloveuFlyTek' 'Theology1937' --ignorestatus [production]
10:25 <xSavitar> T12345 Ran mwscript extensions/CentralAuth/maintenance/fixStuckGlobalRename.php --wiki=commonswiki --logwiki=metawiki 'IloveuFlyTek' 'Theology1937' --ignorestatus [production]
10:18 <jynus@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 3 days, 0:00:00 on db1172.eqiad.wmnet with reason: ongoing schema change [production]
10:18 <jynus@cumin1002> START - Cookbook sre.hosts.downtime for 3 days, 0:00:00 on db1172.eqiad.wmnet with reason: ongoing schema change [production]
09:41 <klausman@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ml-lab1001.eqiad.wmnet with OS bookworm [production]
09:41 <klausman@cumin1002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - klausman@cumin1002" [production]
09:41 <klausman@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - klausman@cumin1002" [production]
09:34 <jayme@cumin1002> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts kafka-main2005.codfw.wmnet [production]
09:34 <jayme@cumin1002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
09:34 <jayme@cumin1002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: kafka-main2005.codfw.wmnet decommissioned, removing all IPs except the asset tag one - jayme@cumin1002" [production]
09:34 <jayme@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: kafka-main2005.codfw.wmnet decommissioned, removing all IPs except the asset tag one - jayme@cumin1002" [production]
09:28 <klausman@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ml-lab1001.eqiad.wmnet with reason: host reimage [production]
09:27 <jayme@cumin1002> START - Cookbook sre.dns.netbox [production]
09:25 <akosiaris@cumin1002> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host mw2305.codfw.wmnet [production]
09:24 <akosiaris@cumin1002> START - Cookbook sre.k8s.pool-depool-node depool for host mw2305.codfw.wmnet [production]
09:24 <klausman@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on ml-lab1001.eqiad.wmnet with reason: host reimage [production]
09:24 <akosiaris@cumin1002> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host mw2304.codfw.wmnet [production]
09:24 <akosiaris@cumin1002> START - Cookbook sre.k8s.pool-depool-node depool for host mw2304.codfw.wmnet [production]
09:24 <akosiaris@cumin1002> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host mw2303.codfw.wmnet [production]
09:23 <akosiaris@cumin1002> START - Cookbook sre.k8s.pool-depool-node depool for host mw2303.codfw.wmnet [production]
09:23 <akosiaris@cumin1002> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host mw2302.codfw.wmnet [production]
09:23 <jayme@deploy1003> helmfile [codfw] DONE helmfile.d/services/rdf-streaming-updater: apply [production]
09:23 <jayme@deploy1003> helmfile [codfw] START helmfile.d/services/rdf-streaming-updater: apply [production]
09:23 <jayme@deploy1003> helmfile [codfw] DONE helmfile.d/services/mw-page-content-change-enrich: apply [production]
09:22 <akosiaris@cumin1002> START - Cookbook sre.k8s.pool-depool-node depool for host mw2302.codfw.wmnet [production]
09:22 <jayme@deploy1003> helmfile [codfw] START helmfile.d/services/mw-page-content-change-enrich: apply [production]
09:22 <akosiaris@cumin1002> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host mw2301.codfw.wmnet [production]
09:22 <jayme@deploy1003> helmfile [codfw] DONE helmfile.d/services/eventstreams: apply [production]
09:22 <akosiaris@cumin1002> START - Cookbook sre.k8s.pool-depool-node depool for host mw2301.codfw.wmnet [production]
09:22 <akosiaris@cumin1002> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host kubernetes2060.codfw.wmnet [production]
09:21 <jayme@deploy1003> helmfile [codfw] START helmfile.d/services/eventstreams: apply [production]
09:21 <jayme@deploy1003> helmfile [codfw] DONE helmfile.d/services/eventgate-main: apply [production]
09:21 <akosiaris@cumin1002> START - Cookbook sre.k8s.pool-depool-node depool for host kubernetes2060.codfw.wmnet [production]
09:21 <akosiaris@cumin1002> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host kubernetes2059.codfw.wmnet [production]
09:21 <jayme@deploy1003> helmfile [codfw] START helmfile.d/services/eventgate-main: apply [production]
09:21 <jayme@deploy1003> helmfile [codfw] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
09:20 <akosiaris@cumin1002> START - Cookbook sre.k8s.pool-depool-node depool for host kubernetes2059.codfw.wmnet [production]
09:20 <jayme@deploy1003> helmfile [codfw] START helmfile.d/services/cirrus-streaming-updater: apply [production]
09:20 <jayme@deploy1003> helmfile [codfw] DONE helmfile.d/services/changeprop: apply [production]
09:20 <jayme@cumin1002> START - Cookbook sre.hosts.decommission for hosts kafka-main2005.codfw.wmnet [production]
09:20 <jayme@deploy1003> helmfile [codfw] START helmfile.d/services/changeprop: apply [production]
09:20 <jayme@deploy1003> helmfile [codfw] DONE helmfile.d/services/changeprop-jobqueue: apply [production]
09:19 <jayme@deploy1003> helmfile [codfw] START helmfile.d/services/changeprop-jobqueue: apply [production]
09:15 <jayme@cumin1002> END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for kafka-main2010.codfw.wmnet [production]
09:15 <jayme@cumin1002> START - Cookbook sre.hosts.remove-downtime for kafka-main2010.codfw.wmnet [production]
09:14 <jayme> restoring leadership for all partitions assigned to broker id 2005 on kafka-main-codfw - T363210 [production]
09:12 <klausman@cumin1002> START - Cookbook sre.hosts.reimage for host ml-lab1001.eqiad.wmnet with OS bookworm [production]
09:09 <klausman@cumin1002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host ml-lab1001.eqiad.wmnet with OS bookworm [production]
09:02 <btullis@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. [production]