101-150 of 10000 results (12ms)
2026-08-24 ยง
17:49 <bd808@deploy1003> helmfile [staging] START helmfile.d/services/shellbox-syntaxhighlight: apply [production]
17:49 <bd808@deploy1003> helmfile [staging] DONE helmfile.d/services/shellbox-media: apply [production]
17:49 <bd808@deploy1003> helmfile [staging] START helmfile.d/services/shellbox-media: apply [production]
17:48 <bd808@deploy1003> helmfile [staging] DONE helmfile.d/services/shellbox-constraints: apply [production]
17:48 <bd808@deploy1003> helmfile [staging] START helmfile.d/services/shellbox-constraints: apply [production]
17:48 <bd808@deploy1003> helmfile [staging] DONE helmfile.d/services/shellbox: apply [production]
17:48 <bd808@deploy1003> helmfile [staging] START helmfile.d/services/shellbox: apply [production]
17:45 <fceratto@cumin1003> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host db2901.codfw.wmnet [production]
17:45 <fceratto@cumin1003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2901.codfw.wmnet with OS trixie [production]
17:45 <bd808@deploy1003> helmfile [staging] DONE helmfile.d/services/shellbox: apply [production]
17:45 <bd808@deploy1003> helmfile [staging] START helmfile.d/services/shellbox: apply [production]
17:43 <bd808@deploy1003> helmfile [staging] DONE helmfile.d/services/shellbox: apply [production]
17:42 <swfrench-wmf> disable-puppet on A:cp for ATS Lua change - T427666 [production]
17:42 <bd808@deploy1003> helmfile [staging] START helmfile.d/services/shellbox: apply [production]
17:38 <cmooney@cumin1003> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
17:38 <cmooney@cumin1003> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update entry for telxius circuit IP in magru - cmooney@cumin1003" [production]
17:38 <cmooney@cumin1003> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: update entry for telxius circuit IP in magru - cmooney@cumin1003" [production]
17:32 <bking@cumin2003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host krb1003.eqiad.wmnet with OS bookworm [production]
17:32 <bking@cumin2003> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - bking@cumin2003" [production]
17:31 <cmooney@cumin1003> START - Cookbook sre.dns.netbox [production]
17:29 <bking@cumin2003> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - bking@cumin2003" [production]
17:27 <ejegg> disabled payment attempt queue consumer [fundraising]
17:26 <jayme@deploy1003> helmfile [ml-serve-codfw] DONE helmfile.d/admin 'sync'. [production]
17:25 <jayme@deploy1003> helmfile [ml-serve-codfw] START helmfile.d/admin 'sync'. [production]
17:25 <jayme@deploy1003> helmfile [ml-serve-eqiad] DONE helmfile.d/admin 'sync'. [production]
17:23 <jayme@deploy1003> helmfile [ml-serve-eqiad] START helmfile.d/admin 'sync'. [production]
17:22 <swfrench@deploy1003> Finished scap sync-world: Noop deployment to validate pretrain httpbb check configuration - T428972 (duration: 02m 30s) [production]
17:21 <jayme@deploy1003> helmfile [eqiad] DONE helmfile.d/admin 'sync'. [production]
17:20 <swfrench@deploy1003> Started scap sync-world: Noop deployment to validate pretrain httpbb check configuration - T428972 [production]
17:19 <damilare> payments-wiki upgraded from c56b3ad7 to c3ed1fa0 [fundraising]
17:19 <jayme@deploy1003> helmfile [eqiad] START helmfile.d/admin 'sync'. [production]
17:19 <jayme@deploy1003> helmfile [codfw] DONE helmfile.d/admin 'sync'. [production]
17:18 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.drain on host 'cloudvirt1078.eqiad.wmnet' (T435537) [admin]
17:18 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary (exit_code=0) on eqiad1, with recreate False, for hosts list: ['cloudvirt1079'] [cloudvirt-canary]
17:18 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary on eqiad1, with recreate False, for hosts list: ['cloudvirt1079'] [cloudvirt-canary]
17:17 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (exit_code=0) (T435537) [admin]
17:17 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.unset_maintenance (T435537) [admin]
17:17 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary (exit_code=99) on eqiad1, with recreate False, for hosts list: ['cloudvirt1040', 'cloudvirt1044', 'cloudvirt1042', 'cloudvirt1045', 'cloudvirt1043', 'cloudvirt1046', 'cloudvirt1041', 'cloudvirt1047', 'cloudvirt1048', 'cloudvirt1049', 'cloudvirt1052', 'cloudvirt1053', 'cloudvirt1050', 'cloudvirt1051', 'cloudvirt1057 [cloudvirt-canary]
17:17 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary on eqiad1, with recreate False, for hosts list: ['cloudvirt1040', 'cloudvirt1044', 'cloudvirt1042', 'cloudvirt1045', 'cloudvirt1043', 'cloudvirt1046', 'cloudvirt1041', 'cloudvirt1047', 'cloudvirt1048', 'cloudvirt1049', 'cloudvirt1052', 'cloudvirt1053', 'cloudvirt1050', 'cloudvirt1051', 'cloudvirt1057', 'cloudvirt1056', [cloudvirt-canary]
17:16 <jayme@deploy1003> helmfile [codfw] START helmfile.d/admin 'sync'. [production]
17:16 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary (exit_code=99) on eqiad1, with recreate False, for hosts list: ['cloudvirt1079'] [cloudvirt-canary]
17:16 <fceratto@cumin1003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2901.codfw.wmnet with reason: host reimage [production]
17:16 <jayme@deploy1003> helmfile [codfw] DONE helmfile.d/admin 'apply'. [production]
17:16 <andrew@cloudcumin1001> START - Cookbook wmcs.openstack.cloudvirt.lib.ensure_canary on eqiad1, with recreate False, for hosts list: ['cloudvirt1079'] [cloudvirt-canary]
17:16 <jayme@deploy1003> helmfile [codfw] START helmfile.d/admin 'apply'. [production]
17:12 <bking@cumin2003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on krb1003.eqiad.wmnet with reason: host reimage [production]
17:10 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.openstack.cloudvirt.drain (exit_code=0) on host 'cloudvirt1079.eqiad.wmnet' (T435537) [admin]
17:08 <fceratto@cumin1003> START - Cookbook sre.hosts.downtime for 2:00:00 on db2901.codfw.wmnet with reason: host reimage [production]
17:07 <bking@cumin2003> START - Cookbook sre.hosts.downtime for 2:00:00 on krb1003.eqiad.wmnet with reason: host reimage [production]
17:05 <taavi@cloudcumin1001> END (PASS) - Cookbook wmcs.vps.delete_project (exit_code=0) for project wikifunctions in eqiad1 [wikifunctions]