101-150 of 10000 results (112ms)
2026-05-22 §
05:17 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on dbproxy2005.codfw.wmnet with reason: Reboot [production]
05:11 <marostegui@cumin1003> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts pc1012.eqiad.wmnet [production]
05:11 <marostegui@cumin1003> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
05:11 <marostegui@cumin1003> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: pc1012.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003" [production]
05:06 <marostegui@cumin1003> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: pc1012.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - marostegui@cumin1003" [production]
05:03 <marostegui@cumin1003> START - Cookbook sre.dns.netbox [production]
04:56 <marostegui@cumin1003> START - Cookbook sre.hosts.decommission for hosts pc1012.eqiad.wmnet [production]
2026-05-21 §
23:43 <dreamyjazz@deploy1003> Finished scap sync-world: Backport for [[gerrit:1290954|Drop not defined config $wgAllowRawHtmlCopyrightMessages]], [[gerrit:1290957|Drop $wgGraphShowInToolbar definition as unused]], [[gerrit:1290958|Drop wgMFSearchGenerator definition as unused]], [[gerrit:1290960|Drop unused wpReportIncidentLocalLinks]] (duration: 06m 42s) [production]
23:38 <dreamyjazz@deploy1003> dreamyjazz: Continuing with deployment [production]
23:38 <dreamyjazz@deploy1003> dreamyjazz: Backport for [[gerrit:1290954|Drop not defined config $wgAllowRawHtmlCopyrightMessages]], [[gerrit:1290957|Drop $wgGraphShowInToolbar definition as unused]], [[gerrit:1290958|Drop wgMFSearchGenerator definition as unused]], [[gerrit:1290960|Drop unused wpReportIncidentLocalLinks]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified [production]
23:36 <dreamyjazz@deploy1003> Started scap sync-world: Backport for [[gerrit:1290954|Drop not defined config $wgAllowRawHtmlCopyrightMessages]], [[gerrit:1290957|Drop $wgGraphShowInToolbar definition as unused]], [[gerrit:1290958|Drop wgMFSearchGenerator definition as unused]], [[gerrit:1290960|Drop unused wpReportIncidentLocalLinks]] [production]
22:26 <dzahn@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host zuul2002.codfw.wmnet with OS trixie [production]
22:08 <dzahn@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on zuul2002.codfw.wmnet with reason: host reimage [production]
22:03 <dzahn@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on zuul2002.codfw.wmnet with reason: host reimage [production]
22:02 <bking@cumin2002> END (ERROR) - Cookbook sre.elasticsearch.rolling-operation (exit_code=97) Operation.REBOOT (3 nodes at a time) for ElasticSearch cluster search_codfw: T426560 - bking@cumin2002 [production]
21:49 <cjming@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen: apply [production]
21:49 <cjming@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen: apply [production]
21:44 <dzahn@cumin2002> START - Cookbook sre.hosts.reimage for host zuul2002.codfw.wmnet with OS trixie [production]
21:25 <cjming@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply [production]
21:25 <cjming@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply [production]
21:20 <cjming@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply [production]
21:19 <cjming@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply [production]
20:26 <cjming@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply [production]
20:16 <cjming@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply [production]
19:22 <eevans@cumin1003> END (PASS) - Cookbook sre.cassandra.roll-reboot (exit_code=0) rolling reboot on A:restbase [production]
19:09 <cjming@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply [production]
18:59 <cjming@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply [production]
18:53 <papaul> rebooting msw1-codfw [production]
18:50 <cjming@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/test-kitchen-next: apply [production]
18:39 <cjming@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/test-kitchen-next: apply [production]
17:54 <swfrench@deploy1003> helmfile [eqiad] DONE helmfile.d/services/shellbox-video: apply [production]
17:53 <swfrench@deploy1003> helmfile [eqiad] START helmfile.d/services/shellbox-video: apply [production]
17:53 <swfrench@deploy1003> helmfile [eqiad] DONE helmfile.d/services/shellbox-timeline: apply [production]
17:52 <cgoubert@deploy1003> helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply [production]
17:52 <swfrench@deploy1003> helmfile [eqiad] START helmfile.d/services/shellbox-timeline: apply [production]
17:52 <cgoubert@deploy1003> helmfile [codfw] START helmfile.d/services/rest-gateway: apply [production]
17:52 <swfrench@deploy1003> helmfile [eqiad] DONE helmfile.d/services/shellbox-syntaxhighlight: apply [production]
17:51 <swfrench@deploy1003> helmfile [eqiad] START helmfile.d/services/shellbox-syntaxhighlight: apply [production]
17:51 <swfrench@deploy1003> helmfile [eqiad] DONE helmfile.d/services/shellbox-media: apply [production]
17:51 <swfrench@deploy1003> helmfile [eqiad] START helmfile.d/services/shellbox-media: apply [production]
17:50 <swfrench@deploy1003> helmfile [eqiad] DONE helmfile.d/services/shellbox-constraints: apply [production]
17:49 <swfrench@deploy1003> helmfile [eqiad] START helmfile.d/services/shellbox-constraints: apply [production]
17:49 <swfrench@deploy1003> helmfile [eqiad] DONE helmfile.d/services/shellbox: apply [production]
17:48 <swfrench@deploy1003> helmfile [eqiad] START helmfile.d/services/shellbox: apply [production]
17:46 <cgoubert@deploy1003> helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply [production]
17:46 <cgoubert@deploy1003> helmfile [eqiad] START helmfile.d/services/rest-gateway: apply [production]
17:43 <swfrench@deploy1003> helmfile [codfw] DONE helmfile.d/services/shellbox-video: apply [production]
17:43 <cgoubert@deploy1003> helmfile [staging] DONE helmfile.d/services/rest-gateway: apply [production]
17:43 <cgoubert@deploy1003> helmfile [staging] START helmfile.d/services/rest-gateway: apply [production]
17:42 <swfrench@deploy1003> helmfile [codfw] START helmfile.d/services/shellbox-video: apply [production]