1-50 of 10000 results (98ms)
2026-04-14 §
06:40 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host puppetserver1002.eqiad.wmnet [production]
06:40 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host puppetserver2002.codfw.wmnet [production]
06:33 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host puppetserver2002.codfw.wmnet [production]
06:30 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-wmde: apply [production]
06:30 <marostegui@cumin1003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1180.eqiad.wmnet with OS trixie [production]
06:30 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-wmde: apply [production]
06:27 <jmm@dns1004> END - running authdns-update [production]
06:25 <jmm@dns1004> START - running authdns-update [production]
06:22 <jmm@dns1004> END - running authdns-update [production]
06:20 <jmm@dns1004> START - running authdns-update [production]
06:13 <marostegui@cumin1003> START - Cookbook sre.mysql.pool pool db2217: repool after reimage to trixie [production]
06:11 <marostegui@cumin1003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2217.codfw.wmnet with OS trixie [production]
06:08 <marostegui@cumin1003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1180.eqiad.wmnet with reason: host reimage [production]
06:02 <marostegui@cumin1003> START - Cookbook sre.hosts.downtime for 2:00:00 on db1180.eqiad.wmnet with reason: host reimage [production]
06:02 <jmm@dns1004> END - running authdns-update [production]
06:00 <jmm@dns1004> START - running authdns-update [production]
05:54 <marostegui@cumin1003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2217.codfw.wmnet with reason: host reimage [production]
05:49 <marostegui@cumin1003> START - Cookbook sre.hosts.downtime for 2:00:00 on db2217.codfw.wmnet with reason: host reimage [production]
05:47 <marostegui@cumin1003> START - Cookbook sre.hosts.reimage for host db1180.eqiad.wmnet with OS trixie [production]
05:46 <marostegui@cumin1003> END (FAIL) - Cookbook sre.mysql.depool (exit_code=99) depool db1180: Upgrade package [production]
05:35 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1180.eqiad.wmnet with reason: Reimage to Trixie [production]
05:35 <marostegui@cumin1003> START - Cookbook sre.mysql.depool depool db1180: Upgrade package [production]
05:30 <marostegui@cumin1003> START - Cookbook sre.hosts.reimage for host db2217.codfw.wmnet with OS trixie [production]
05:28 <marostegui@cumin1003> END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2217: Reimage [production]
05:27 <marostegui@cumin1003> START - Cookbook sre.mysql.depool depool db2217: Reimage [production]
05:27 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2217.codfw.wmnet with reason: Reimage to Trixie [production]
04:02 <mwpresync@deploy1003> Pruned MediaWiki: 1.46.0-wmf.21 (duration: 02m 34s) [production]
03:39 <mwpresync@deploy1003> Finished scap sync-world: testwikis to 1.46.0-wmf.24 refs T420482 (duration: 35m 44s) [production]
03:03 <mwpresync@deploy1003> Started scap sync-world: testwikis to 1.46.0-wmf.24 refs T420482 [production]
00:57 <ladsgroup@cumin1003> END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1227: Work done [production]
00:51 <eevans@cumin1003> END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for aqs1025.eqiad.wmnet [production]
00:51 <eevans@cumin1003> START - Cookbook sre.hosts.remove-downtime for aqs1025.eqiad.wmnet [production]
00:39 <ladsgroup@cumin1003> END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2208: Work done [production]
00:09 <ladsgroup@cumin1003> START - Cookbook sre.mysql.pool pool db1227: Work done [production]
00:08 <eevans@deploy1003> helmfile [staging] DONE helmfile.d/services/linked-artifacts: sync [production]
00:05 <eevans@deploy1003> helmfile [staging] START helmfile.d/services/linked-artifacts: sync [production]
2026-04-13 §
23:54 <eevans@deploy1003> helmfile [staging] DONE helmfile.d/services/data-gateway: sync [production]
23:53 <eevans@deploy1003> helmfile [staging] START helmfile.d/services/data-gateway: sync [production]
23:49 <eevans@deploy1003> helmfile [staging] DONE helmfile.d/services/linked-artifacts: sync [production]
23:49 <eevans@deploy1003> helmfile [staging] START helmfile.d/services/linked-artifacts: sync [production]
23:49 <ladsgroup@cumin1003> START - Cookbook sre.mysql.pool pool db2208: Work done [production]
23:05 <brett@cumin2002> END (PASS) - Cookbook sre.cdn.roll-upgrade-ats (exit_code=0) Rolling upgrade of ATS on A:cp-text_codfw - 9.2.13 Upgrade () [production]
23:02 <brett@cumin2002> END (PASS) - Cookbook sre.cdn.roll-upgrade-ats (exit_code=0) Rolling upgrade of ATS on A:cp-upload_codfw - 9.2.13 Upgrade () [production]
22:29 <brett@puppetserver1001> conftool action : set/pooled=yes; selector: name=cp5024.* [production]
22:29 <brett@puppetserver1001> conftool action : set/pooled=yes; selector: name=cp5023.* [production]
22:26 <brett@cumin2002> START - Cookbook sre.cdn.roll-upgrade-ats Rolling upgrade of ATS on A:cp-upload_codfw - 9.2.13 Upgrade () [production]
22:26 <brett@cumin2002> START - Cookbook sre.cdn.roll-upgrade-ats Rolling upgrade of ATS on A:cp-text_codfw - 9.2.13 Upgrade () [production]
22:23 <brett@cumin2002> END (PASS) - Cookbook sre.cdn.roll-upgrade-ats (exit_code=0) Rolling upgrade of ATS on P{cp[5023-5024].eqsin.wmnet} and A:cp - 9.2.13 Upgrade () [production]
22:15 <sbassett@deploy1003> Finished scap sync-world: Deployed security fix for T422085 (duration: 30m 14s) [production]
22:08 <brett@cumin2002> START - Cookbook sre.cdn.roll-upgrade-ats Rolling upgrade of ATS on P{cp[5023-5024].eqsin.wmnet} and A:cp - 9.2.13 Upgrade () [production]