51-100 of 10000 results (26ms)
2026-09-29 ยง
07:44 <mlitn@deploy1003> mlitn: Continuing with deployment [production]
07:43 <mlitn@deploy1003> mlitn: Backport for [[gerrit:1345862|carousel: keep the hide button attached to the carousel (T439507)]], [[gerrit:1345863|carousel: keep the hide button attached to the carousel (T439507)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
07:39 <mlitn@deploy1003> Started scap sync-world: Backport for [[gerrit:1345862|carousel: keep the hide button attached to the carousel (T439507)]], [[gerrit:1345863|carousel: keep the hide button attached to the carousel (T439507)]] [production]
07:35 <kharlan@deploy1003> Finished scap sync-world: Backport for [[gerrit:1345854|CheckUser: Enable ClientHintsInEditRequest on group0 (T401808)]] (duration: 12m 41s) [production]
07:33 <jmm@cumin2003> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host sretest2004.codfw.wmnet [production]
07:30 <kharlan@deploy1003> kharlan: Continuing with deployment [production]
07:27 <jmm@cumin2003> START - Cookbook sre.hosts.reboot-single for host sretest2004.codfw.wmnet [production]
07:27 <kharlan@deploy1003> kharlan: Backport for [[gerrit:1345854|CheckUser: Enable ClientHintsInEditRequest on group0 (T401808)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
07:26 <kart_> Updated cxserver to 2026-09-25-114231-production [production]
07:24 <kartik@deploy1003> helmfile [eqiad] DONE helmfile.d/services/cxserver: apply [production]
07:24 <kartik@deploy1003> helmfile [eqiad] START helmfile.d/services/cxserver: apply [production]
07:23 <kartik@deploy1003> helmfile [codfw] DONE helmfile.d/services/cxserver: apply [production]
07:22 <kharlan@deploy1003> Started scap sync-world: Backport for [[gerrit:1345854|CheckUser: Enable ClientHintsInEditRequest on group0 (T401808)]] [production]
07:22 <kartik@deploy1003> helmfile [codfw] START helmfile.d/services/cxserver: apply [production]
07:18 <jmm@cumin2003> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host idp-test2005.wikimedia.org [production]
07:18 <mlitn@deploy1003> Finished scap sync-world: Backport for [[gerrit:1345626|Skip expensive work if known to be useless (T439182)]], [[gerrit:1345632|Cache extracted carousel data (T439182)]], [[gerrit:1345627|Make carousel images are visible in the margins (T439422)]], [[gerrit:1345633|carousel: fix carousel image preload observer root (T439422)]] (duration: 13m 39s) [production]
07:14 <jmm@cumin2003> START - Cookbook sre.hosts.reboot-single for host idp-test2005.wikimedia.org [production]
07:11 <mlitn@deploy1003> mlitn: Continuing with deployment [production]
07:09 <mlitn@deploy1003> mlitn: Backport for [[gerrit:1345626|Skip expensive work if known to be useless (T439182)]], [[gerrit:1345632|Cache extracted carousel data (T439182)]], [[gerrit:1345627|Make carousel images are visible in the margins (T439422)]], [[gerrit:1345633|carousel: fix carousel image preload observer root (T439422)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now [production]
07:04 <mlitn@deploy1003> Started scap sync-world: Backport for [[gerrit:1345626|Skip expensive work if known to be useless (T439182)]], [[gerrit:1345632|Cache extracted carousel data (T439182)]], [[gerrit:1345627|Make carousel images are visible in the margins (T439422)]], [[gerrit:1345633|carousel: fix carousel image preload observer root (T439422)]] [production]
07:02 <kartik@deploy1003> helmfile [staging] DONE helmfile.d/services/cxserver: apply [production]
07:01 <kartik@deploy1003> helmfile [staging] START helmfile.d/services/cxserver: apply [production]
06:57 <jelto@deploy1003> helmfile [aux-k8s-eqiad] DONE helmfile.d/aux-k8s-services/etherpad: apply [production]
06:56 <jelto@deploy1003> helmfile [aux-k8s-eqiad] START helmfile.d/aux-k8s-services/etherpad: apply [production]
05:54 <ryankemper@cumin2003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wdqs1035.eqiad.wmnet with OS bookworm [production]
05:50 <ryankemper@cumin2003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wdqs1034.eqiad.wmnet with OS bookworm [production]
05:38 <ryankemper@cumin2003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wdqs1035.eqiad.wmnet with reason: host reimage [production]
05:34 <ryankemper@cumin2003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wdqs1034.eqiad.wmnet with reason: host reimage [production]
05:32 <ryankemper@cumin2003> START - Cookbook sre.hosts.downtime for 2:00:00 on wdqs1035.eqiad.wmnet with reason: host reimage [production]
05:31 <ryankemper@cumin2003> START - Cookbook sre.hosts.downtime for 2:00:00 on wdqs1034.eqiad.wmnet with reason: host reimage [production]
05:21 <ryankemper@cumin2003> END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host wdqs1035 [production]
05:21 <ryankemper@cumin2003> START - Cookbook sre.hosts.move-vlan for host wdqs1035 [production]
05:21 <ryankemper@cumin2003> START - Cookbook sre.hosts.reimage for host wdqs1035.eqiad.wmnet with OS bookworm [production]
05:19 <ryankemper@cumin2003> END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host wdqs1034 [production]
05:19 <ryankemper@cumin2003> START - Cookbook sre.hosts.move-vlan for host wdqs1034 [production]
05:19 <ryankemper@cumin2003> START - Cookbook sre.hosts.reimage for host wdqs1034.eqiad.wmnet with OS bookworm [production]
05:03 <ryankemper@cumin2003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wdqs1033.eqiad.wmnet with OS bookworm [production]
04:51 <fabfur> manually restarting dump_cloud_ip_range on puppetserver1001 after failure [production]
04:47 <ryankemper@cumin2003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wdqs1033.eqiad.wmnet with reason: host reimage [production]
04:43 <ryankemper@cumin2003> START - Cookbook sre.hosts.downtime for 2:00:00 on wdqs1033.eqiad.wmnet with reason: host reimage [production]
04:32 <ryankemper@cumin2003> END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host wdqs1033 [production]
04:32 <ryankemper@cumin2003> START - Cookbook sre.hosts.move-vlan for host wdqs1033 [production]
04:31 <ryankemper@cumin2003> START - Cookbook sre.hosts.reimage for host wdqs1033.eqiad.wmnet with OS bookworm [production]
04:01 <mwpresync@deploy1003> Pruned MediaWiki: 1.47.0-wmf.19 (duration: 01m 00s) [production]
03:39 <mwpresync@deploy1003> Finished scap sync-world: testwikis to 1.47.0-wmf.22 refs T438218 (duration: 36m 03s) [production]
03:03 <mwpresync@deploy1003> Started scap sync-world: testwikis to 1.47.0-wmf.22 refs T438218 [production]
02:08 <mwpresync@deploy1003> Finished scap build-images: Publishing wmf/next image (duration: 07m 28s) [production]
02:00 <mwpresync@deploy1003> Started scap build-images: Publishing wmf/next image [production]
01:49 <brett@cumin1004> END (PASS) - Cookbook sre.cdn.roll-upgrade-varnish (exit_code=0) rolling upgrade of Varnish on A:cp-text_esams - 7.1.1-2~bpo13+wmf3 () [production]
00:22 <dzahn@cumin1004> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on zuul1005.eqiad.wmnet with reason: host reimage [production]