1-50 of 10000 results (119ms)
2026-08-18 ยง
18:54 <reedy@deploy1003> Started scap sync-world: Backport for [[gerrit:1324751|InitialiseSettings: Enable 2FA enforcement on more private wikis (T428103)]], [[gerrit:1326875|Add banner notifying of upcoming 2FA enforcement (T420792)]] [production]
18:50 <jasmine@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-ctrl2006.codfw.wmnet with OS trixie [production]
18:44 <aklapper@deploy1003> rebuilt and synchronized wikiversions files: group0 to 1.47.0-wmf.16 refs T430835 [production]
18:34 <jasmine@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-ctrl2006.codfw.wmnet with reason: host reimage [production]
18:31 <jasmine@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-ctrl2006.codfw.wmnet with reason: host reimage [production]
18:22 <aklapper@deploy1003> Finished scap sync-world: Backport for [[gerrit:1326852|CategoryTree: Allow null $html in the CategoryViewerGenerateLink handler (T435161)]], [[gerrit:1326853|CategoryViewer: Allow null $html in the CategoryViewerGenerateLink hook (T435161)]], [[gerrit:1326865|Flow: Allow null $html in the CategoryViewerGenerateLink handler (T435161)]] (duration: 09m 57s) [production]
18:18 <aklapper@deploy1003> jforrester, aklapper: Continuing with deployment [production]
18:17 <jasmine@cumin2002> START - Cookbook sre.hosts.reimage for host wikikube-ctrl2006.codfw.wmnet with OS trixie [production]
18:14 <aklapper@deploy1003> jforrester, aklapper: Backport for [[gerrit:1326852|CategoryTree: Allow null $html in the CategoryViewerGenerateLink handler (T435161)]], [[gerrit:1326853|CategoryViewer: Allow null $html in the CategoryViewerGenerateLink hook (T435161)]], [[gerrit:1326865|Flow: Allow null $html in the CategoryViewerGenerateLink handler (T435161)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki [production]
18:13 <btullis@cumin1003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1156.eqiad.wmnet with OS bookworm [production]
18:12 <ladsgroup@deploy1003> helmfile [eqiad] DONE helmfile.d/services/mw-experimental: apply [production]
18:12 <aklapper@deploy1003> Started scap sync-world: Backport for [[gerrit:1326852|CategoryTree: Allow null $html in the CategoryViewerGenerateLink handler (T435161)]], [[gerrit:1326853|CategoryViewer: Allow null $html in the CategoryViewerGenerateLink hook (T435161)]], [[gerrit:1326865|Flow: Allow null $html in the CategoryViewerGenerateLink handler (T435161)]] [production]
18:11 <ladsgroup@deploy1003> helmfile [eqiad] START helmfile.d/services/mw-experimental: apply [production]
18:08 <btullis@cumin1003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1146.eqiad.wmnet with OS bookworm [production]
18:07 <btullis@cumin1003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1177.eqiad.wmnet with OS bookworm [production]
18:00 <ladsgroup@deploy1003> Finished scap sync-world: Backport for [[gerrit:1326380|Introduce main lock manager service (T366938 T427999)]] (duration: 11m 25s) [production]
17:58 <ladsgroup@deploy1003> ladsgroup: Rolling back deployment [production]
17:58 <btullis@cumin1003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1202.eqiad.wmnet with OS bookworm [production]
17:55 <btullis@cumin1003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1201.eqiad.wmnet with OS bookworm [production]
17:50 <ladsgroup@deploy1003> ladsgroup: Backport for [[gerrit:1326380|Introduce main lock manager service (T366938 T427999)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
17:48 <ladsgroup@deploy1003> Started scap sync-world: Backport for [[gerrit:1326380|Introduce main lock manager service (T366938 T427999)]] [production]
17:48 <btullis@cumin1003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1156.eqiad.wmnet with reason: host reimage [production]
17:46 <btullis@cumin1003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1146.eqiad.wmnet with reason: host reimage [production]
17:45 <cdobbins@cumin1003> END (PASS) - Cookbook sre.cdn.roll-upgrade-ats (exit_code=0) Rolling upgrade of ATS on A:cp-esams and A:cp - 9.2.15 upgrade (T434478) [production]
17:42 <jasmine@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-ctrl2006.codfw.wmnet with OS trixie [production]
17:41 <btullis@cumin1003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1177.eqiad.wmnet with reason: host reimage [production]
17:38 <btullis@cumin1003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1202.eqiad.wmnet with reason: host reimage [production]
17:34 <btullis@cumin1003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-worker1201.eqiad.wmnet with reason: host reimage [production]
17:30 <btullis@cumin1003> START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1156.eqiad.wmnet with reason: host reimage [production]
17:29 <btullis@cumin1003> START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1177.eqiad.wmnet with reason: host reimage [production]
17:28 <btullis@cumin1003> START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1146.eqiad.wmnet with reason: host reimage [production]
17:28 <btullis@cumin1003> START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1202.eqiad.wmnet with reason: host reimage [production]
17:27 <btullis@cumin1003> START - Cookbook sre.hosts.downtime for 2:00:00 on an-worker1201.eqiad.wmnet with reason: host reimage [production]
17:25 <jasmine@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-ctrl2006.codfw.wmnet with reason: host reimage [production]
17:21 <jasmine@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-ctrl2006.codfw.wmnet with reason: host reimage [production]
17:14 <btullis@cumin1003> START - Cookbook sre.hosts.reimage for host an-worker1202.eqiad.wmnet with OS bookworm [production]
17:14 <btullis@cumin1003> START - Cookbook sre.hosts.reimage for host an-worker1201.eqiad.wmnet with OS bookworm [production]
17:14 <btullis@cumin1003> START - Cookbook sre.hosts.reimage for host an-worker1177.eqiad.wmnet with OS bookworm [production]
17:14 <btullis@cumin1003> START - Cookbook sre.hosts.reimage for host an-worker1156.eqiad.wmnet with OS bookworm [production]
17:14 <btullis@cumin1003> START - Cookbook sre.hosts.reimage for host an-worker1146.eqiad.wmnet with OS bookworm [production]
17:09 <dancy@deploy1003> Finished scap sync-world: Backport for [[gerrit:1326885|w/deployment-info.php: Handle new file format (T434726)]] (duration: 07m 15s) [production]
17:08 <jasmine@cumin2002> START - Cookbook sre.hosts.reimage for host wikikube-ctrl2006.codfw.wmnet with OS trixie [production]
17:05 <dancy@deploy1003> dancy: Continuing with deployment [production]
17:04 <dancy@deploy1003> dancy: Backport for [[gerrit:1326885|w/deployment-info.php: Handle new file format (T434726)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
17:02 <dancy@deploy1003> Started scap sync-world: Backport for [[gerrit:1326885|w/deployment-info.php: Handle new file format (T434726)]] [production]
16:50 <dancy@deploy1003> Finished scap sync-world: Testing T434726 (duration: 06m 40s) [production]
16:43 <dancy@deploy1003> Started scap sync-world: Testing T434726 [production]
16:43 <dancy@deploy1003> Installation of scap version "4.282.0" completed for 3 hosts [production]
16:41 <dancy@deploy1003> Installing scap version "4.282.0" for 3 host(s) [production]
16:41 <btullis@cumin1003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-worker1145.eqiad.wmnet with OS bookworm [production]