101-150 of 10000 results (11ms)
2026-04-07 §
07:15 <taavi@cloudcumin1001> END (FAIL) - Cookbook wmcs.toolforge.component.deploy (exit_code=99) for component istio-gateway [toolsbeta]
07:14 <taavi@cloudcumin1001> START - Cookbook wmcs.toolforge.component.deploy for component istio-gateway [toolsbeta]
07:14 <marostegui@cumin1003> END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2157: after upgrade [production]
07:14 <marostegui@cumin1003> END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1159: after upgrade [production]
06:32 <marostegui@cumin1003> END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2142: Upgrade package [production]
06:32 <marostegui@cumin1003> END (PASS) - Cookbook sre.mysql.parsercache (exit_code=0) [production]
06:32 <marostegui@cumin1003> START - Cookbook sre.mysql.parsercache [production]
06:32 <marostegui@cumin1003> START - Cookbook sre.mysql.pool pool db2142: Upgrade package [production]
06:32 <marostegui@cumin1003> END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2248: Upgrade package [production]
06:29 <marostegui@cumin1003> START - Cookbook sre.mysql.pool pool db2157: after upgrade [production]
06:28 <marostegui@cumin1003> START - Cookbook sre.mysql.pool pool db1159: after upgrade [production]
06:16 <marostegui@cumin1003> START - Cookbook sre.mysql.pool pool db2248: Upgrade package [production]
06:16 <marostegui@cumin1003> END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db2249: Upgrade package [production]
06:02 <marostegui@cumin1003> END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool pc1011: after upgrade [production]
06:02 <marostegui@cumin1003> END (PASS) - Cookbook sre.mysql.parsercache (exit_code=0) [production]
06:02 <marostegui@cumin1003> START - Cookbook sre.mysql.pool pool pc1011: after upgrade [production]
06:02 <marostegui@cumin1003> START - Cookbook sre.mysql.parsercache [production]
06:01 <marostegui@cumin1003> START - Cookbook sre.mysql.pool pool db2249: Upgrade package [production]
06:01 <marostegui@cumin1003> END (PASS) - Cookbook sre.mysql.pool (exit_code=0) pool db1169: Upgrade package [production]
05:45 <marostegui@cumin1003> START - Cookbook sre.mysql.pool pool db1169: Upgrade package [production]
05:37 <marostegui@cumin1003> END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db1169: Upgrade package [production]
05:37 <marostegui@cumin1003> START - Cookbook sre.mysql.depool depool db1169: Upgrade package [production]
05:37 <marostegui@cumin1003> END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2142: Upgrade package [production]
05:36 <marostegui@cumin1003> END (PASS) - Cookbook sre.mysql.parsercache (exit_code=0) [production]
05:36 <marostegui@cumin1003> START - Cookbook sre.mysql.parsercache [production]
05:36 <marostegui@cumin1003> START - Cookbook sre.mysql.depool depool db2142: Upgrade package [production]
05:35 <marostegui@cumin1003> END (PASS) - Cookbook sre.mysql.depool (exit_code=0) depool db2248: Upgrade package [production]
05:35 <marostegui@cumin1003> START - Cookbook sre.mysql.depool depool db2248: Upgrade package [production]
05:33 <marostegui@cumin1003> END (FAIL) - Cookbook sre.mysql.depool (exit_code=99) depool db2248: Upgrade package [production]
05:33 <marostegui@cumin1003> START - Cookbook sre.mysql.depool depool db2248: Upgrade package [production]
05:31 <marostegui@cumin1003> END (FAIL) - Cookbook sre.mysql.depool (exit_code=99) depool db2249.codfw.wmnet: Upgrade package [production]
05:31 <marostegui@cumin1003> START - Cookbook sre.mysql.depool depool db2249.codfw.wmnet: Upgrade package [production]
05:31 <marostegui@cumin1003> END (FAIL) - Cookbook sre.mysql.depool (exit_code=99) depool db2249: Upgrade package [production]
05:31 <marostegui@cumin1003> START - Cookbook sre.mysql.depool depool db2249: Upgrade package [production]
05:30 <marostegui@cumin1003> END (FAIL) - Cookbook sre.mysql.depool (exit_code=99) depool db2249: Upgrade package [production]
05:30 <marostegui@cumin1003> START - Cookbook sre.mysql.depool depool db2249: Upgrade package [production]
05:29 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on db[2142,2248-2249].codfw.wmnet,db1169.eqiad.wmnet with reason: Upgrade to 10.11.16.v3 [production]
05:29 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on db[2142,2248-2249].codfw.wmnet with reason: Upgrade to 10.11.16.v3 [production]
05:21 <marostegui@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on pc2011.codfw.wmnet,pc1011.eqiad.wmnet with reason: Upgrade to 10.11.16.v3 [production]
05:20 <marostegui@cumin1003> END (PASS) - Cookbook sre.mysql.parsercache (exit_code=0) [production]
05:20 <marostegui@cumin1003> START - Cookbook sre.mysql.parsercache [production]
04:02 <mwpresync@deploy1003> Pruned MediaWiki: 1.46.0-wmf.20 (duration: 02m 27s) [production]
03:39 <mwpresync@deploy1003> Finished scap sync-world: testwikis to 1.46.0-wmf.23 refs T420481 (duration: 35m 55s) [production]
03:03 <mwpresync@deploy1003> Started scap sync-world: testwikis to 1.46.0-wmf.23 refs T420481 [production]
00:10 <zabe@deploy1003> Finished scap sync-world: Backport for [[gerrit:1268291|Start reading from the new file tables on more large wikis (T416548)]] (duration: 06m 22s) [production]
00:05 <zabe@deploy1003> zabe: Continuing with sync [production]
00:05 <zabe@deploy1003> zabe: Backport for [[gerrit:1268291|Start reading from the new file tables on more large wikis (T416548)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
00:03 <zabe@deploy1003> Started scap sync-world: Backport for [[gerrit:1268291|Start reading from the new file tables on more large wikis (T416548)]] [production]
00:02 <dzahn@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host gitlab-runner1003.eqiad.wmnet with OS bookworm [production]
2026-04-06 §
23:43 <dzahn@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on gitlab-runner1003.eqiad.wmnet with reason: host reimage [production]