3001-3050 of 10000 results (95ms)
2023-12-13 §
06:10 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on dbproxy1021.eqiad.wmnet with reason: host reimage [production]
06:06 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on dbproxy1021.eqiad.wmnet with reason: host reimage [production]
05:54 <marostegui@cumin1001> START - Cookbook sre.hosts.reimage for host dbproxy1021.eqiad.wmnet with OS bookworm [production]
03:41 <hashar@deploy2002> Finished deploy [gerrit/gerrit@9bf8914]: Add a banner for the 2023 developer survey - T351109 (duration: 00m 08s) [production]
03:41 <hashar@deploy2002> Started deploy [gerrit/gerrit@9bf8914]: Add a banner for the 2023 developer survey - T351109 [production]
2023-12-12 §
23:56 <ejegg> donorwiki upgraded from f7407053 to bc49e5a6 [production]
23:26 <tzatziki> removing 2 files for legal compliance [production]
23:05 <tzatziki> removing 2 files for legal compliance [production]
22:57 <mutante> planet - switched to eqiad and bookworm backend (T348392 T345617) - https://meta.wikimedia.org/wiki/Planet_Wikimedia [production]
22:43 <mutante> planet2003 -manually upgrade rawdog package to 3.0.2 T348392 [production]
21:33 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on planet1003.eqiad.wmnet with reason: debugging [production]
21:33 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 7 days, 0:00:00 on planet1003.eqiad.wmnet with reason: debugging [production]
21:33 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on planet1003.eqiad.wmnet with reason: debugging [production]
21:32 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on planet2003.codfw.wmnet with reason: debugging [production]
21:32 <dzahn@cumin1001> START - Cookbook sre.hosts.downtime for 7 days, 0:00:00 on planet2003.codfw.wmnet with reason: debugging [production]
21:32 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on planet2003.codfw.wmnet with reason: reimage [production]
21:18 <samtar@deploy2002> Finished scap: Backport for [[gerrit:980963|Add stream config for Android article instruments (T351292)]] (duration: 11m 59s) [production]
21:10 <samtar@deploy2002> cjming and samtar: Continuing with sync [production]
21:07 <samtar@deploy2002> cjming and samtar: Backport for [[gerrit:980963|Add stream config for Android article instruments (T351292)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
21:06 <samtar@deploy2002> Started scap: Backport for [[gerrit:980963|Add stream config for Android article instruments (T351292)]] [production]
20:42 <rzl@deploy2002> helmfile [eqiad] DONE helmfile.d/admin 'apply'. [production]
20:40 <rzl@deploy2002> helmfile [eqiad] START helmfile.d/admin 'apply'. [production]
20:38 <rzl@deploy2002> helmfile [codfw] DONE helmfile.d/admin 'apply'. [production]
20:37 <rzl@deploy2002> helmfile [codfw] START helmfile.d/admin 'apply'. [production]
20:33 <rzl@deploy2002> helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. [production]
20:30 <rzl@deploy2002> helmfile [staging-eqiad] START helmfile.d/admin 'apply'. [production]
20:28 <rzl@deploy2002> helmfile [staging-codfw] DONE helmfile.d/admin 'apply'. [production]
20:17 <rzl@deploy2002> helmfile [staging-codfw] START helmfile.d/admin 'apply'. [production]
20:05 <rzl@deploy2002> helmfile [eqiad] DONE helmfile.d/admin 'apply'. [production]
20:04 <rzl@deploy2002> helmfile [eqiad] START helmfile.d/admin 'apply'. [production]
19:59 <bking@cumin2002> END (ERROR) - Cookbook sre.elasticsearch.rolling-operation (exit_code=97) Operation.RESTART (1 nodes at a time) for ElasticSearch cluster relforge: enable new wmf-elasticsearch-search-plugins - bking@cumin2002 - T353270 [production]
19:57 <rzl@deploy2002> helmfile [codfw] DONE helmfile.d/admin 'apply'. [production]
19:56 <rzl@deploy2002> helmfile [codfw] START helmfile.d/admin 'apply'. [production]
19:46 <ryankemper@deploy2002> helmfile [staging] DONE helmfile.d/services/cirrus-streaming-updater: apply [production]
19:46 <ryankemper@deploy2002> helmfile [staging] START helmfile.d/services/cirrus-streaming-updater: apply [production]
19:43 <brennen@deploy2002> rebuilt and synchronized wikiversions files: group0 wikis to 1.42.0-wmf.9 refs T350085 [production]
19:33 <brennen@deploy2002> Finished scap: Backport for [[gerrit:982237|ParserOutput::getText(): do not clone ParserOutput when invoking pipeline (T353257)]] (duration: 09m 41s) [production]
19:26 <brennen@deploy2002> brennen and ssastry: Continuing with sync [production]
19:25 <brennen@deploy2002> brennen and ssastry: Backport for [[gerrit:982237|ParserOutput::getText(): do not clone ParserOutput when invoking pipeline (T353257)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
19:23 <brennen@deploy2002> Started scap: Backport for [[gerrit:982237|ParserOutput::getText(): do not clone ParserOutput when invoking pipeline (T353257)]] [production]
19:18 <bking@cumin2002> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (1 nodes at a time) for ElasticSearch cluster relforge: enable new wmf-elasticsearch-search-plugins - bking@cumin2002 - T353270 [production]
19:08 <brennen> 1.42.0-wmf.9 (T350085) status: deploying a fix for T353257 and then will proceed to group0. [production]
19:03 <bking@cumin2002> END (FAIL) - Cookbook sre.elasticsearch.rolling-operation (exit_code=99) Operation.RESTART (1 nodes at a time) for ElasticSearch cluster relforge: enable new wmf-elasticsearch-search-plugins - bking@cumin2002 - T353270 [production]
19:03 <bking@cumin2002> START - Cookbook sre.elasticsearch.rolling-operation Operation.RESTART (1 nodes at a time) for ElasticSearch cluster relforge: enable new wmf-elasticsearch-search-plugins - bking@cumin2002 - T353270 [production]
18:55 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host phab2002.codfw.wmnet with OS bullseye [production]
18:33 <dzahn@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on phab2002.codfw.wmnet with reason: host reimage [production]
18:32 <rzl@deploy2002> helmfile [staging-eqiad] DONE helmfile.d/admin 'apply'. [production]
18:31 <rzl@deploy2002> helmfile [staging-eqiad] START helmfile.d/admin 'apply'. [production]
18:29 <rzl@deploy2002> helmfile [staging-codfw] DONE helmfile.d/admin 'apply'. [production]
18:28 <rzl@deploy2002> helmfile [staging-codfw] START helmfile.d/admin 'apply'. [production]