8101-8150 of 10000 results (107ms)
2023-10-26 ยง
15:42 <sukhe> sudo cumin -b1 -s600 'A:dns-rec and (A:eqiad or A:codfw)' 'systemctl restart ntp.service' [production]
15:42 <sukhe@cumin2002> END (PASS) - Cookbook sre.dns.roll-restart-reboot-wikimedia-dns (exit_code=0) rolling restart_daemons on A:wikidough and A:wikidough [production]
15:35 <jgiannelos@deploy2002> Finished deploy [restbase/deploy@4c14785]: (no justification provided) (duration: 13m 21s) [production]
15:30 <XioNoX> test add BGP session between ssw1-e1-eqiad and lsw1-e8-eqiad [production]
15:22 <jgiannelos@deploy2002> Started deploy [restbase/deploy@4c14785]: (no justification provided) [production]
15:15 <bking@cumin1001> END (PASS) - Cookbook sre.wdqs.data-transfer (exit_code=0) [production]
15:12 <cmooney@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on sretest2003.codfw.wmnet with reason: host reimage [production]
15:09 <cmooney@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on sretest2003.codfw.wmnet with reason: host reimage [production]
14:53 <Lucas_WMDE> UTC afternoon backport+config window (belatedly) done [production]
14:52 <lucaswerkmeister-wmde@deploy2002> Finished scap: Backport for [[gerrit:968776|CX3 Build 0.2.0+20231026 (T348563 T308836)]] (duration: 14m 01s) [production]
14:49 <sukhe@cumin2002> START - Cookbook sre.dns.roll-restart-reboot-wikimedia-dns rolling restart_daemons on A:wikidough and A:wikidough [production]
14:46 <lucaswerkmeister-wmde@deploy2002> kartik and lucaswerkmeister-wmde: Continuing with sync [production]
14:42 <jgiannelos@deploy2002> Finished deploy [restbase/deploy@ff46322]: (no justification provided) (duration: 01m 38s) [production]
14:40 <jgiannelos@deploy2002> Started deploy [restbase/deploy@ff46322]: (no justification provided) [production]
14:39 <lucaswerkmeister-wmde@deploy2002> kartik and lucaswerkmeister-wmde: Backport for [[gerrit:968776|CX3 Build 0.2.0+20231026 (T348563 T308836)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
14:38 <lucaswerkmeister-wmde@deploy2002> Started scap: Backport for [[gerrit:968776|CX3 Build 0.2.0+20231026 (T348563 T308836)]] [production]
14:36 <filippo@deploy2002> helmfile [staging] DONE helmfile.d/services/opentelemetry-collector: apply [production]
14:36 <filippo@deploy2002> helmfile [staging] START helmfile.d/services/opentelemetry-collector: apply [production]
14:36 <filippo@deploy2002> helmfile [codfw] DONE helmfile.d/services/opentelemetry-collector: apply [production]
14:36 <filippo@deploy2002> helmfile [codfw] START helmfile.d/services/opentelemetry-collector: apply [production]
14:36 <filippo@deploy2002> helmfile [eqiad] DONE helmfile.d/services/opentelemetry-collector: apply [production]
14:35 <filippo@deploy2002> helmfile [eqiad] START helmfile.d/services/opentelemetry-collector: apply [production]
14:33 <cmooney@cumin1001> START - Cookbook sre.hosts.reimage for host sretest2003.codfw.wmnet with OS bullseye [production]
14:23 <lucaswerkmeister-wmde@deploy2002> Finished scap: Backport for [[gerrit:968783|Remove broken QUnit test (T349485)]] (duration: 06m 53s) [production]
14:20 <ejegg> donorwiki upgraded from 894eacce to f7407053 [production]
14:17 <lucaswerkmeister-wmde@deploy2002> abi and lucaswerkmeister-wmde: Continuing with sync [production]
14:17 <lucaswerkmeister-wmde@deploy2002> abi and lucaswerkmeister-wmde: Backport for [[gerrit:968783|Remove broken QUnit test (T349485)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
14:16 <lucaswerkmeister-wmde@deploy2002> Started scap: Backport for [[gerrit:968783|Remove broken QUnit test (T349485)]] [production]
14:14 <jayme@deploy2002> helmfile [codfw] START helmfile.d/services/similar-users: apply [production]
14:14 <jayme@deploy2002> helmfile [eqiad] DONE helmfile.d/services/similar-users: apply [production]
14:14 <jayme@deploy2002> helmfile [eqiad] START helmfile.d/services/similar-users: apply [production]
14:09 <jayme@deploy2002> helmfile [staging] DONE helmfile.d/services/similar-users: apply [production]
14:09 <jayme@deploy2002> helmfile [staging] START helmfile.d/services/similar-users: apply [production]
13:56 <lucaswerkmeister-wmde@deploy2002> Finished scap: Backport for [[gerrit:969064|cirrus: disable canary events for update & error streams]] (duration: 07m 19s) [production]
13:51 <lucaswerkmeister-wmde@deploy2002> dcausse and lucaswerkmeister-wmde: Continuing with sync [production]
13:50 <lucaswerkmeister-wmde@deploy2002> dcausse and lucaswerkmeister-wmde: Backport for [[gerrit:969064|cirrus: disable canary events for update & error streams]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
13:49 <lucaswerkmeister-wmde@deploy2002> Started scap: Backport for [[gerrit:969064|cirrus: disable canary events for update & error streams]] [production]
13:46 <moritzm> installing cpio security updates [production]
13:45 <lucaswerkmeister-wmde@deploy2002> Finished scap: Backport for [[gerrit:968777|CX3 Build 0.2.0+20231026 (T348563 T308836)]] (duration: 14m 48s) [production]
13:40 <lucaswerkmeister-wmde@deploy2002> lucaswerkmeister-wmde and kartik: Continuing with sync [production]
13:32 <lucaswerkmeister-wmde@deploy2002> lucaswerkmeister-wmde and kartik: Backport for [[gerrit:968777|CX3 Build 0.2.0+20231026 (T348563 T308836)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
13:32 <jmm@cumin2002> END (PASS) - Cookbook sre.maps.roll-restart-reboot (exit_code=0) rolling restart_daemons on A:maps-replica-eqiad [production]
13:31 <moritzm> installing curl security updates on buster [production]
13:31 <lucaswerkmeister-wmde@deploy2002> Started scap: Backport for [[gerrit:968777|CX3 Build 0.2.0+20231026 (T348563 T308836)]] [production]
13:30 <lucaswerkmeister-wmde@deploy2002> Finished scap: Backport for [[gerrit:968737|Add throttle rule for Edit-a-Thon on 2023-11-03 (T349234)]] (duration: 06m 43s) [production]
13:27 <jmm@cumin2002> START - Cookbook sre.maps.roll-restart-reboot rolling restart_daemons on A:maps-replica-eqiad [production]
13:26 <jmm@cumin2002> END (PASS) - Cookbook sre.maps.roll-restart-reboot (exit_code=0) rolling restart_daemons on A:maps-replica-codfw [production]
13:25 <lucaswerkmeister-wmde@deploy2002> zoranzoki21 and lucaswerkmeister-wmde: Continuing with sync [production]
13:24 <lucaswerkmeister-wmde@deploy2002> zoranzoki21 and lucaswerkmeister-wmde: Backport for [[gerrit:968737|Add throttle rule for Edit-a-Thon on 2023-11-03 (T349234)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
13:23 <lucaswerkmeister-wmde@deploy2002> Started scap: Backport for [[gerrit:968737|Add throttle rule for Edit-a-Thon on 2023-11-03 (T349234)]] [production]