651-700 of 10000 results (101ms)
2024-11-07 ยง
14:15 <jhancock@cumin2002> START - Cookbook sre.dns.netbox [production]
14:15 <lucaswerkmeister-wmde@deploy2002> Started scap sync-world: Backport for [[gerrit:1087927|Deploy EditCheck (references) to hiwiki, bnwiki, idwiki (T366381)]] [production]
14:13 <kartik@deploy2002> Finished scap sync-world: Backport for [[gerrit:1088215|Enable Section Translation in ann, iba, nr and, tdd Wikipedias (T371420)]] (duration: 10m 08s) [production]
14:09 <kartik@deploy2002> kartik: Continuing with sync [production]
14:06 <kartik@deploy2002> kartik: Backport for [[gerrit:1088215|Enable Section Translation in ann, iba, nr and, tdd Wikipedias (T371420)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
14:04 <joal@deploy2002> Finished deploy [airflow-dags/analytics@23bc4ad]: Regular analytics weekly train [airflow-dags/analytics@23bc4ad3] (duration: 01m 44s) [production]
14:03 <kartik@deploy2002> Started scap sync-world: Backport for [[gerrit:1088215|Enable Section Translation in ann, iba, nr and, tdd Wikipedias (T371420)]] [production]
14:03 <joal@deploy2002> Started deploy [airflow-dags/analytics@23bc4ad]: Regular analytics weekly train [airflow-dags/analytics@23bc4ad3] [production]
13:52 <cwhite> running thanos bucket cleanup on titan1001 - T351927 [production]
13:37 <jmm@cumin2002> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host ganeti1048 [production]
13:36 <jmm@cumin2002> START - Cookbook sre.network.configure-switch-interfaces for host ganeti1048 [production]
13:35 <jmm@cumin2002> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host ganeti1047 [production]
13:34 <jmm@cumin2002> START - Cookbook sre.network.configure-switch-interfaces for host ganeti1047 [production]
13:23 <joal@deploy2002> Finished deploy [analytics/refinery@4bec064] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@4bec0640] (duration: 03m 44s) [production]
13:20 <joal@deploy2002> Started deploy [analytics/refinery@4bec064] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@4bec0640] [production]
13:13 <joal@deploy2002> Finished deploy [analytics/refinery@4bec064] (thin): Regular analytics weekly train THIN [analytics/refinery@4bec0640] (duration: 05m 03s) [production]
13:08 <joal@deploy2002> Started deploy [analytics/refinery@4bec064] (thin): Regular analytics weekly train THIN [analytics/refinery@4bec0640] [production]
12:53 <joal@deploy2002> Finished deploy [analytics/refinery@4bec064]: Regular analytics weekly train [analytics/refinery@4bec0640] (duration: 16m 47s) [production]
12:40 <jmm@cumin2002> END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host ganeti1047 [production]
12:40 <jmm@cumin2002> START - Cookbook sre.network.configure-switch-interfaces for host ganeti1047 [production]
12:39 <jmm@cumin2002> END (FAIL) - Cookbook sre.network.configure-switch-interfaces (exit_code=99) for host ganeti1047 [production]
12:37 <jmm@cumin2002> START - Cookbook sre.network.configure-switch-interfaces for host ganeti1047 [production]
12:36 <joal@deploy2002> Started deploy [analytics/refinery@4bec064]: Regular analytics weekly train [analytics/refinery@4bec0640] [production]
12:16 <vgutierrez> repool liberica on lvs1013 [production]
11:44 <sfaci@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mpic: apply [production]
11:44 <sfaci@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mpic: apply [production]
11:27 <jgiannelos@deploy2002> helmfile [eqiad] DONE helmfile.d/services/proton: sync [production]
11:26 <jgiannelos@deploy2002> helmfile [eqiad] START helmfile.d/services/proton: sync [production]
11:26 <jgiannelos@deploy2002> helmfile [codfw] DONE helmfile.d/services/proton: sync [production]
11:25 <jgiannelos@deploy2002> helmfile [codfw] START helmfile.d/services/proton: sync [production]
11:24 <jgiannelos@deploy2002> helmfile [staging] DONE helmfile.d/services/proton: sync [production]
11:24 <jgiannelos@deploy2002> helmfile [staging] START helmfile.d/services/proton: sync [production]
11:19 <isaranto@deploy2002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-editquality-reverted' for release 'main' . [production]
11:19 <sfaci@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mpic-next: apply [production]
11:19 <sfaci@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mpic-next: apply [production]
11:18 <isaranto@deploy2002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-editquality-goodfaith' for release 'main' . [production]
11:17 <isaranto@deploy2002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-editquality-damaging' for release 'main' . [production]
11:17 <isaranto@deploy2002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-drafttopic' for release 'main' . [production]
11:17 <isaranto@deploy2002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-draftquality' for release 'main' . [production]
11:17 <isaranto@deploy2002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-articletopic' for release 'main' . [production]
11:16 <isaranto@deploy2002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-articlequality' for release 'main' . [production]
11:11 <isaranto@deploy2002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-editquality-reverted' for release 'main' . [production]
11:10 <isaranto@deploy2002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-editquality-goodfaith' for release 'main' . [production]
11:09 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1010.eqiad.wmnet [production]
11:09 <isaranto@deploy2002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-editquality-damaging' for release 'main' . [production]
11:04 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1010.eqiad.wmnet [production]
11:03 <vgutierrez> depool liberica on lvs1013 [production]
11:01 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1010.eqiad.wmnet [production]
10:58 <elukey@cumin1002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ms-be2082.mgmt.codfw.wmnet with chassis set policy FORCE_RESTART [production]
10:55 <jmm@cumin2002> END (PASS) - Cookbook sre.kafka.roll-restart-reboot-brokers (exit_code=0) rolling restart_daemons on A:kafka-test-eqiad [production]