151-200 of 10000 results (96ms)
2024-09-09 ยง
14:21 <cgoubert@cumin1002> START - Cookbook sre.k8s.pool-depool-node depool for host mw2429.codfw.wmnet [production]
14:21 <cgoubert@cumin1002> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) depool for host mw2428.codfw.wmnet [production]
14:21 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2140', diff saved to https://phabricator.wikimedia.org/P68757 and previous config saved to /var/cache/conftool/dbconfig/20240909-142131-ladsgroup.json [production]
14:20 <cgoubert@cumin1002> START - Cookbook sre.k8s.pool-depool-node depool for host mw2428.codfw.wmnet [production]
14:17 <jforrester@deploy1003> Finished scap sync-world: Backport for [[gerrit:1071566|Revert "Activate feature flag for moving wikibase item to Other Projects sidebar in pilot wikis." (T66315)]], [[gerrit:1070948|Enable Copyupload-allowed-domains on test2wiki (T356241)]] (duration: 12m 16s) [production]
14:13 <jforrester@deploy1003> seanleong-wmde, jforrester, hnowlan: Continuing with sync [production]
14:08 <jforrester@deploy1003> seanleong-wmde, jforrester, hnowlan: Backport for [[gerrit:1071566|Revert "Activate feature flag for moving wikibase item to Other Projects sidebar in pilot wikis." (T66315)]], [[gerrit:1070948|Enable Copyupload-allowed-domains on test2wiki (T356241)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
14:07 <sukhe@cumin1002> START - Cookbook sre.dns.roll-restart-ntp rolling restart_daemons on A:dnsbox and not P{dns1004* or dns1005*} and A:dnsbox [production]
14:06 <ladsgroup@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2140 (T371742)', diff saved to https://phabricator.wikimedia.org/P68756 and previous config saved to /var/cache/conftool/dbconfig/20240909-140623-ladsgroup.json [production]
14:05 <jforrester@deploy1003> Started scap sync-world: Backport for [[gerrit:1071566|Revert "Activate feature flag for moving wikibase item to Other Projects sidebar in pilot wikis." (T66315)]], [[gerrit:1070948|Enable Copyupload-allowed-domains on test2wiki (T356241)]] [production]
14:03 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-test-k8s: apply [production]
14:02 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-test-k8s: apply [production]
14:01 <kamila@cumin1002> END (PASS) - Cookbook sre.k8s.renumber-node (exit_code=0) Renumbering for host wikikube-worker2104.codfw.wmnet [production]
14:01 <kamila@cumin1002> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker2104.codfw.wmnet [production]
14:01 <kamila@cumin1002> START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker2104.codfw.wmnet [production]
14:00 <sukhe@cumin1002> END (PASS) - Cookbook sre.dns.roll-restart-reboot-wikimedia-dns (exit_code=0) rolling restart_daemons on A:wikidough and A:wikidough [production]
13:58 <jforrester@deploy1003> Finished scap sync-world: Backport for [[gerrit:1071251|Define wgCheckUserCentralIndexRangesToExclude to exclude WMCS (T373021)]], [[gerrit:1071253|tests: Disable all Beta Cluster CI testing, all failing (T374242)]], [[gerrit:1071254|Don't pass empty type/returnType to zobject lookup when undefined (T374199)]], [[gerrit:1071265|Use default width/height on gallery to avoid parser instance (T37414 [production]
13:55 <kamila_> homer lsw1-b6-codfw* commit 'T372878' [production]
13:55 <kamila_> homer cr*codfw* commit 'T372878' [production]
13:54 <kamila@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker2104.codfw.wmnet with OS bullseye [production]
13:52 <jforrester@deploy1003> dreamyjazz, jforrester: Continuing with sync [production]
13:50 <jforrester@deploy1003> dreamyjazz, jforrester: Backport for [[gerrit:1071251|Define wgCheckUserCentralIndexRangesToExclude to exclude WMCS (T373021)]], [[gerrit:1071253|tests: Disable all Beta Cluster CI testing, all failing (T374242)]], [[gerrit:1071254|Don't pass empty type/returnType to zobject lookup when undefined (T374199)]], [[gerrit:1071265|Use default width/height on gallery to avoid parser instance (T374146) [production]
13:49 <aqu@deploy1003> Finished deploy [airflow-dags/analytics_test@5315c8d]: Test Refine through Airflow (duration: 00m 11s) [production]
13:49 <aqu@deploy1003> Started deploy [airflow-dags/analytics_test@5315c8d]: Test Refine through Airflow [production]
13:47 <mszabo@deploy1003> helmfile [eqiad] DONE helmfile.d/services/ipoid: sync [production]
13:46 <sukhe@cumin1002> START - Cookbook sre.dns.roll-restart-reboot-wikimedia-dns rolling restart_daemons on A:wikidough and A:wikidough [production]
13:46 <mszabo@deploy1003> helmfile [eqiad] START helmfile.d/services/ipoid: sync [production]
13:46 <jforrester@deploy1003> Started scap sync-world: Backport for [[gerrit:1071251|Define wgCheckUserCentralIndexRangesToExclude to exclude WMCS (T373021)]], [[gerrit:1071253|tests: Disable all Beta Cluster CI testing, all failing (T374242)]], [[gerrit:1071254|Don't pass empty type/returnType to zobject lookup when undefined (T374199)]], [[gerrit:1071265|Use default width/height on gallery to avoid parser instance (T374146 [production]
13:45 <James_F> kill 4135240 # scap thread with no attached screen [production]
13:41 <mszabo@deploy1003> helmfile [codfw] DONE helmfile.d/services/ipoid: apply [production]
13:40 <mszabo@deploy1003> helmfile [codfw] START helmfile.d/services/ipoid: apply [production]
13:40 <jforrester@deploy1003> dreamyjazz, jforrester: Backport for [[gerrit:1071251|Define wgCheckUserCentralIndexRangesToExclude to exclude WMCS (T373021)]], [[gerrit:1071253|tests: Disable all Beta Cluster CI testing, all failing (T374242)]], [[gerrit:1071254|Don't pass empty type/returnType to zobject lookup when undefined (T374199)]], [[gerrit:1071265|Use default width/height on gallery to avoid parser instance (T374146) [production]
13:39 <mszabo@deploy1003> helmfile [eqiad] DONE helmfile.d/services/ipoid: apply [production]
13:39 <mszabo@deploy1003> helmfile [eqiad] START helmfile.d/services/ipoid: apply [production]
13:38 <mszabo@deploy1003> helmfile [staging] DONE helmfile.d/services/ipoid: apply [production]
13:37 <mszabo@deploy1003> helmfile [staging] START helmfile.d/services/ipoid: apply [production]
13:36 <jforrester@deploy1003> Started scap sync-world: Backport for [[gerrit:1071251|Define wgCheckUserCentralIndexRangesToExclude to exclude WMCS (T373021)]], [[gerrit:1071253|tests: Disable all Beta Cluster CI testing, all failing (T374242)]], [[gerrit:1071254|Don't pass empty type/returnType to zobject lookup when undefined (T374199)]], [[gerrit:1071265|Use default width/height on gallery to avoid parser instance (T374146 [production]
13:34 <sukhe> sudo cumin "A:dnsbox" 'disable-puppet "merging CR 1071616"' [production]
13:32 <milimetric@deploy1003> Finished deploy [airflow-dags/platform_eng@574f0de]: (no justification provided) (duration: 00m 26s) [production]
13:32 <milimetric@deploy1003> Started deploy [airflow-dags/platform_eng@574f0de]: (no justification provided) [production]
13:13 <dreamyjazz@deploy1003> dreamyjazz: Continuing with sync [production]
13:13 <dreamyjazz@deploy1003> dreamyjazz: Backport for [[gerrit:1071251|Define wgCheckUserCentralIndexRangesToExclude to exclude WMCS (T373021)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
13:09 <kamila@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker2104.codfw.wmnet with reason: host reimage [production]
13:08 <dreamyjazz@deploy1003> Started scap sync-world: Backport for [[gerrit:1071251|Define wgCheckUserCentralIndexRangesToExclude to exclude WMCS (T373021)]] [production]
13:06 <kamila@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker2104.codfw.wmnet with reason: host reimage [production]
12:59 <dreamyjazz@deploy1003> Started scap sync-world: Backport for [[gerrit:1071251|Define wgCheckUserCentralIndexRangesToExclude to exclude WMCS (T373021)]] [production]
12:48 <kamila@cumin1002> END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host wikikube-worker2104 [production]
12:48 <kamila@cumin1002> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker2104 [production]
12:48 <kamila@cumin1002> START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker2104 [production]
12:48 <kamila@cumin1002> END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) wikikube-worker2104.codfw.wmnet 61.16.192.10.in-addr.arpa 1.6.0.0.6.1.0.0.2.9.1.0.0.1.0.0.2.0.1.0.0.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors [production]