201-250 of 10000 results (128ms)
2024-11-05 ยง
14:09 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on pc1017.eqiad.wmnet with reason: host reimage [production]
14:08 <moritzm> installing PHP 7.4 security updates on bullseye (as packaged in Debian) [production]
14:08 <akosiaris@deploy2002> helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply [production]
14:07 <akosiaris@deploy2002> helmfile [codfw] START helmfile.d/services/rest-gateway: apply [production]
14:07 <akosiaris@deploy2002> helmfile [staging] DONE helmfile.d/services/rest-gateway: apply [production]
14:07 <akosiaris@deploy2002> helmfile [staging] START helmfile.d/services/rest-gateway: apply [production]
13:57 <moritzm> installed libapache2-mod-auth-openidc bugfix updates from Bookworm point release [production]
13:54 <arnaudb> reimage pc1017 T378068 [production]
13:53 <arnaudb@cumin1002> START - Cookbook sre.hosts.reimage for host pc1017.eqiad.wmnet with OS bookworm [production]
13:52 <akosiaris@deploy2002> helmfile [staging] DONE helmfile.d/services/rest-gateway: apply [production]
13:52 <akosiaris@deploy2002> helmfile [staging] START helmfile.d/services/rest-gateway: apply [production]
13:44 <akosiaris@deploy2002> helmfile [staging] DONE helmfile.d/services/rest-gateway: apply [production]
13:44 <akosiaris@deploy2002> helmfile [staging] START helmfile.d/services/rest-gateway: apply [production]
13:42 <brouberol@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-test-k8s: apply [production]
13:42 <brouberol@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-test-k8s: apply [production]
13:41 <akosiaris@deploy2002> helmfile [staging] START helmfile.d/services/rest-gateway: apply [production]
13:39 <akosiaris@deploy2002> helmfile [staging] DONE helmfile.d/services/rest-gateway: apply [production]
13:34 <moritzm> imported jenkins 2.479.1 to thirdparty/ci for bullseye-wikimedia T379059 [production]
13:29 <akosiaris@deploy2002> helmfile [staging] START helmfile.d/services/rest-gateway: apply [production]
13:16 <arnaudb@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on pc1017.eqiad.wmnet with reason: T378068, host is not pooled [production]
13:16 <arnaudb@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on pc1017.eqiad.wmnet with reason: T378068, host is not pooled [production]
13:10 <cmooney@cumin1002> END (PASS) - Cookbook sre.netbox.update-extras (exit_code=0) rolling restart_daemons on A:netbox [production]
13:10 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti1042.eqiad.wmnet [production]
13:10 <cmooney@cumin1002> START - Cookbook sre.netbox.update-extras rolling restart_daemons on A:netbox [production]
13:09 <cmooney@cumin1002> END (PASS) - Cookbook sre.netbox.update-extras (exit_code=0) rolling restart_daemons on A:netbox-canary [production]
13:09 <cmooney@cumin1002> START - Cookbook sre.netbox.update-extras rolling restart_daemons on A:netbox-canary [production]
13:08 <moritzm> installing php7.4 security updates on remaining non-wikikube servers T378173 [production]
13:03 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host ganeti1042.eqiad.wmnet [production]
12:56 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti1041.eqiad.wmnet [production]
12:50 <kharlan@deploy2002> Finished scap sync-world: Backport for [[gerrit:1087424|Revert^2 "temp accounts: Enable temp account creation on second-round pilots" (T378336)]] (duration: 11m 46s) [production]
12:49 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host ganeti1041.eqiad.wmnet [production]
12:46 <kharlan@deploy2002> kharlan: Continuing with sync [production]
12:42 <kharlan@deploy2002> kharlan: Backport for [[gerrit:1087424|Revert^2 "temp accounts: Enable temp account creation on second-round pilots" (T378336)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
12:40 <fnegri@cumin1002> END (PASS) - Cookbook sre.wikireplicas.update-views (exit_code=0) [production]
12:39 <kharlan@deploy2002> Started scap sync-world: Backport for [[gerrit:1087424|Revert^2 "temp accounts: Enable temp account creation on second-round pilots" (T378336)]] [production]
12:35 <fnegri@cumin1002> START - Cookbook sre.wikireplicas.update-views [production]
12:35 <fnegri@cumin1002> END (FAIL) - Cookbook sre.wikireplicas.update-views (exit_code=93) [production]
12:35 <fnegri@cumin1002> START - Cookbook sre.wikireplicas.update-views [production]
12:34 <fnegri@cumin1002> END (FAIL) - Cookbook sre.wikireplicas.update-views (exit_code=93) [production]
12:34 <fnegri@cumin1002> START - Cookbook sre.wikireplicas.update-views [production]
12:33 <urbanecm> eswiki,x1: `delete from growthexperiments_link_recommendations where gelr_page=10598298;` (to verify updates are flowing in; T378983) [production]
12:33 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1013.eqiad.wmnet [production]
12:33 <urbanecm> mwmaint2002: kill all instances of refreshLinkRecommendation (T378983) [production]
12:32 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1013.eqiad.wmnet [production]
12:28 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1013.eqiad.wmnet [production]
12:23 <urbanecm@deploy2002> Finished scap sync-world: Backport for [[gerrit:1087407|CirrusSearch: Disable updating weighted tags via EventBus (T378983 T377150)]] (duration: 07m 39s) [production]
12:18 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 6:00:00 on db1125.eqiad.wmnet with reason: testing [production]
12:18 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 6:00:00 on db1125.eqiad.wmnet with reason: testing [production]
12:18 <marostegui@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 6:00:00 on db2230.codfw.wmnet with reason: testing [production]
12:17 <marostegui@cumin1002> START - Cookbook sre.hosts.downtime for 1 day, 6:00:00 on db2230.codfw.wmnet with reason: testing [production]