1-50 of 10000 results (69ms)
2025-07-04 ยง
21:39 <krinkle@deploy1003> Finished scap sync-world: Backport for [[gerrit:1166438|beta: Change loginwiki/metawiki/auth canonical to beta.wmcloud.org (T289318)]] (duration: 18m 12s) [production]
21:33 <krinkle@deploy1003> krinkle: Continuing with sync [production]
21:23 <krinkle@deploy1003> krinkle: Backport for [[gerrit:1166438|beta: Change loginwiki/metawiki/auth canonical to beta.wmcloud.org (T289318)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
21:21 <krinkle@deploy1003> Started scap sync-world: Backport for [[gerrit:1166438|beta: Change loginwiki/metawiki/auth canonical to beta.wmcloud.org (T289318)]] [production]
20:32 <krinkle@deploy1003> Finished scap sync-world: Backport for [[gerrit:1165989|beta: Include allowance for wmcloud.org in wgGraphAllowedDomains (T289318)]], [[gerrit:1165999|beta: Change Beta wikidata canonical to beta.wmcloud.org (T289318)]] (duration: 94m 52s) [production]
20:26 <krinkle@deploy1003> krinkle: Continuing with sync [production]
18:59 <krinkle@deploy1003> krinkle: Backport for [[gerrit:1165989|beta: Include allowance for wmcloud.org in wgGraphAllowedDomains (T289318)]], [[gerrit:1165999|beta: Change Beta wikidata canonical to beta.wmcloud.org (T289318)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
18:57 <krinkle@deploy1003> Started scap sync-world: Backport for [[gerrit:1165989|beta: Include allowance for wmcloud.org in wgGraphAllowedDomains (T289318)]], [[gerrit:1165999|beta: Change Beta wikidata canonical to beta.wmcloud.org (T289318)]] [production]
15:14 <vgutierrez> fetch haproxy 2.8.15 on thirdparty/haproxy28 component for bullseye-wikimedia (apt.wm.o) [production]
14:46 <elukey@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host cp2043.codfw.wmnet with OS bullseye [production]
14:40 <stevemunene@cumin1002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host an-worker1179.eqiad.wmnet with OS bullseye [production]
14:36 <elukey@cumin2002> START - Cookbook sre.hosts.reimage for host cp2043.codfw.wmnet with OS bullseye [production]
14:29 <elukey@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host sretest2006.codfw.wmnet with OS bookworm [production]
14:20 <vgutierrez> repooling cp7006 [production]
14:20 <elukey@cumin2002> START - Cookbook sre.hosts.reimage for host sretest2006.codfw.wmnet with OS bookworm [production]
14:12 <vgutierrez> depooling cp7006 for testing purposes [production]
14:09 <elukey@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host sretest2006.codfw.wmnet with OS bookworm [production]
14:06 <stevemunene@cumin1002> START - Cookbook sre.hosts.reimage for host an-worker1179.eqiad.wmnet with OS bullseye [production]
14:01 <elukey@cumin2002> START - Cookbook sre.hosts.reimage for host sretest2006.codfw.wmnet with OS bookworm [production]
13:15 <elukey@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host sretest2006.codfw.wmnet with OS bookworm [production]
13:08 <elukey@cumin2002> START - Cookbook sre.hosts.reimage for host sretest2006.codfw.wmnet with OS bookworm [production]
12:59 <elukey@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host sretest2006.codfw.wmnet with OS bookworm [production]
12:51 <elukey@cumin2002> START - Cookbook sre.hosts.reimage for host sretest2006.codfw.wmnet with OS bookworm [production]
12:31 <vgutierrez> repool cp7006 [production]
12:31 <vgutierrez@cumin1002> END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for cp7006.magru.wmnet [production]
12:31 <vgutierrez@cumin1002> START - Cookbook sre.hosts.remove-downtime for cp7006.magru.wmnet [production]
12:11 <mfossati@deploy1003> Finished deploy [airflow-dags/platform_eng@38ba3ec]: bump section topics to v1.8.0 (duration: 00m 49s) [production]
12:11 <mfossati@deploy1003> Started deploy [airflow-dags/platform_eng@38ba3ec]: bump section topics to v1.8.0 [production]
11:08 <cmooney@cumin1003> END (PASS) - Cookbook sre.deploy.python-code (exit_code=0) homer to cumin2002.codfw.wmnet,cumin[1002-1003].eqiad.wmnet with reason: Release v10.0.2 with ibgp function in plugin - cmooney@cumin1003 [production]
11:05 <cmooney@cumin1003> START - Cookbook sre.deploy.python-code homer to cumin2002.codfw.wmnet,cumin[1002-1003].eqiad.wmnet with reason: Release v10.0.2 with ibgp function in plugin - cmooney@cumin1003 [production]
10:56 <marostegui@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 0:30:00 on 32 hosts with reason: maintenance [production]
10:51 <elukey@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host sretest2006.codfw.wmnet with OS bookworm [production]
10:43 <marostegui@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1:00:00 on db[2203,2212].codfw.wmnet with reason: Maintenance [production]
10:41 <elukey@cumin2002> START - Cookbook sre.hosts.reimage for host sretest2006.codfw.wmnet with OS bookworm [production]
10:27 <cgoubert@deploy1003> Unlocked for deployment [ALL REPOSITORIES]: Dragonfly supernodes reboot (duration: 09m 07s) [production]
10:26 <cgoubert@cumin1003> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host dragonfly-supernode1001.eqiad.wmnet [production]
10:23 <cgoubert@cumin1003> START - Cookbook sre.hosts.reboot-single for host dragonfly-supernode1001.eqiad.wmnet [production]
10:22 <cgoubert@cumin1003> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host dragonfly-supernode2001.codfw.wmnet [production]
10:18 <cgoubert@cumin1003> START - Cookbook sre.hosts.reboot-single for host dragonfly-supernode2001.codfw.wmnet [production]
10:18 <cgoubert@deploy1003> Locking from deployment [ALL REPOSITORIES]: Dragonfly supernodes reboot [production]
10:13 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/airflow-ml: apply [production]
10:13 <brouberol@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/airflow-ml: apply [production]
10:01 <jynus@cumin1002> DONE (PASS) - Cookbook sre.puppet.renew-cert (exit_code=0) for backupmon1001.eqiad.wmnet: Renew puppet certificate - jynus@cumin1002 [production]
09:16 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.changedisk (exit_code=0) for changing disk type of install6002.wikimedia.org to drbd [production]
09:07 <jynus@cumin1002> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on backupmon1001.eqiad.wmnet with reason: Maintenance and reboot [production]
08:59 <jmm@cumin2002> START - Cookbook sre.ganeti.changedisk for changing disk type of install6002.wikimedia.org to drbd [production]
08:58 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.changedisk (exit_code=0) for changing disk type of ncredir6001.drmrs.wmnet to drbd [production]
08:48 <mvernon@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ms-be2077.codfw.wmnet [production]
08:48 <jmm@cumin2002> START - Cookbook sre.ganeti.changedisk for changing disk type of ncredir6001.drmrs.wmnet to drbd [production]
08:37 <mvernon@cumin2002> START - Cookbook sre.hosts.reboot-single for host ms-be2077.codfw.wmnet [production]