51-100 of 10000 results (43ms)
2026-09-17 ยง
15:48 <moritzm> installing libde265 security updates [production]
15:44 <btullis@cumin1004> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ceph-admin2001.codfw.wmnet with reason: host reimage [production]
15:40 <btullis@cumin1004> END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0) for new host ceph-admin1001.eqiad.wmnet [production]
15:40 <btullis@cumin1004> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ceph-admin1001.eqiad.wmnet with OS bookworm [production]
15:39 <cdobbins@cumin1004> conftool action : set/pooled=yes; selector: name=ncredir4003.* [production]
15:37 <btullis@cumin1004> START - Cookbook sre.hosts.downtime for 2:00:00 on ceph-admin2001.codfw.wmnet with reason: host reimage [production]
15:26 <cdobbins@cumin1004> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host ncredir4003.ulsfo.wmnet with OS trixie [production]
15:25 <aokoth@cumin1004> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host phab2003.codfw.wmnet with OS trixie [production]
15:23 <btullis@cumin1004> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ceph-admin1001.eqiad.wmnet with reason: host reimage [production]
15:19 <elukey@cumin1004> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host registry1005.eqiad.wmnet with OS trixie [production]
15:17 <btullis@cumin1004> START - Cookbook sre.hosts.downtime for 2:00:00 on ceph-admin1001.eqiad.wmnet with reason: host reimage [production]
15:16 <btullis@cumin1004> START - Cookbook sre.hosts.reimage for host ceph-admin2001.codfw.wmnet with OS bookworm [production]
15:16 <btullis@cumin1004> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM ceph-admin2001.codfw.wmnet - btullis@cumin1004" [production]
15:16 <btullis@cumin1004> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM ceph-admin2001.codfw.wmnet - btullis@cumin1004" [production]
15:15 <btullis@cumin1004> END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) ceph-admin2001.codfw.wmnet on all recursors [production]
15:15 <btullis@cumin1004> START - Cookbook sre.dns.wipe-cache ceph-admin2001.codfw.wmnet on all recursors [production]
15:15 <btullis@cumin1004> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
15:15 <btullis@cumin1004> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM ceph-admin2001.codfw.wmnet - btullis@cumin1004" [production]
15:15 <btullis@cumin1004> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM ceph-admin2001.codfw.wmnet - btullis@cumin1004" [production]
15:08 <aokoth@cumin1004> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on phab2003.codfw.wmnet with reason: host reimage [production]
15:06 <Msz2001> Deployed private code changes to Suggestedinvestigations [production]
15:05 <cdobbins@cumin1004> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on ncredir4003.ulsfo.wmnet with reason: host reimage [production]
15:05 <aokoth@cumin1004> START - Cookbook sre.hosts.downtime for 2:00:00 on phab2003.codfw.wmnet with reason: host reimage [production]
15:04 <btullis@cumin1004> START - Cookbook sre.hosts.reimage for host ceph-admin1001.eqiad.wmnet with OS bookworm [production]
15:04 <btullis@cumin1004> START - Cookbook sre.dns.netbox [production]
15:04 <btullis@cumin1004> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM ceph-admin1001.eqiad.wmnet - btullis@cumin1004" [production]
15:04 <btullis@cumin1004> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.ganeti.makevm: created new VM ceph-admin1001.eqiad.wmnet - btullis@cumin1004" [production]
15:04 <btullis@cumin1004> END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) ceph-admin1001.eqiad.wmnet on all recursors [production]
15:04 <btullis@cumin1004> START - Cookbook sre.dns.wipe-cache ceph-admin1001.eqiad.wmnet on all recursors [production]
15:04 <btullis@cumin1004> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
15:04 <btullis@cumin1004> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM ceph-admin1001.eqiad.wmnet - btullis@cumin1004" [production]
15:04 <btullis@cumin1004> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add records for VM ceph-admin1001.eqiad.wmnet - btullis@cumin1004" [production]
15:02 <elukey@cumin1004> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on registry1005.eqiad.wmnet with reason: host reimage [production]
15:01 <btullis@cumin1004> START - Cookbook sre.ganeti.makevm for new host ceph-admin2001.codfw.wmnet [production]
15:00 <urbanecm@deploy1003> Finished scap sync-world: Backport for [[gerrit:1342265|JsonSchemaBuilder: Cache the root schema in the process (T437588)]], [[gerrit:1342264|JsonSchemaBuilder: Cache the root schema in the process (T437588)]], [[gerrit:1342684|SI: Preserve the username filter when switching queues (T438308)]] (duration: 12m 34s) [production]
15:00 <btullis@cumin1004> START - Cookbook sre.dns.netbox [production]
15:00 <btullis@cumin1004> START - Cookbook sre.ganeti.makevm for new host ceph-admin1001.eqiad.wmnet [production]
14:58 <cdobbins@cumin1004> START - Cookbook sre.hosts.downtime for 2:00:00 on ncredir4003.ulsfo.wmnet with reason: host reimage [production]
14:58 <elukey@cumin1004> START - Cookbook sre.hosts.downtime for 2:00:00 on registry1005.eqiad.wmnet with reason: host reimage [production]
14:55 <urbanecm@deploy1003> mszwarc, urbanecm: Continuing with deployment [production]
14:51 <urbanecm@deploy1003> mszwarc, urbanecm: Backport for [[gerrit:1342265|JsonSchemaBuilder: Cache the root schema in the process (T437588)]], [[gerrit:1342264|JsonSchemaBuilder: Cache the root schema in the process (T437588)]], [[gerrit:1342684|SI: Preserve the username filter when switching queues (T438308)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
14:51 <aokoth@cumin1004> START - Cookbook sre.hosts.reimage for host phab2003.codfw.wmnet with OS trixie [production]
14:50 <aokoth@cumin1004> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on phab2003.codfw.wmnet with reason: Reimage [production]
14:47 <urbanecm@deploy1003> Started scap sync-world: Backport for [[gerrit:1342265|JsonSchemaBuilder: Cache the root schema in the process (T437588)]], [[gerrit:1342264|JsonSchemaBuilder: Cache the root schema in the process (T437588)]], [[gerrit:1342684|SI: Preserve the username filter when switching queues (T438308)]] [production]
14:42 <cscott@deploy1003> Finished scap sync-world: Backport for [[gerrit:1331830|Keep Balinese Palm Leaf variants enabled on wikisource (T436398)]], [[gerrit:1340216|Turn on variant conversion for PageAssessments (T328012)]], [[gerrit:1341949|Parsoid Read Views: Enable on 61 wikiquote wikis (T437917)]] (duration: 15m 31s) [production]
14:39 <elukey@cumin1004> START - Cookbook sre.hosts.reimage for host registry1005.eqiad.wmnet with OS trixie [production]
14:35 <cscott@deploy1003> ssastry, cscott: Continuing with deployment [production]
14:33 <cdobbins@cumin1004> START - Cookbook sre.hosts.reimage for host ncredir4003.ulsfo.wmnet with OS trixie [production]
14:32 <cscott@deploy1003> ssastry, cscott: Backport for [[gerrit:1331830|Keep Balinese Palm Leaf variants enabled on wikisource (T436398)]], [[gerrit:1340216|Turn on variant conversion for PageAssessments (T328012)]], [[gerrit:1341949|Parsoid Read Views: Enable on 61 wikiquote wikis (T437917)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there. [production]
14:26 <cscott@deploy1003> Started scap sync-world: Backport for [[gerrit:1331830|Keep Balinese Palm Leaf variants enabled on wikisource (T436398)]], [[gerrit:1340216|Turn on variant conversion for PageAssessments (T328012)]], [[gerrit:1341949|Parsoid Read Views: Enable on 61 wikiquote wikis (T437917)]] [production]