151-200 of 10000 results (13ms)
2026-04-01 ยง
08:52 <ayounsi@cumin1003> START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker1368 [production]
08:52 <ayounsi@cumin1003> END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) wikikube-worker1368.eqiad.wmnet 202.48.64.10.in-addr.arpa 2.0.2.0.8.4.0.0.4.6.0.0.0.1.0.0.7.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors [production]
08:52 <ayounsi@cumin1003> START - Cookbook sre.dns.wipe-cache wikikube-worker1368.eqiad.wmnet 202.48.64.10.in-addr.arpa 2.0.2.0.8.4.0.0.4.6.0.0.0.1.0.0.7.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors [production]
08:52 <ayounsi@cumin1003> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
08:52 <ayounsi@cumin1003> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host wikikube-worker1368 - ayounsi@cumin1003" [production]
08:52 <ayounsi@cumin1003> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host wikikube-worker1368 - ayounsi@cumin1003" [production]
08:50 <fceratto@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1162', diff saved to https://phabricator.wikimedia.org/P90139 and previous config saved to /var/cache/conftool/dbconfig/20260401-085053-fceratto.json [production]
08:45 <javiermonton@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply [production]
08:45 <javiermonton@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply [production]
08:44 <moritzm> installing Apache security updates [production]
08:44 <btullis@cumin1003> END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host an-druid1003 [production]
08:44 <btullis@cumin1003> START - Cookbook sre.hosts.move-vlan for host an-druid1003 [production]
08:43 <btullis@cumin1003> START - Cookbook sre.hosts.reimage for host an-druid1003.eqiad.wmnet with OS bookworm [production]
08:42 <javiermonton@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply [production]
08:42 <javiermonton@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply [production]
08:40 <fceratto@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1162 (T419635)', diff saved to https://phabricator.wikimedia.org/P90138 and previous config saved to /var/cache/conftool/dbconfig/20260401-084047-fceratto.json [production]
08:38 <ayounsi@cumin1003> START - Cookbook sre.dns.netbox [production]
08:37 <fceratto@cumin1003> dbctl commit (dc=all): 'Depooling db1162 (T419635)', diff saved to https://phabricator.wikimedia.org/P90137 and previous config saved to /var/cache/conftool/dbconfig/20260401-083733-fceratto.json [production]
08:37 <fceratto@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1162.eqiad.wmnet with reason: Maintenance [production]
08:37 <fceratto@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1156 (T419635)', diff saved to https://phabricator.wikimedia.org/P90136 and previous config saved to /var/cache/conftool/dbconfig/20260401-083709-fceratto.json [production]
08:28 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host maps1014.eqiad.wmnet [production]
08:27 <fceratto@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1156', diff saved to https://phabricator.wikimedia.org/P90135 and previous config saved to /var/cache/conftool/dbconfig/20260401-082701-fceratto.json [production]
08:25 <kevinbazira@deploy1003> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'experimental' for release 'main' . [production]
08:24 <ayounsi@cumin1003> START - Cookbook sre.hosts.move-vlan for host wikikube-worker1368 [production]
08:23 <ayounsi@cumin1003> START - Cookbook sre.hosts.reimage for host wikikube-worker1368.eqiad.wmnet with OS trixie [production]
08:21 <jnuche@deploy1003> rebuilt and synchronized wikiversions files: group0 to 1.46.0-wmf.22 refs T420480 [production]
08:21 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host maps1014.eqiad.wmnet [production]
08:18 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host maps1013.eqiad.wmnet [production]
08:16 <fceratto@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1156', diff saved to https://phabricator.wikimedia.org/P90134 and previous config saved to /var/cache/conftool/dbconfig/20260401-081652-fceratto.json [production]
08:16 <ayounsi@cumin1003> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker1367.eqiad.wmnet with OS trixie [production]
08:15 <godog> extend cloudrabbit1* root with an additional 100G - T421054 [admin]
08:10 <jmm@cumin2002> START - Cookbook sre.hosts.reboot-single for host maps1013.eqiad.wmnet [production]
08:07 <moritzm> upgrading Envoy on the Puppet servers to 1.35.9 T419637 T410975 [production]
08:06 <fceratto@cumin1003> dbctl commit (dc=all): 'Repooling after maintenance db1156 (T419635)', diff saved to https://phabricator.wikimedia.org/P90132 and previous config saved to /var/cache/conftool/dbconfig/20260401-080644-fceratto.json [production]
08:03 <kevinbazira@deploy1003> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'experimental' for release 'main' . [production]
07:59 <ayounsi@cumin1003> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1367.eqiad.wmnet with reason: host reimage [production]
07:52 <ayounsi@cumin1003> START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1367.eqiad.wmnet with reason: host reimage [production]
07:47 <fceratto@cumin1003> dbctl commit (dc=all): 'Depooling db1156 (T419635)', diff saved to https://phabricator.wikimedia.org/P90130 and previous config saved to /var/cache/conftool/dbconfig/20260401-074704-fceratto.json [production]
07:46 <fceratto@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on an-redacteddb1001.eqiad.wmnet,clouddb[1014,1018].eqiad.wmnet,db1155.eqiad.wmnet with reason: Maintenance [production]
07:46 <fceratto@cumin1003> DONE (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1156.eqiad.wmnet with reason: Maintenance [production]
07:40 <ayounsi@cumin1003> END (PASS) - Cookbook sre.hosts.move-vlan (exit_code=0) for host wikikube-worker1367 [production]
07:40 <ayounsi@cumin1003> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker1367 [production]
07:39 <ayounsi@cumin1003> START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker1367 [production]
07:39 <ayounsi@cumin1003> END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) wikikube-worker1367.eqiad.wmnet 201.48.64.10.in-addr.arpa 1.0.2.0.8.4.0.0.4.6.0.0.0.1.0.0.7.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors [production]
07:39 <ayounsi@cumin1003> START - Cookbook sre.dns.wipe-cache wikikube-worker1367.eqiad.wmnet 201.48.64.10.in-addr.arpa 1.0.2.0.8.4.0.0.4.6.0.0.0.1.0.0.7.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa on all recursors [production]
07:39 <ayounsi@cumin1003> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
07:39 <ayounsi@cumin1003> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host wikikube-worker1367 - ayounsi@cumin1003" [production]
07:39 <ayounsi@cumin1003> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Update records for host wikikube-worker1367 - ayounsi@cumin1003" [production]
07:36 <javiermonton@deploy1003> helmfile [dse-k8s-eqiad] DONE helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply [production]
07:36 <javiermonton@deploy1003> helmfile [dse-k8s-eqiad] START helmfile.d/dse-k8s-services/mw-page-html-content-change-enrich: apply [production]