1451-1500 of 10000 results (109ms)
2024-06-28 ยง
12:45 <cgoubert@cumin1002> START - Cookbook sre.hosts.reimage for host wikikube-worker1027.eqiad.wmnet with OS bullseye [production]
12:44 <cgoubert@cumin1002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host wikikube-worker1027.eqiad.wmnet with OS bullseye [production]
12:44 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2204', diff saved to https://phabricator.wikimedia.org/P65546 and previous config saved to /var/cache/conftool/dbconfig/20240628-124419-marostegui.json [production]
12:44 <jclark@cumin1002> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host an-conf1004 [production]
12:44 <jclark@cumin1002> START - Cookbook sre.network.configure-switch-interfaces for host an-conf1004 [production]
12:21 <jclark@cumin1002> START - Cookbook sre.network.configure-switch-interfaces for host an-conf1004 [production]
12:18 <jclark@cumin1002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
12:18 <jclark@cumin1002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: added network and mgmt for an-conf1005,6 - jclark@cumin1002" [production]
12:17 <jclark@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: added network and mgmt for an-conf1005,6 - jclark@cumin1002" [production]
12:15 <jclark@cumin1002> START - Cookbook sre.dns.netbox [production]
12:14 <marostegui@cumin1002> dbctl commit (dc=all): 'Repooling after maintenance db2204 (T367856)', diff saved to https://phabricator.wikimedia.org/P65544 and previous config saved to /var/cache/conftool/dbconfig/20240628-121404-marostegui.json [production]
12:13 <cgoubert@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker1030.eqiad.wmnet with OS bullseye [production]
12:06 <cgoubert@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker1028.eqiad.wmnet with OS bullseye [production]
12:05 <cgoubert@cumin1002> START - Cookbook sre.hosts.reimage for host wikikube-worker1027.eqiad.wmnet with OS bullseye [production]
12:05 <cgoubert@cumin1002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host wikikube-worker1027.eqiad.wmnet with OS bullseye [production]
11:55 <cgoubert@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1030.eqiad.wmnet with reason: host reimage [production]
11:54 <cgoubert@cumin1002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host wikikube-worker1031.eqiad.wmnet with OS bullseye [production]
11:51 <cgoubert@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1030.eqiad.wmnet with reason: host reimage [production]
11:50 <Dreamy_Jazz> Finished run on `medium.dblist` [production]
11:47 <cgoubert@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1028.eqiad.wmnet with reason: host reimage [production]
11:45 <btullis@deploy1002> helmfile [eqiad] DONE helmfile.d/services/eventgate-analytics-external: apply [production]
11:45 <btullis@deploy1002> helmfile [eqiad] START helmfile.d/services/eventgate-analytics-external: apply [production]
11:45 <btullis@deploy1002> helmfile [codfw] DONE helmfile.d/services/eventgate-analytics-external: apply [production]
11:44 <cgoubert@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1028.eqiad.wmnet with reason: host reimage [production]
11:44 <cgoubert@cumin1002> START - Cookbook sre.hosts.reimage for host wikikube-worker1029.eqiad.wmnet with OS bullseye [production]
11:44 <btullis@deploy1002> helmfile [codfw] START helmfile.d/services/eventgate-analytics-external: apply [production]
11:44 <cgoubert@cumin1002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host wikikube-worker1029.eqiad.wmnet with OS bullseye [production]
11:38 <cgoubert@cumin1002> START - Cookbook sre.hosts.reimage for host wikikube-worker1030.eqiad.wmnet with OS bullseye [production]
11:38 <cgoubert@cumin1002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host wikikube-worker1030.eqiad.wmnet with OS bullseye [production]
11:35 <cgoubert@cumin1002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on wikikube-worker1031.eqiad.wmnet with reason: host reimage [production]
11:31 <cgoubert@cumin1002> START - Cookbook sre.hosts.downtime for 2:00:00 on wikikube-worker1031.eqiad.wmnet with reason: host reimage [production]
11:31 <cgoubert@cumin1002> START - Cookbook sre.hosts.reimage for host wikikube-worker1027.eqiad.wmnet with OS bullseye [production]
11:30 <cgoubert@cumin1002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker1027.eqiad.wmnet with OS bullseye [production]
11:29 <jnuche@deploy1002> Finished deploy [releng/jenkins-deploy@9b733de] (releasing): (no justification provided) (duration: 00m 44s) [production]
11:29 <cgoubert@cumin1002> START - Cookbook sre.hosts.reimage for host wikikube-worker1029.eqiad.wmnet with OS bullseye [production]
11:29 <cgoubert@cumin1002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host wikikube-worker1029.eqiad.wmnet with OS bullseye [production]
11:29 <jnuche@deploy1002> Started deploy [releng/jenkins-deploy@9b733de] (releasing): (no justification provided) [production]
11:26 <btullis@deploy1002> helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. [production]
11:24 <cgoubert@cumin1002> START - Cookbook sre.hosts.reimage for host wikikube-worker1028.eqiad.wmnet with OS bullseye [production]
11:23 <cgoubert@cumin1002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host wikikube-worker1028.eqiad.wmnet with OS bullseye [production]
11:18 <cgoubert@cumin1002> START - Cookbook sre.hosts.reimage for host wikikube-worker1031.eqiad.wmnet with OS bullseye [production]
11:18 <cgoubert@cumin1002> END (PASS) - Cookbook sre.dns.wipe-cache (exit_code=0) wikikube-worker1031.eqiad.wmnet on all recursors [production]
11:18 <cgoubert@cumin1002> START - Cookbook sre.dns.wipe-cache wikikube-worker1031.eqiad.wmnet on all recursors [production]
11:17 <cgoubert@cumin1002> END (PASS) - Cookbook sre.hosts.rename (exit_code=0) from mw1450 to wikikube-worker1031 [production]
11:17 <cgoubert@cumin1002> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host wikikube-worker1031 [production]
11:16 <cgoubert@cumin1002> START - Cookbook sre.network.configure-switch-interfaces for host wikikube-worker1031 [production]
11:16 <cgoubert@cumin1002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
11:16 <cgoubert@cumin1002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Renaming mw1450 to wikikube-worker1031 - cgoubert@cumin1002" [production]
11:16 <btullis@deploy1002> helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. [production]
11:15 <cgoubert@cumin1002> START - Cookbook sre.hosts.reimage for host wikikube-worker1028.eqiad.wmnet with OS bullseye [production]