151-200 of 10000 results (46ms)
2023-03-10 §
09:58 <cmooney@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Remove netbox-generated DNS records which have been defined manually. - cmooney@cumin1001" [production]
09:57 <cmooney@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Remove netbox-generated DNS records which have been defined manually. - cmooney@cumin1001" [production]
09:55 <cmooney@cumin1001> START - Cookbook sre.dns.netbox [production]
02:09 <zabe@deploy2002> Finished scap: T331685 (duration: 07m 52s) [production]
02:02 <zabe@deploy2002> Started scap: T331685 [production]
02:01 <zabe@deploy2002> Finished scap: T331685 (duration: 07m 28s) [production]
02:00 <ejegg> SmashPig upgraded from c6775c60 to 3b84e4cb [production]
01:55 <ejegg> payments-wiki upgraded from 05a5e09a to 61c30a4f [production]
01:54 <zabe@deploy2002> Started scap: T331685 [production]
01:28 <pt1979@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host sretest2002.codfw.wmnet with OS bullseye [production]
00:33 <pt1979@cumin2002> START - Cookbook sre.hosts.reimage for host sretest2002.codfw.wmnet with OS bullseye [production]
2023-03-09 §
23:52 <ebernhardson@deploy2002> Finished deploy [airflow-dags/search@7b25fbf]: import_ttl: correct date formatting (duration: 00m 14s) [production]
23:52 <ebernhardson@deploy2002> Started deploy [airflow-dags/search@7b25fbf]: import_ttl: correct date formatting [production]
23:33 <ebernhardson@deploy2002> Finished deploy [airflow-dags/search@b122672]: import_ttl: replace HdfsSensor with URLSensor (duration: 00m 14s) [production]
23:32 <ebernhardson@deploy2002> Started deploy [airflow-dags/search@b122672]: import_ttl: replace HdfsSensor with URLSensor [production]
23:09 <jhathaway@deploy1002> helmfile [aux-k8s-eqiad] DONE helmfile.d/admin 'apply'. [production]
23:09 <jhathaway@deploy1002> helmfile [aux-k8s-eqiad] START helmfile.d/admin 'apply'. [production]
23:04 <jhathaway@deploy1002> helmfile [aux-k8s-eqiad] DONE helmfile.d/admin 'sync'. [production]
23:04 <jhathaway@deploy1002> helmfile [aux-k8s-eqiad] START helmfile.d/admin 'sync'. [production]
23:01 <sukhe> pool new dns hosts dns1003 and dns2003: T330670 [production]
22:53 <sukhe> run homer in cr*-{codfw,eqiad} for CR 896190 [production]
22:47 <pt1979@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host sretest2002.codfw.wmnet with OS bullseye [production]
22:43 <sukhe@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host dns2003.wikimedia.org with OS bullseye [production]
22:43 <sukhe@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - sukhe@cumin2002" [production]
22:41 <sukhe@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - sukhe@cumin2002" [production]
22:40 <bd808> Forced puppet run on cloudweb100[34] to apply quick fix for T331674 [production]
22:25 <cmooney@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
22:25 <cmooney@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add DNS records for new links to cloudsw1-b1-codfw - cmooney@cumin1001" [production]
22:24 <cmooney@cumin1001> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add DNS records for new links to cloudsw1-b1-codfw - cmooney@cumin1001" [production]
22:20 <sukhe@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host dns1003.wikimedia.org with OS bullseye [production]
22:20 <sukhe@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - sukhe@cumin2002" [production]
22:19 <sukhe@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on dns2003.wikimedia.org with reason: host reimage [production]
22:18 <cmooney@cumin1001> START - Cookbook sre.dns.netbox [production]
22:16 <sukhe@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on dns2003.wikimedia.org with reason: host reimage [production]
22:14 <sukhe@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - sukhe@cumin2002" [production]
22:03 <sukhe@cumin2002> START - Cookbook sre.hosts.reimage for host dns2003.wikimedia.org with OS bullseye [production]
22:02 <sukhe@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host dns2003.wikimedia.org with OS bullseye [production]
21:56 <sukhe@cumin2002> START - Cookbook sre.hosts.reimage for host dns2003.wikimedia.org with OS bullseye [production]
21:53 <sukhe@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on dns1003.wikimedia.org with reason: host reimage [production]
21:52 <pt1979@cumin2002> START - Cookbook sre.hosts.reimage for host sretest2002.codfw.wmnet with OS bullseye [production]
21:49 <sukhe@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on dns1003.wikimedia.org with reason: host reimage [production]
21:38 <TheresNoTime> close UTC late backport [production]
21:37 <samtar@deploy2002> Finished scap: Backport for [[gerrit:893542|Replace Cleopatra page with United_States to facilitate synthetic testing of T326829 (T326829)]] (duration: 10m 43s) [production]
21:35 <sukhe@cumin2002> START - Cookbook sre.hosts.reimage for host dns1003.wikimedia.org with OS bullseye [production]
21:35 <sukhe@cumin2002> END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host dns1003.wikimedia.org with OS bullseye [production]
21:28 <samtar@deploy2002> samtar and nray: Backport for [[gerrit:893542|Replace Cleopatra page with United_States to facilitate synthetic testing of T326829 (T326829)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug1001.eqiad.wmnet [production]
21:27 <samtar@deploy2002> Started scap: Backport for [[gerrit:893542|Replace Cleopatra page with United_States to facilitate synthetic testing of T326829 (T326829)]] [production]
21:24 <samtar@deploy2002> Finished scap: Backport for [[gerrit:895352|Unload RenameUser, now part of core: Part II of II]] (duration: 07m 38s) [production]
21:20 <cmooney@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
21:20 <cmooney@cumin1001> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Adjust and remove reverse DNS records after cloudsw1-b1-codfw migration. - cmooney@cumin1001" [production]