2051-2100 of 10000 results (43ms)
2024-08-16 ยง
15:31 <jhancock@cumin2002> START - Cookbook sre.hosts.provision for host logging-sd2003.mgmt.codfw.wmnet with reboot policy FORCED [production]
15:31 <jhancock@cumin2002> START - Cookbook sre.hosts.provision for host logging-sd2002.mgmt.codfw.wmnet with reboot policy FORCED [production]
15:31 <jhancock@cumin2002> START - Cookbook sre.hosts.provision for host logging-sd2001.mgmt.codfw.wmnet with reboot policy FORCED [production]
15:31 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.bootstrap_and_add [admin]
15:30 <jhancock@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
15:30 <jhancock@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: adding logging-sd2004 to codfw - jhancock@cumin2002" [production]
15:30 <jhancock@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: adding logging-sd2004 to codfw - jhancock@cumin2002" [production]
15:26 <jhancock@cumin2002> START - Cookbook sre.dns.netbox [production]
15:20 <andrew@cloudcumin1001> END (ERROR) - Cookbook wmcs.ceph.osd.bootstrap_and_add (exit_code=97) [admin]
15:19 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.bootstrap_and_add [admin]
15:15 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.ceph.osd.depool_and_destroy (exit_code=99) [admin]
15:15 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.depool_and_destroy [admin]
15:12 <hnowlan@deploy1003> helmfile [eqiad] DONE helmfile.d/services/changeprop-jobqueue: apply [production]
15:11 <hnowlan@deploy1003> helmfile [eqiad] START helmfile.d/services/changeprop-jobqueue: apply [production]
15:10 <hnowlan@deploy1003> helmfile [codfw] DONE helmfile.d/services/changeprop-jobqueue: apply [production]
15:10 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.ceph.osd.depool_and_destroy (exit_code=99) [admin]
15:10 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.depool_and_destroy [admin]
15:10 <hnowlan@deploy1003> helmfile [codfw] START helmfile.d/services/changeprop-jobqueue: apply [production]
15:09 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.osd.depool_and_destroy (exit_code=0) [admin]
15:09 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.depool_and_destroy [admin]
15:09 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.osd.depool_and_destroy (exit_code=0) [admin]
15:09 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.depool_and_destroy [admin]
15:09 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.ceph.osd.depool_and_destroy (exit_code=99) [admin]
15:09 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.depool_and_destroy [admin]
15:09 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.osd.depool_and_destroy (exit_code=0) [admin]
15:09 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.depool_and_destroy [admin]
15:09 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.osd.depool_and_destroy (exit_code=0) [admin]
15:09 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.depool_and_destroy [admin]
15:08 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.ceph.osd.depool_and_destroy (exit_code=99) [admin]
15:08 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.depool_and_destroy [admin]
15:08 <jhancock@cumin2002> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host ganeti2043 [production]
15:08 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.ceph.osd.depool_and_destroy (exit_code=99) [admin]
15:08 <jhancock@cumin2002> START - Cookbook sre.network.configure-switch-interfaces for host ganeti2043 [production]
15:08 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.depool_and_destroy [admin]
15:08 <andrew@cloudcumin1001> END (PASS) - Cookbook wmcs.ceph.osd.depool_and_destroy (exit_code=0) [admin]
15:08 <jhancock@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
15:08 <jhancock@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: adding ganeti2043 to codfw - jhancock@cumin2002" [production]
15:07 <jhancock@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: adding ganeti2043 to codfw - jhancock@cumin2002" [production]
15:07 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.depool_and_destroy [admin]
15:07 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.ceph.osd.depool_and_destroy (exit_code=99) [admin]
15:07 <andrew@cloudcumin1001> START - Cookbook wmcs.ceph.osd.depool_and_destroy [admin]
15:06 <isaranto@deploy1003> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-articlequality' for release 'main' . [production]
15:05 <isaranto@deploy1003> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-articlequality' for release 'main' . [production]
15:04 <jhancock@cumin2002> START - Cookbook sre.dns.netbox [production]
15:04 <jhancock@cumin2002> END (PASS) - Cookbook sre.network.configure-switch-interfaces (exit_code=0) for host ganeti2042 [production]
15:04 <jhancock@cumin2002> START - Cookbook sre.network.configure-switch-interfaces for host ganeti2042 [production]
15:04 <jhancock@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
15:03 <jhancock@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: adding ganeti2042 to codfw - jhancock@cumin2002" [production]
15:03 <jhancock@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: adding ganeti2042 to codfw - jhancock@cumin2002" [production]
15:02 <andrew@cloudcumin1001> END (FAIL) - Cookbook wmcs.ceph.osd.depool_and_destroy (exit_code=99) [admin]