6001-6050 of 10000 results (106ms)
2024-03-26 ยง
18:47 <vriley@cumin1002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host dbprov1006.eqiad.wmnet with OS bullseye [production]
18:42 <cstone> civicrm upgraded from b8a84b22 to 143aa0bf [production]
18:36 <denisse@cumin2002> END (FAIL) - Cookbook sre.puppet.migrate-host (exit_code=99) for host alert2001.wikimedia.org [production]
18:36 <denisse@cumin2002> START - Cookbook sre.puppet.migrate-host for host alert2001.wikimedia.org [production]
18:34 <jhuneidi@deploy1002> rebuilt and synchronized wikiversions files: group0 wikis to 1.42.0-wmf.24 refs T360156 [production]
18:21 <hnowlan@deploy1002> helmfile [staging] DONE helmfile.d/services/mobileapps: apply [production]
18:21 <hnowlan@deploy1002> helmfile [staging] START helmfile.d/services/mobileapps: apply [production]
18:20 <hnowlan@deploy1002> helmfile [staging] DONE helmfile.d/services/mobileapps: apply [production]
18:20 <hnowlan@deploy1002> helmfile [staging] START helmfile.d/services/mobileapps: apply [production]
18:00 <pt1979@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host dbprov2005.codfw.wmnet with OS bullseye [production]
17:45 <hnowlan@deploy1002> helmfile [staging] DONE helmfile.d/services/mobileapps: apply [production]
17:44 <hnowlan@deploy1002> helmfile [staging] START helmfile.d/services/mobileapps: apply [production]
17:22 <vriley@cumin1002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=1) upgrade firmware for hosts ['dbprov1006'] [production]
17:22 <vriley@cumin1002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['dbprov1006'] [production]
17:22 <vriley@cumin1002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=1) upgrade firmware for hosts ['dbprov1006'] [production]
17:22 <vriley@cumin1002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['dbprov1006'] [production]
17:21 <vriley@cumin1002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=1) upgrade firmware for hosts ['dbprov1006'] [production]
17:21 <vriley@cumin1002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['dbprov1006'] [production]
17:13 <vriley@cumin1002> START - Cookbook sre.hosts.reimage for host dbprov1006.eqiad.wmnet with OS bullseye [production]
17:06 <jynus> add georgemikesell to wmf ldap group T358922 [production]
16:54 <jebe@deploy1002> Finished deploy [analytics/refinery@07a0290] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@07a0290a] (duration: 03m 38s) [production]
16:51 <jebe@deploy1002> Started deploy [analytics/refinery@07a0290] (hadoop-test): Regular analytics weekly train TEST [analytics/refinery@07a0290a] [production]
16:50 <jebe@deploy1002> Finished deploy [analytics/refinery@07a0290] (thin): Regular analytics weekly train THIN [analytics/refinery@07a0290a] (duration: 00m 04s) [production]
16:50 <jebe@deploy1002> Started deploy [analytics/refinery@07a0290] (thin): Regular analytics weekly train THIN [analytics/refinery@07a0290a] [production]
16:50 <jebe@deploy1002> Finished deploy [analytics/refinery@07a0290]: Regular analytics weekly train [analytics/refinery@07a0290a] (duration: 00m 14s) [production]
16:49 <jebe@deploy1002> Started deploy [analytics/refinery@07a0290]: Regular analytics weekly train [analytics/refinery@07a0290a] [production]
16:40 <pt1979@cumin2002> START - Cookbook sre.hosts.reimage for host dbprov2005.codfw.wmnet with OS bullseye [production]
16:39 <sukhe> restart pybal on lvs2013 and lvs2014 [production]
16:37 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2116 (re)pooling @ 100%: Post clone (src)', diff saved to https://phabricator.wikimedia.org/P58929 and previous config saved to /var/cache/conftool/dbconfig/20240326-163744-arnaudb.json [production]
16:22 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2116 (re)pooling @ 75%: Post clone (src)', diff saved to https://phabricator.wikimedia.org/P58928 and previous config saved to /var/cache/conftool/dbconfig/20240326-162238-arnaudb.json [production]
16:18 <denisse> Importing karma 0.119 to reprepro - T333615 [production]
16:12 <pt1979@cumin2002> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=93) for host dbprov2005.codfw.wmnet with OS bullseye [production]
16:09 <sukhe> restart pybal on lvs2013 [production]
16:07 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2116 (re)pooling @ 50%: Post clone (src)', diff saved to https://phabricator.wikimedia.org/P58927 and previous config saved to /var/cache/conftool/dbconfig/20240326-160733-arnaudb.json [production]
16:01 <jebe@deploy1002> Finished deploy [analytics/refinery@07a0290]: Regular analytics weekly train [analytics/refinery@07a0290a] (duration: 01m 18s) [production]
16:00 <jebe@deploy1002> Started deploy [analytics/refinery@07a0290]: Regular analytics weekly train [analytics/refinery@07a0290a] [production]
15:52 <arnaudb@cumin1002> dbctl commit (dc=all): 'db2116 (re)pooling @ 25%: Post clone (src)', diff saved to https://phabricator.wikimedia.org/P58926 and previous config saved to /var/cache/conftool/dbconfig/20240326-155227-arnaudb.json [production]
15:46 <jebe@deploy1002> Finished deploy [analytics/refinery@07a0290]: Regular analytics weekly train [analytics/refinery@07a0290a] (duration: 02m 26s) [production]
15:43 <jebe@deploy1002> Started deploy [analytics/refinery@07a0290]: Regular analytics weekly train [analytics/refinery@07a0290a] [production]
15:43 <jebe@deploy1002> Finished deploy [analytics/refinery@07a0290]: Regular analytics weekly train [analytics/refinery@07a0290a] (duration: 13m 05s) [production]
15:39 <brouberol@cumin1002> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) for hosts an-coord1002.eqiad.wmnet [production]
15:39 <brouberol@cumin1002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
15:39 <brouberol@cumin1002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: an-coord1002.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - brouberol@cumin1002" [production]
15:39 <pt1979@cumin2002> START - Cookbook sre.hosts.reimage for host dbprov2005.codfw.wmnet with OS bullseye [production]
15:37 <brouberol@cumin1002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: an-coord1002.eqiad.wmnet decommissioned, removing all IPs except the asset tag one - brouberol@cumin1002" [production]
15:35 <gmodena@deploy1002> helmfile [eqiad] DONE helmfile.d/services/eventstreams: apply [production]
15:34 <gmodena@deploy1002> helmfile [eqiad] START helmfile.d/services/eventstreams: apply [production]
15:33 <gmodena@deploy1002> helmfile [codfw] DONE helmfile.d/services/eventstreams: apply [production]
15:33 <gmodena@deploy1002> helmfile [codfw] START helmfile.d/services/eventstreams: apply [production]
15:32 <brouberol@cumin1002> START - Cookbook sre.dns.netbox [production]