2051-2100 of 10000 results (94ms)
2023-02-07 ยง
19:46 <pt1979@cumin2002> START - Cookbook sre.hosts.provision for host mw2423.mgmt.codfw.wmnet with reboot policy FORCED [production]
19:45 <pt1979@cumin2002> START - Cookbook sre.hosts.provision for host mw2422.mgmt.codfw.wmnet with reboot policy FORCED [production]
19:44 <pt1979@cumin2002> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host mw2423.mgmt.codfw.wmnet with reboot policy FORCED [production]
19:44 <pt1979@cumin2002> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host mw2422.mgmt.codfw.wmnet with reboot policy FORCED [production]
19:39 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mc2049.codfw.wmnet with OS bullseye [production]
19:33 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mc1049.eqiad.wmnet with OS bullseye [production]
19:23 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mc2049.codfw.wmnet with reason: host reimage [production]
19:20 <jiji@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mc2049.codfw.wmnet with reason: host reimage [production]
19:18 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mc1049.eqiad.wmnet with reason: host reimage [production]
19:15 <jiji@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mc1049.eqiad.wmnet with reason: host reimage [production]
19:04 <jiji@cumin1001> START - Cookbook sre.hosts.reimage for host mc1049.eqiad.wmnet with OS bullseye [production]
19:03 <jiji@cumin1001> START - Cookbook sre.hosts.reimage for host mc2049.codfw.wmnet with OS bullseye [production]
19:03 <pt1979@cumin2002> START - Cookbook sre.hosts.provision for host mw2423.mgmt.codfw.wmnet with reboot policy FORCED [production]
19:01 <pt1979@cumin2002> START - Cookbook sre.hosts.provision for host mw2422.mgmt.codfw.wmnet with reboot policy FORCED [production]
19:00 <pt1979@cumin2002> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
19:00 <pt1979@cumin2002> END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add mw2423,25,26,27 DNS - pt1979@cumin2002" [production]
19:00 <pt1979@cumin2002> START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.dns.netbox: Add mw2423,25,26,27 DNS - pt1979@cumin2002" [production]
18:57 <pt1979@cumin2002> START - Cookbook sre.dns.netbox [production]
18:53 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mc2048.codfw.wmnet with OS bullseye [production]
18:47 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mc1047.eqiad.wmnet with OS bullseye [production]
18:37 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mc2048.codfw.wmnet with reason: host reimage [production]
18:34 <jiji@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mc2048.codfw.wmnet with reason: host reimage [production]
18:32 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mc1047.eqiad.wmnet with reason: host reimage [production]
18:29 <jiji@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mc1047.eqiad.wmnet with reason: host reimage [production]
18:18 <jiji@cumin1001> START - Cookbook sre.hosts.reimage for host mc2048.codfw.wmnet with OS bullseye [production]
18:17 <jiji@cumin1001> START - Cookbook sre.hosts.reimage for host mc1047.eqiad.wmnet with OS bullseye [production]
18:02 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.remove-downtime (exit_code=0) for 13 hosts [production]
18:02 <bking@cumin2002> START - Cookbook sre.hosts.remove-downtime for 13 hosts [production]
17:55 <inflatador> bking@cumin1001 repooling elastic and wdqs hosts post-maintenance T327925 [production]
17:53 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mc2047.codfw.wmnet with OS bullseye [production]
17:51 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mc1046.eqiad.wmnet with OS bullseye [production]
17:40 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mc2047.codfw.wmnet with reason: host reimage [production]
17:37 <jiji@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mc2047.codfw.wmnet with reason: host reimage [production]
17:37 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mc1046.eqiad.wmnet with reason: host reimage [production]
17:34 <jiji@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mc1046.eqiad.wmnet with reason: host reimage [production]
17:22 <jiji@cumin1001> START - Cookbook sre.hosts.reimage for host mc1046.eqiad.wmnet with OS bullseye [production]
17:21 <jiji@cumin1001> START - Cookbook sre.hosts.reimage for host mc2047.codfw.wmnet with OS bullseye [production]
16:50 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mc2046.codfw.wmnet with OS bullseye [production]
16:48 <urbanecm@deploy1002> Finished scap: 58f4d877: Finalize mediawiki/page/change schema, produce at rc1.mediawiki.page_change (T308017), 854ff4ac: Finalize mediawiki/page/change schema at 1.0.0 (T308017) (duration: 07m 32s) [production]
16:46 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host mc1045.eqiad.wmnet with OS bullseye [production]
16:41 <urbanecm@deploy1002> Started scap: 58f4d877: Finalize mediawiki/page/change schema, produce at rc1.mediawiki.page_change (T308017), 854ff4ac: Finalize mediawiki/page/change schema at 1.0.0 (T308017) [production]
16:39 <marostegui@cumin1001> dbctl commit (dc=all): 'db1187 (re)pooling @ 100%: Repooling', diff saved to https://phabricator.wikimedia.org/P43765 and previous config saved to /var/cache/conftool/dbconfig/20230207-163902-root.json [production]
16:34 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mc2046.codfw.wmnet with reason: host reimage [production]
16:31 <jiji@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mc2046.codfw.wmnet with reason: host reimage [production]
16:31 <jiji@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on mc1045.eqiad.wmnet with reason: host reimage [production]
16:26 <jiji@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on mc1045.eqiad.wmnet with reason: host reimage [production]
16:23 <marostegui@cumin1001> dbctl commit (dc=all): 'db1187 (re)pooling @ 75%: Repooling', diff saved to https://phabricator.wikimedia.org/P43764 and previous config saved to /var/cache/conftool/dbconfig/20230207-162357-root.json [production]
16:18 <urbanecm@deploy1002> Finished scap: Backport for [[gerrit:886985|Restore mediawiki.page-undelete hook (T329064)]], [[gerrit:887346|Restore mediawiki.page-undelete hook (T329064)]] (duration: 17m 44s) [production]
16:15 <jiji@cumin1001> START - Cookbook sre.hosts.reimage for host mc2046.codfw.wmnet with OS bullseye [production]
16:14 <jiji@cumin1001> START - Cookbook sre.hosts.reimage for host mc1045.eqiad.wmnet with OS bullseye [production]