3101-3150 of 10000 results (120ms)
2024-11-04 §
08:06 <brouberol@deploy2002> helmfile [dse-k8s-eqiad] DONE helmfile.d/admin 'apply'. [production]
08:05 <brouberol@deploy2002> helmfile [dse-k8s-eqiad] START helmfile.d/admin 'apply'. [production]
08:03 <jmm@cumin2002> START - Cookbook sre.dns.netbox [production]
07:59 <jmm@cumin2002> START - Cookbook sre.hosts.decommission for hosts ganeti2013.codfw.wmnet [production]
2024-11-02 §
15:48 <lucaswerkmeister-wmde@deploy2002> Finished scap sync-world: Backport for [[gerrit:1085922|Remove 'mainpage' from $wgForceUIMsgAsContentMsg for Wikidata (T184386)]] (duration: 12m 09s) [production]
15:44 <lucaswerkmeister-wmde@deploy2002> lucaswerkmeister-wmde, ladsgroup: Continuing with sync [production]
15:38 <lucaswerkmeister-wmde@deploy2002> lucaswerkmeister-wmde, ladsgroup: Backport for [[gerrit:1085922|Remove 'mainpage' from $wgForceUIMsgAsContentMsg for Wikidata (T184386)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug) [production]
15:36 <lucaswerkmeister-wmde@deploy2002> Started scap sync-world: Backport for [[gerrit:1085922|Remove 'mainpage' from $wgForceUIMsgAsContentMsg for Wikidata (T184386)]] [production]
15:26 <reedy@deploy2002> Finished scap sync-world: use statemnts (duration: 07m 13s) [production]
15:19 <reedy@deploy2002> Started scap sync-world: use statemnts [production]
15:13 <reedy@deploy2002> Synchronized wmf-config/: Comment updates (duration: 07m 31s) [production]
2024-11-01 §
20:27 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host an-presto1016.eqiad.wmnet with OS bullseye [production]
19:47 <inflatador> bking@an-presto[1016:1020].eqiad.wmnet temporarily install perccli to check disk status without requiring reboot T374924 [production]
19:34 <bking@cumin2002> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on an-presto1016.eqiad.wmnet with reason: host reimage [production]
19:31 <bking@cumin2002> START - Cookbook sre.hosts.downtime for 2:00:00 on an-presto1016.eqiad.wmnet with reason: host reimage [production]
19:16 <bking@cumin2002> START - Cookbook sre.hosts.reimage for host an-presto1016.eqiad.wmnet with OS bullseye [production]
19:12 <bking@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['an-presto1017.eqiad.wmnet'] [production]
19:07 <bking@cumin2002> END (PASS) - Cookbook sre.hardware.upgrade-firmware (exit_code=0) upgrade firmware for hosts ['an-presto1016.eqiad.wmnet'] [production]
19:02 <bking@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['an-presto1017.eqiad.wmnet'] [production]
18:56 <bking@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['an-presto1016.eqiad.wmnet'] [production]
18:56 <bking@cumin2002> END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts ['an-presto1017.eqiad.wmnet'] [production]
18:56 <bking@cumin2002> START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['an-presto1017.eqiad.wmnet'] [production]
18:51 <jclark@cumin1002> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host ganeti1041.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
18:51 <jclark@cumin1002> START - Cookbook sre.hosts.provision for host ganeti1041.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
18:51 <vriley@cumin1002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti1052.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
18:47 <jclark@cumin1002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti1051.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
18:46 <jclark@cumin1002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti1050.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
18:46 <vriley@cumin1002> START - Cookbook sre.hosts.provision for host ganeti1052.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
18:46 <jclark@cumin1002> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host ganeti1041.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
18:46 <jclark@cumin1002> START - Cookbook sre.hosts.provision for host ganeti1041.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
18:44 <jclark@cumin1002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti1049.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
18:44 <jclark@cumin1002> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host ganeti1044.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
18:44 <jclark@cumin1002> START - Cookbook sre.hosts.provision for host ganeti1044.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
18:43 <jclark@cumin1002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti1048.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
18:42 <vriley@cumin1002> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host ganeti1044.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
18:42 <vriley@cumin1002> START - Cookbook sre.hosts.provision for host ganeti1044.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
18:41 <jclark@cumin1002> START - Cookbook sre.hosts.provision for host ganeti1051.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
18:41 <jclark@cumin1002> START - Cookbook sre.hosts.provision for host ganeti1050.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
18:40 <jclark@cumin1002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti1046.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
18:40 <jclark@cumin1002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti1047.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
18:39 <jclark@cumin1002> START - Cookbook sre.hosts.provision for host ganeti1049.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
18:39 <vriley@cumin1002> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host ganeti1044.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
18:39 <vriley@cumin1002> START - Cookbook sre.hosts.provision for host ganeti1044.mgmt.eqiad.wmnet with chassis set policy FORCE_RESTART [production]
18:38 <jclark@cumin1002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti1045.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
18:38 <jclark@cumin1002> START - Cookbook sre.hosts.provision for host ganeti1048.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
18:35 <jclark@cumin1002> END (FAIL) - Cookbook sre.hosts.provision (exit_code=99) for host ganeti1041.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
18:35 <jclark@cumin1002> START - Cookbook sre.hosts.provision for host ganeti1046.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
18:35 <jclark@cumin1002> START - Cookbook sre.hosts.provision for host ganeti1047.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
18:35 <jclark@cumin1002> START - Cookbook sre.hosts.provision for host ganeti1041.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]
18:34 <jclark@cumin1002> END (PASS) - Cookbook sre.hosts.provision (exit_code=0) for host ganeti1043.mgmt.eqiad.wmnet with chassis set policy GRACEFUL_RESTART [production]