9401-9450 of 10000 results (15ms)
2025-04-11 §
15:19 <cgoubert@cumin1002> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) check for host wikikube-worker2142.codfw.wmnet [production]
15:19 <cgoubert@cumin1002> START - Cookbook sre.k8s.pool-depool-node check for host wikikube-worker2142.codfw.wmnet [production]
2025-04-10 §
06:45 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool pc2 T391454', diff saved to https://phabricator.wikimedia.org/P74822 and previous config saved to /var/cache/conftool/dbconfig/20250410-064511-marostegui.json [production]
2025-04-09 §
09:36 <cgoubert@cumin1002> END (FAIL) - Cookbook sre.k8s.pool-depool-node (exit_code=99) check for host wikikube-worker2142.codfw.wmnet [production]
09:36 <cgoubert@cumin1002> START - Cookbook sre.k8s.pool-depool-node check for host wikikube-worker2142.codfw.wmnet [production]
05:50 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool ms1 T391317', diff saved to https://phabricator.wikimedia.org/P74800 and previous config saved to /var/cache/conftool/dbconfig/20250409-055028-marostegui.json [production]
2025-04-08 §
22:02 <ryankemper> T388610 Elasticsearch->Opensearch row a data node migration ongoing [production]
11:30 <cgoubert@cumin1002> END (FAIL) - Cookbook sre.k8s.pool-depool-node (exit_code=99) depool for host wikikube-worker2142.codfw.wmnet [production]
11:27 <cgoubert@cumin1002> START - Cookbook sre.k8s.pool-depool-node depool for host wikikube-worker2142.codfw.wmnet [production]
08:12 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool ms1 T391317', diff saved to https://phabricator.wikimedia.org/P74694 and previous config saved to /var/cache/conftool/dbconfig/20250408-081224-marostegui.json [production]
06:42 <marostegui@cumin1002> dbctl commit (dc=all): 'Depool ms2 T391317', diff saved to https://phabricator.wikimedia.org/P74692 and previous config saved to /var/cache/conftool/dbconfig/20250408-064250-marostegui.json [production]
2025-04-07 §
19:19 <bking@cumin2002> END (PASS) - Cookbook sre.elasticsearch.ban (exit_code=0) Banning hosts: elastic1096* for ban node to stop high rejection rates - bking@cumin2002 [production]
19:19 <bking@cumin2002> START - Cookbook sre.elasticsearch.ban Banning hosts: elastic1096* for ban node to stop high rejection rates - bking@cumin2002 [production]
13:32 <sukhe> depool cp4037: reverting to ATS 9.2.9 [production]
2025-04-03 §
16:22 <hnowlan> decommissioning all but 1 eqiad jobrunner node in confctl [production]
16:06 <hnowlan@cumin1002> END (PASS) - Cookbook sre.k8s.pool-depool-node (exit_code=0) pool for host wikikube-worker[1166-1168].eqiad.wmnet [production]
16:06 <hnowlan@cumin1002> START - Cookbook sre.k8s.pool-depool-node pool for host wikikube-worker[1166-1168].eqiad.wmnet [production]
14:42 <bking@cumin2002> END (PASS) - Cookbook sre.elasticsearch.ban (exit_code=0) Banning hosts: elastic2056* for ban node before reimaging - bking@cumin2002 - T388610 [production]
14:42 <bking@cumin2002> START - Cookbook sre.elasticsearch.ban Banning hosts: elastic2056* for ban node before reimaging - bking@cumin2002 - T388610 [production]
14:42 <bking@cumin2002> END (FAIL) - Cookbook sre.elasticsearch.ban (exit_code=99) Banning hosts: elastic2056 for ban node before reimaging - bking@cumin2002 - T388610 [production]
14:42 <bking@cumin2002> START - Cookbook sre.elasticsearch.ban Banning hosts: elastic2056 for ban node before reimaging - bking@cumin2002 - T388610 [production]
09:08 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti3006.esams.wmnet [production]
08:42 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti3006.esams.wmnet [production]
08:29 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti3005.esams.wmnet [production]
08:05 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti3005.esams.wmnet [production]
07:47 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti3008.esams.wmnet [production]
07:28 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti3008.esams.wmnet [production]
07:08 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti3007.esams.wmnet [production]
06:54 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti3007.esams.wmnet [production]
2025-04-02 §
14:15 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1042.eqiad.wmnet [production]
14:13 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1042.eqiad.wmnet [production]
13:43 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1041.eqiad.wmnet [production]
13:40 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1041.eqiad.wmnet [production]
13:37 <akosiaris> depool cp3066 for debugging T390854 [production]
12:06 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1040.eqiad.wmnet [production]
12:04 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1040.eqiad.wmnet [production]
10:21 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1039.eqiad.wmnet [production]
10:19 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1039.eqiad.wmnet [production]
10:18 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti6004.drmrs.wmnet [production]
09:59 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti6004.drmrs.wmnet [production]
09:59 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti6003.drmrs.wmnet [production]
09:40 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti6003.drmrs.wmnet [production]
09:14 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti6002.drmrs.wmnet [production]
08:56 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti6002.drmrs.wmnet [production]
08:56 <jmm@cumin2002> END (FAIL) - Cookbook sre.ganeti.drain-node (exit_code=99) for draining ganeti node ganeti6001.drmrs.wmnet [production]
08:56 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti6001.drmrs.wmnet [production]
08:50 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti1036.eqiad.wmnet [production]
08:48 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti1036.eqiad.wmnet [production]
08:47 <jmm@cumin2002> END (PASS) - Cookbook sre.ganeti.drain-node (exit_code=0) for draining ganeti node ganeti6001.drmrs.wmnet [production]
08:32 <jmm@cumin2002> START - Cookbook sre.ganeti.drain-node for draining ganeti node ganeti6001.drmrs.wmnet [production]