901-950 of 10000 results (74ms)
2022-11-22 ยง
13:04 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1174.eqiad.wmnet with reason: Maintenance [production]
13:04 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1171.eqiad.wmnet with reason: Maintenance [production]
13:04 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1171.eqiad.wmnet with reason: Maintenance [production]
13:04 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1170:3317 (T322618)', diff saved to https://phabricator.wikimedia.org/P40572 and previous config saved to /var/cache/conftool/dbconfig/20221122-130403-ladsgroup.json [production]
13:03 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1119', diff saved to https://phabricator.wikimedia.org/P40571 and previous config saved to /var/cache/conftool/dbconfig/20221122-130325-marostegui.json [production]
12:53 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2159', diff saved to https://phabricator.wikimedia.org/P40570 and previous config saved to /var/cache/conftool/dbconfig/20221122-125333-ladsgroup.json [production]
12:49 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1200', diff saved to https://phabricator.wikimedia.org/P40569 and previous config saved to /var/cache/conftool/dbconfig/20221122-124941-marostegui.json [production]
12:49 <jnuche@deploy1002> Finished scap: testing k8s deploys (duration: 06m 20s) [production]
12:48 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1170:3317', diff saved to https://phabricator.wikimedia.org/P40568 and previous config saved to /var/cache/conftool/dbconfig/20221122-124856-ladsgroup.json [production]
12:48 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1119 (T321130)', diff saved to https://phabricator.wikimedia.org/P40567 and previous config saved to /var/cache/conftool/dbconfig/20221122-124818-marostegui.json [production]
12:43 <aborrero@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cloudvirt1048.eqiad.wmnet with reason: host reimage [production]
12:42 <jnuche@deploy1002> Started scap: testing k8s deploys [production]
12:40 <aborrero@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on cloudvirt1048.eqiad.wmnet with reason: host reimage [production]
12:38 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2159', diff saved to https://phabricator.wikimedia.org/P40565 and previous config saved to /var/cache/conftool/dbconfig/20221122-123827-ladsgroup.json [production]
12:37 <jnuche@deploy1002> Installation of scap version "4.29.1" completed for 559 hosts [production]
12:36 <jnuche@deploy1002> Installing scap version "4.29.1" for 559 hosts [production]
12:35 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1119 (T321130)', diff saved to https://phabricator.wikimedia.org/P40564 and previous config saved to /var/cache/conftool/dbconfig/20221122-123505-marostegui.json [production]
12:35 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1119.eqiad.wmnet with reason: Maintenance [production]
12:34 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 5:00:00 on db1119.eqiad.wmnet with reason: Maintenance [production]
12:34 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1118 (T321130)', diff saved to https://phabricator.wikimedia.org/P40563 and previous config saved to /var/cache/conftool/dbconfig/20221122-123444-marostegui.json [production]
12:34 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1200', diff saved to https://phabricator.wikimedia.org/P40562 and previous config saved to /var/cache/conftool/dbconfig/20221122-123435-marostegui.json [production]
12:33 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1170:3317', diff saved to https://phabricator.wikimedia.org/P40561 and previous config saved to /var/cache/conftool/dbconfig/20221122-123350-ladsgroup.json [production]
12:25 <aborrero@cumin1001> START - Cookbook sre.hosts.reimage for host cloudvirt1048.eqiad.wmnet with OS bullseye [production]
12:23 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2159 (T322618)', diff saved to https://phabricator.wikimedia.org/P40560 and previous config saved to /var/cache/conftool/dbconfig/20221122-122320-ladsgroup.json [production]
12:21 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2159 (T322618)', diff saved to https://phabricator.wikimedia.org/P40559 and previous config saved to /var/cache/conftool/dbconfig/20221122-122103-ladsgroup.json [production]
12:20 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db2095.codfw.wmnet with reason: Maintenance [production]
12:20 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 12:00:00 on db2095.codfw.wmnet with reason: Maintenance [production]
12:20 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2159.codfw.wmnet with reason: Maintenance [production]
12:20 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db2159.codfw.wmnet with reason: Maintenance [production]
12:20 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2150 (T322618)', diff saved to https://phabricator.wikimedia.org/P40558 and previous config saved to /var/cache/conftool/dbconfig/20221122-122025-ladsgroup.json [production]
12:19 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1118', diff saved to https://phabricator.wikimedia.org/P40557 and previous config saved to /var/cache/conftool/dbconfig/20221122-121938-marostegui.json [production]
12:19 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1200 (T321126)', diff saved to https://phabricator.wikimedia.org/P40556 and previous config saved to /var/cache/conftool/dbconfig/20221122-121928-marostegui.json [production]
12:18 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1170:3317 (T322618)', diff saved to https://phabricator.wikimedia.org/P40555 and previous config saved to /var/cache/conftool/dbconfig/20221122-121843-ladsgroup.json [production]
12:16 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1200 (T321126)', diff saved to https://phabricator.wikimedia.org/P40554 and previous config saved to /var/cache/conftool/dbconfig/20221122-121657-marostegui.json [production]
12:16 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db1200.eqiad.wmnet with reason: Maintenance [production]
12:16 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 5:00:00 on db1200.eqiad.wmnet with reason: Maintenance [production]
12:16 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1161 (T321126)', diff saved to https://phabricator.wikimedia.org/P40553 and previous config saved to /var/cache/conftool/dbconfig/20221122-121647-marostegui.json [production]
12:16 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1170:3317 (T322618)', diff saved to https://phabricator.wikimedia.org/P40552 and previous config saved to /var/cache/conftool/dbconfig/20221122-121633-ladsgroup.json [production]
12:16 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1170.eqiad.wmnet with reason: Maintenance [production]
12:16 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1170.eqiad.wmnet with reason: Maintenance [production]
12:16 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1158 (T322618)', diff saved to https://phabricator.wikimedia.org/P40551 and previous config saved to /var/cache/conftool/dbconfig/20221122-121612-ladsgroup.json [production]
12:14 <hnowlan@deploy1002> helmfile [staging] DONE helmfile.d/services/thumbor: sync [production]
12:14 <hnowlan@deploy1002> helmfile [staging] START helmfile.d/services/thumbor: sync [production]
12:10 <hnowlan@deploy1002> helmfile [staging] DONE helmfile.d/services/thumbor: sync [production]
12:10 <hnowlan@deploy1002> helmfile [staging] START helmfile.d/services/thumbor: sync [production]
12:05 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2150', diff saved to https://phabricator.wikimedia.org/P40550 and previous config saved to /var/cache/conftool/dbconfig/20221122-120519-ladsgroup.json [production]
12:04 <jmm@cumin2002> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host ganeti1031.eqiad.wmnet [production]
12:04 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1118', diff saved to https://phabricator.wikimedia.org/P40549 and previous config saved to /var/cache/conftool/dbconfig/20221122-120431-marostegui.json [production]
12:01 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1161', diff saved to https://phabricator.wikimedia.org/P40548 and previous config saved to /var/cache/conftool/dbconfig/20221122-120140-marostegui.json [production]
12:01 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1158', diff saved to https://phabricator.wikimedia.org/P40547 and previous config saved to /var/cache/conftool/dbconfig/20221122-120106-ladsgroup.json [production]