2101-2150 of 10000 results (81ms)
2023-04-12 ยง
15:30 <hnowlan@deploy2002> helmfile [staging] START helmfile.d/services/rest-gateway: apply [production]
15:25 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2108 (T333332)', diff saved to https://phabricator.wikimedia.org/P46569 and previous config saved to /var/cache/conftool/dbconfig/20230412-152553-ladsgroup.json [production]
15:23 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2108 (T333332)', diff saved to https://phabricator.wikimedia.org/P46568 and previous config saved to /var/cache/conftool/dbconfig/20230412-152320-ladsgroup.json [production]
15:23 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db2108.codfw.wmnet with reason: Maintenance [production]
15:23 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 8:00:00 on db2108.codfw.wmnet with reason: Maintenance [production]
15:22 <sukhe@cumin2002> START - Cookbook sre.hosts.reimage for host lvs2010.codfw.wmnet with OS bullseye [production]
15:22 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db2100.codfw.wmnet with reason: Maintenance [production]
15:22 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 8:00:00 on db2100.codfw.wmnet with reason: Maintenance [production]
15:21 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db2098.codfw.wmnet with reason: Maintenance [production]
15:21 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 8:00:00 on db2098.codfw.wmnet with reason: Maintenance [production]
15:21 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2155', diff saved to https://phabricator.wikimedia.org/P46567 and previous config saved to /var/cache/conftool/dbconfig/20230412-152120-ladsgroup.json [production]
15:21 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on dbstore1003.eqiad.wmnet with reason: Maintenance [production]
15:21 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 8:00:00 on dbstore1003.eqiad.wmnet with reason: Maintenance [production]
15:21 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1202 (T333332)', diff saved to https://phabricator.wikimedia.org/P46566 and previous config saved to /var/cache/conftool/dbconfig/20230412-152104-ladsgroup.json [production]
15:14 <otto@deploy2002> helmfile [staging] DONE helmfile.d/services/eventgate-main: apply [production]
15:14 <otto@deploy2002> helmfile [staging] START helmfile.d/services/eventgate-main: apply [production]
15:06 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2155', diff saved to https://phabricator.wikimedia.org/P46565 and previous config saved to /var/cache/conftool/dbconfig/20230412-150614-ladsgroup.json [production]
15:05 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1202', diff saved to https://phabricator.wikimedia.org/P46564 and previous config saved to /var/cache/conftool/dbconfig/20230412-150557-ladsgroup.json [production]
15:05 <kamila@deploy2002> helmfile [codfw] DONE helmfile.d/services/rest-gateway: apply [production]
15:05 <kamila@deploy2002> helmfile [codfw] START helmfile.d/services/rest-gateway: apply [production]
15:04 <kamila@deploy2002> helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply [production]
15:04 <kamila@deploy2002> helmfile [eqiad] START helmfile.d/services/rest-gateway: apply [production]
15:02 <cmooney@cumin1001> END (FAIL) - Cookbook sre.hosts.reimage (exit_code=99) for host cloudvirtlocal1001.eqiad.wmnet with OS bullseye [production]
15:00 <otto@deploy2002> helmfile [staging] START helmfile.d/services/eventgate-main: apply [production]
15:00 <otto@deploy2002> helmfile [eqiad] DONE helmfile.d/services/eventgate-logging-external: apply [production]
14:59 <otto@deploy2002> helmfile [eqiad] START helmfile.d/services/eventgate-logging-external: apply [production]
14:59 <otto@deploy2002> helmfile [codfw] DONE helmfile.d/services/eventgate-logging-external: apply [production]
14:59 <otto@deploy2002> helmfile [codfw] START helmfile.d/services/eventgate-logging-external: apply [production]
14:59 <otto@deploy2002> helmfile [staging] DONE helmfile.d/services/eventgate-logging-external: apply [production]
14:58 <otto@deploy2002> helmfile [staging] START helmfile.d/services/eventgate-logging-external: apply [production]
14:58 <otto@deploy2002> helmfile [eqiad] DONE helmfile.d/services/eventgate-analytics-external: apply [production]
14:57 <otto@deploy2002> helmfile [eqiad] START helmfile.d/services/eventgate-analytics-external: apply [production]
14:57 <otto@deploy2002> helmfile [codfw] DONE helmfile.d/services/eventgate-analytics-external: apply [production]
14:57 <otto@deploy2002> helmfile [codfw] START helmfile.d/services/eventgate-analytics-external: apply [production]
14:56 <otto@deploy2002> helmfile [staging] DONE helmfile.d/services/eventgate-analytics-external: apply [production]
14:56 <otto@deploy2002> helmfile [staging] START helmfile.d/services/eventgate-analytics-external: apply [production]
14:55 <otto@deploy2002> helmfile [eqiad] DONE helmfile.d/services/eventgate-analytics: apply [production]
14:55 <otto@deploy2002> helmfile [eqiad] START helmfile.d/services/eventgate-analytics: apply [production]
14:54 <otto@deploy2002> helmfile [codfw] DONE helmfile.d/services/eventgate-analytics: apply [production]
14:53 <otto@deploy2002> helmfile [codfw] START helmfile.d/services/eventgate-analytics: apply [production]
14:53 <otto@deploy2002> helmfile [staging] DONE helmfile.d/services/eventgate-analytics: apply [production]
14:53 <otto@deploy2002> helmfile [staging] START helmfile.d/services/eventgate-analytics: apply [production]
14:52 <otto@deploy2002> helmfile [staging] DONE helmfile.d/services/eventgate-analytics: apply [production]
14:52 <kamila@deploy2002> helmfile [eqiad] DONE helmfile.d/services/rest-gateway: apply [production]
14:52 <otto@deploy2002> helmfile [staging] START helmfile.d/services/eventgate-analytics: apply [production]
14:51 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2155 (T333332)', diff saved to https://phabricator.wikimedia.org/P46563 and previous config saved to /var/cache/conftool/dbconfig/20230412-145108-ladsgroup.json [production]
14:50 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1202', diff saved to https://phabricator.wikimedia.org/P46562 and previous config saved to /var/cache/conftool/dbconfig/20230412-145051-ladsgroup.json [production]
14:48 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2155 (T333332)', diff saved to https://phabricator.wikimedia.org/P46561 and previous config saved to /var/cache/conftool/dbconfig/20230412-144856-ladsgroup.json [production]
14:48 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 16:00:00 on db2187.codfw.wmnet with reason: Maintenance [production]
14:48 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 16:00:00 on db2187.codfw.wmnet with reason: Maintenance [production]