7251-7300 of 10000 results (94ms)
2022-11-25 ยง
18:53 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1112 (T323827)', diff saved to https://phabricator.wikimedia.org/P41168 and previous config saved to /var/cache/conftool/dbconfig/20221125-185312-ladsgroup.json [production]
18:52 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1146:3312 (T323827)', diff saved to https://phabricator.wikimedia.org/P41167 and previous config saved to /var/cache/conftool/dbconfig/20221125-185257-ladsgroup.json [production]
18:52 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on db1146.eqiad.wmnet with reason: Maintenance [production]
18:52 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 10:00:00 on db1146.eqiad.wmnet with reason: Maintenance [production]
18:49 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2177 (T321126)', diff saved to https://phabricator.wikimedia.org/P41166 and previous config saved to /var/cache/conftool/dbconfig/20221125-184943-marostegui.json [production]
18:49 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2138:3312', diff saved to https://phabricator.wikimedia.org/P41165 and previous config saved to /var/cache/conftool/dbconfig/20221125-184902-ladsgroup.json [production]
18:38 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1112', diff saved to https://phabricator.wikimedia.org/P41164 and previous config saved to /var/cache/conftool/dbconfig/20221125-183806-ladsgroup.json [production]
18:33 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2138:3312 (T323827)', diff saved to https://phabricator.wikimedia.org/P41163 and previous config saved to /var/cache/conftool/dbconfig/20221125-183356-ladsgroup.json [production]
18:23 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1112', diff saved to https://phabricator.wikimedia.org/P41162 and previous config saved to /var/cache/conftool/dbconfig/20221125-182259-ladsgroup.json [production]
18:21 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db2177 (T321126)', diff saved to https://phabricator.wikimedia.org/P41161 and previous config saved to /var/cache/conftool/dbconfig/20221125-182126-marostegui.json [production]
18:21 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2177.codfw.wmnet with reason: Maintenance [production]
18:21 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 5:00:00 on db2177.codfw.wmnet with reason: Maintenance [production]
18:21 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2156 (T321126)', diff saved to https://phabricator.wikimedia.org/P41160 and previous config saved to /var/cache/conftool/dbconfig/20221125-182105-marostegui.json [production]
18:19 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on db1139.eqiad.wmnet with reason: Maintenance [production]
18:19 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 10:00:00 on db1139.eqiad.wmnet with reason: Maintenance [production]
18:19 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1129 (T323827)', diff saved to https://phabricator.wikimedia.org/P41159 and previous config saved to /var/cache/conftool/dbconfig/20221125-181900-ladsgroup.json [production]
18:07 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1112 (T323827)', diff saved to https://phabricator.wikimedia.org/P41158 and previous config saved to /var/cache/conftool/dbconfig/20221125-180753-ladsgroup.json [production]
18:05 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2156', diff saved to https://phabricator.wikimedia.org/P41157 and previous config saved to /var/cache/conftool/dbconfig/20221125-180558-marostegui.json [production]
18:03 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1129', diff saved to https://phabricator.wikimedia.org/P41156 and previous config saved to /var/cache/conftool/dbconfig/20221125-180353-ladsgroup.json [production]
17:56 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2138:3312 (T323827)', diff saved to https://phabricator.wikimedia.org/P41155 and previous config saved to /var/cache/conftool/dbconfig/20221125-175624-ladsgroup.json [production]
17:56 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on db2138.codfw.wmnet with reason: Maintenance [production]
17:55 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 10:00:00 on db2138.codfw.wmnet with reason: Maintenance [production]
17:55 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2126 (T323827)', diff saved to https://phabricator.wikimedia.org/P41154 and previous config saved to /var/cache/conftool/dbconfig/20221125-175551-ladsgroup.json [production]
17:51 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1112 (T323827)', diff saved to https://phabricator.wikimedia.org/P41153 and previous config saved to /var/cache/conftool/dbconfig/20221125-175114-ladsgroup.json [production]
17:51 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 20:00:00 on clouddb[1013,1017,1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance [production]
17:51 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 20:00:00 on clouddb[1013,1017,1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance [production]
17:50 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2156', diff saved to https://phabricator.wikimedia.org/P41152 and previous config saved to /var/cache/conftool/dbconfig/20221125-175052-marostegui.json [production]
17:50 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on db1112.eqiad.wmnet with reason: Maintenance [production]
17:50 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 10:00:00 on db1112.eqiad.wmnet with reason: Maintenance [production]
17:48 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1129', diff saved to https://phabricator.wikimedia.org/P41151 and previous config saved to /var/cache/conftool/dbconfig/20221125-174847-ladsgroup.json [production]
17:40 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2126', diff saved to https://phabricator.wikimedia.org/P41150 and previous config saved to /var/cache/conftool/dbconfig/20221125-174045-ladsgroup.json [production]
17:38 <urandom> initiating Cassandra bootstrap, aqs1021-a -- T307802 [production]
17:35 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2156 (T321126)', diff saved to https://phabricator.wikimedia.org/P41149 and previous config saved to /var/cache/conftool/dbconfig/20221125-173545-marostegui.json [production]
17:33 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1129 (T323827)', diff saved to https://phabricator.wikimedia.org/P41148 and previous config saved to /var/cache/conftool/dbconfig/20221125-173340-ladsgroup.json [production]
17:25 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2126', diff saved to https://phabricator.wikimedia.org/P41147 and previous config saved to /var/cache/conftool/dbconfig/20221125-172538-ladsgroup.json [production]
17:21 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on db1102.eqiad.wmnet with reason: Maintenance [production]
17:21 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 10:00:00 on db1102.eqiad.wmnet with reason: Maintenance [production]
17:17 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1129 (T323827)', diff saved to https://phabricator.wikimedia.org/P41146 and previous config saved to /var/cache/conftool/dbconfig/20221125-171729-ladsgroup.json [production]
17:17 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on db1129.eqiad.wmnet with reason: Maintenance [production]
17:17 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 10:00:00 on db1129.eqiad.wmnet with reason: Maintenance [production]
17:17 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1122 (T323827)', diff saved to https://phabricator.wikimedia.org/P41145 and previous config saved to /var/cache/conftool/dbconfig/20221125-171707-ladsgroup.json [production]
17:10 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2126 (T323827)', diff saved to https://phabricator.wikimedia.org/P41144 and previous config saved to /var/cache/conftool/dbconfig/20221125-171032-ladsgroup.json [production]
17:08 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db2156 (T321126)', diff saved to https://phabricator.wikimedia.org/P41143 and previous config saved to /var/cache/conftool/dbconfig/20221125-170859-marostegui.json [production]
17:08 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on db2094.codfw.wmnet with reason: Maintenance [production]
17:08 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 10:00:00 on db2094.codfw.wmnet with reason: Maintenance [production]
17:08 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2156.codfw.wmnet with reason: Maintenance [production]
17:08 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 5:00:00 on db2156.codfw.wmnet with reason: Maintenance [production]
17:08 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2149 (T321126)', diff saved to https://phabricator.wikimedia.org/P41142 and previous config saved to /var/cache/conftool/dbconfig/20221125-170811-marostegui.json [production]
17:02 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1122', diff saved to https://phabricator.wikimedia.org/P41141 and previous config saved to /var/cache/conftool/dbconfig/20221125-170200-ladsgroup.json [production]
16:53 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2126 (T323827)', diff saved to https://phabricator.wikimedia.org/P41140 and previous config saved to /var/cache/conftool/dbconfig/20221125-165341-ladsgroup.json [production]