2151-2200 of 10000 results (78ms)
2022-11-25 ยง
17:55 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2126 (T323827)', diff saved to https://phabricator.wikimedia.org/P41154 and previous config saved to /var/cache/conftool/dbconfig/20221125-175551-ladsgroup.json [production]
17:51 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1112 (T323827)', diff saved to https://phabricator.wikimedia.org/P41153 and previous config saved to /var/cache/conftool/dbconfig/20221125-175114-ladsgroup.json [production]
17:51 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 20:00:00 on clouddb[1013,1017,1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance [production]
17:51 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 20:00:00 on clouddb[1013,1017,1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance [production]
17:50 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2156', diff saved to https://phabricator.wikimedia.org/P41152 and previous config saved to /var/cache/conftool/dbconfig/20221125-175052-marostegui.json [production]
17:50 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on db1112.eqiad.wmnet with reason: Maintenance [production]
17:50 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 10:00:00 on db1112.eqiad.wmnet with reason: Maintenance [production]
17:48 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1129', diff saved to https://phabricator.wikimedia.org/P41151 and previous config saved to /var/cache/conftool/dbconfig/20221125-174847-ladsgroup.json [production]
17:40 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2126', diff saved to https://phabricator.wikimedia.org/P41150 and previous config saved to /var/cache/conftool/dbconfig/20221125-174045-ladsgroup.json [production]
17:38 <urandom> initiating Cassandra bootstrap, aqs1021-a -- T307802 [production]
17:35 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2156 (T321126)', diff saved to https://phabricator.wikimedia.org/P41149 and previous config saved to /var/cache/conftool/dbconfig/20221125-173545-marostegui.json [production]
17:33 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1129 (T323827)', diff saved to https://phabricator.wikimedia.org/P41148 and previous config saved to /var/cache/conftool/dbconfig/20221125-173340-ladsgroup.json [production]
17:25 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2126', diff saved to https://phabricator.wikimedia.org/P41147 and previous config saved to /var/cache/conftool/dbconfig/20221125-172538-ladsgroup.json [production]
17:21 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on db1102.eqiad.wmnet with reason: Maintenance [production]
17:21 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 10:00:00 on db1102.eqiad.wmnet with reason: Maintenance [production]
17:17 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1129 (T323827)', diff saved to https://phabricator.wikimedia.org/P41146 and previous config saved to /var/cache/conftool/dbconfig/20221125-171729-ladsgroup.json [production]
17:17 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on db1129.eqiad.wmnet with reason: Maintenance [production]
17:17 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 10:00:00 on db1129.eqiad.wmnet with reason: Maintenance [production]
17:17 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1122 (T323827)', diff saved to https://phabricator.wikimedia.org/P41145 and previous config saved to /var/cache/conftool/dbconfig/20221125-171707-ladsgroup.json [production]
17:10 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2126 (T323827)', diff saved to https://phabricator.wikimedia.org/P41144 and previous config saved to /var/cache/conftool/dbconfig/20221125-171032-ladsgroup.json [production]
17:08 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db2156 (T321126)', diff saved to https://phabricator.wikimedia.org/P41143 and previous config saved to /var/cache/conftool/dbconfig/20221125-170859-marostegui.json [production]
17:08 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on db2094.codfw.wmnet with reason: Maintenance [production]
17:08 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 10:00:00 on db2094.codfw.wmnet with reason: Maintenance [production]
17:08 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2156.codfw.wmnet with reason: Maintenance [production]
17:08 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 5:00:00 on db2156.codfw.wmnet with reason: Maintenance [production]
17:08 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2149 (T321126)', diff saved to https://phabricator.wikimedia.org/P41142 and previous config saved to /var/cache/conftool/dbconfig/20221125-170811-marostegui.json [production]
17:02 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1122', diff saved to https://phabricator.wikimedia.org/P41141 and previous config saved to /var/cache/conftool/dbconfig/20221125-170200-ladsgroup.json [production]
16:53 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db2126 (T323827)', diff saved to https://phabricator.wikimedia.org/P41140 and previous config saved to /var/cache/conftool/dbconfig/20221125-165341-ladsgroup.json [production]
16:53 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 20:00:00 on db2095.codfw.wmnet with reason: Maintenance [production]
16:53 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 20:00:00 on db2095.codfw.wmnet with reason: Maintenance [production]
16:53 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on db2126.codfw.wmnet with reason: Maintenance [production]
16:53 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 10:00:00 on db2126.codfw.wmnet with reason: Maintenance [production]
16:53 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2125 (T323827)', diff saved to https://phabricator.wikimedia.org/P41139 and previous config saved to /var/cache/conftool/dbconfig/20221125-165315-ladsgroup.json [production]
16:53 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2149', diff saved to https://phabricator.wikimedia.org/P41138 and previous config saved to /var/cache/conftool/dbconfig/20221125-165304-marostegui.json [production]
16:49 <mfossati@deploy1002> Finished deploy [airflow-dags/platform_eng@f6b8a0a]: (no justification provided) (duration: 00m 18s) [production]
16:49 <mfossati@deploy1002> Started deploy [airflow-dags/platform_eng@f6b8a0a]: (no justification provided) [production]
16:46 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1122', diff saved to https://phabricator.wikimedia.org/P41137 and previous config saved to /var/cache/conftool/dbconfig/20221125-164654-ladsgroup.json [production]
16:38 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2125', diff saved to https://phabricator.wikimedia.org/P41136 and previous config saved to /var/cache/conftool/dbconfig/20221125-163808-ladsgroup.json [production]
16:37 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2149', diff saved to https://phabricator.wikimedia.org/P41135 and previous config saved to /var/cache/conftool/dbconfig/20221125-163758-marostegui.json [production]
16:31 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1122 (T323827)', diff saved to https://phabricator.wikimedia.org/P41134 and previous config saved to /var/cache/conftool/dbconfig/20221125-163147-ladsgroup.json [production]
16:24 <volans|off> restarted turnilo on an-tool1007 [production]
16:23 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2125', diff saved to https://phabricator.wikimedia.org/P41133 and previous config saved to /var/cache/conftool/dbconfig/20221125-162302-ladsgroup.json [production]
16:22 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2149 (T321126)', diff saved to https://phabricator.wikimedia.org/P41132 and previous config saved to /var/cache/conftool/dbconfig/20221125-162251-marostegui.json [production]
16:11 <_joe_> upgraded vopsbot to 0.3.2 [production]
16:07 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db2125 (T323827)', diff saved to https://phabricator.wikimedia.org/P41131 and previous config saved to /var/cache/conftool/dbconfig/20221125-160755-ladsgroup.json [production]
15:54 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db2149 (T321126)', diff saved to https://phabricator.wikimedia.org/P41130 and previous config saved to /var/cache/conftool/dbconfig/20221125-155447-marostegui.json [production]
15:54 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 5:00:00 on db2149.codfw.wmnet with reason: Maintenance [production]
15:54 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 5:00:00 on db2149.codfw.wmnet with reason: Maintenance [production]
15:53 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1122 (T323827)', diff saved to https://phabricator.wikimedia.org/P41129 and previous config saved to /var/cache/conftool/dbconfig/20221125-155300-ladsgroup.json [production]
15:52 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on db1122.eqiad.wmnet with reason: Maintenance [production]