401-450 of 10000 results (51ms)
2022-05-02 ยง
04:48 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1119 (T306560)', diff saved to https://phabricator.wikimedia.org/P27252 and previous config saved to /var/cache/conftool/dbconfig/20220502-044834-ladsgroup.json [production]
04:46 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1119 (T306560)', diff saved to https://phabricator.wikimedia.org/P27251 and previous config saved to /var/cache/conftool/dbconfig/20220502-044614-ladsgroup.json [production]
04:46 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1119.eqiad.wmnet with reason: Maintenance [production]
04:46 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1119.eqiad.wmnet with reason: Maintenance [production]
04:46 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1099:3311 (T306560)', diff saved to https://phabricator.wikimedia.org/P27250 and previous config saved to /var/cache/conftool/dbconfig/20220502-044606-ladsgroup.json [production]
04:41 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166', diff saved to https://phabricator.wikimedia.org/P27249 and previous config saved to /var/cache/conftool/dbconfig/20220502-044151-ladsgroup.json [production]
04:31 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1099:3311', diff saved to https://phabricator.wikimedia.org/P27248 and previous config saved to /var/cache/conftool/dbconfig/20220502-043101-ladsgroup.json [production]
04:26 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166', diff saved to https://phabricator.wikimedia.org/P27247 and previous config saved to /var/cache/conftool/dbconfig/20220502-042646-ladsgroup.json [production]
04:15 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1099:3311', diff saved to https://phabricator.wikimedia.org/P27246 and previous config saved to /var/cache/conftool/dbconfig/20220502-041556-ladsgroup.json [production]
04:11 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1166 (T298565)', diff saved to https://phabricator.wikimedia.org/P27245 and previous config saved to /var/cache/conftool/dbconfig/20220502-041141-ladsgroup.json [production]
04:10 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 20:00:00 on 8 hosts with reason: Maintenance [production]
04:10 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 20:00:00 on 8 hosts with reason: Maintenance [production]
04:10 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on db2129.codfw.wmnet with reason: Maintenance [production]
04:10 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 10:00:00 on db2129.codfw.wmnet with reason: Maintenance [production]
04:09 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1130 (T298295)', diff saved to https://phabricator.wikimedia.org/P27244 and previous config saved to /var/cache/conftool/dbconfig/20220502-040908-ladsgroup.json [production]
04:07 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1130 (T298295)', diff saved to https://phabricator.wikimedia.org/P27243 and previous config saved to /var/cache/conftool/dbconfig/20220502-040754-ladsgroup.json [production]
04:07 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1130.eqiad.wmnet with reason: Maintenance [production]
04:07 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1130.eqiad.wmnet with reason: Maintenance [production]
04:07 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1096:3315 (T298295)', diff saved to https://phabricator.wikimedia.org/P27242 and previous config saved to /var/cache/conftool/dbconfig/20220502-040745-ladsgroup.json [production]
04:05 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
04:01 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
04:01 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
04:00 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1099:3311 (T306560)', diff saved to https://phabricator.wikimedia.org/P27241 and previous config saved to /var/cache/conftool/dbconfig/20220502-040051-ladsgroup.json [production]
03:58 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1099:3311 (T306560)', diff saved to https://phabricator.wikimedia.org/P27240 and previous config saved to /var/cache/conftool/dbconfig/20220502-035830-ladsgroup.json [production]
03:58 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1099.eqiad.wmnet with reason: Maintenance [production]
03:58 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1099.eqiad.wmnet with reason: Maintenance [production]
03:58 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1132.eqiad.wmnet with reason: Maintenance [production]
03:58 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1132.eqiad.wmnet with reason: Maintenance [production]
03:57 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on 14 hosts with reason: Maintenance [production]
03:57 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
03:57 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 12:00:00 on 14 hosts with reason: Maintenance [production]
03:57 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db2103.codfw.wmnet with reason: Maintenance [production]
03:57 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db2103.codfw.wmnet with reason: Maintenance [production]
03:57 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1163 (T306560)', diff saved to https://phabricator.wikimedia.org/P27239 and previous config saved to /var/cache/conftool/dbconfig/20220502-035733-ladsgroup.json [production]
03:55 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1166 (T298565)', diff saved to https://phabricator.wikimedia.org/P27238 and previous config saved to /var/cache/conftool/dbconfig/20220502-035522-ladsgroup.json [production]
03:55 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1166.eqiad.wmnet with reason: Maintenance [production]
03:55 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1166.eqiad.wmnet with reason: Maintenance [production]
03:55 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1112 (T298565)', diff saved to https://phabricator.wikimedia.org/P27237 and previous config saved to /var/cache/conftool/dbconfig/20220502-035514-ladsgroup.json [production]
03:52 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1096:3315', diff saved to https://phabricator.wikimedia.org/P27236 and previous config saved to /var/cache/conftool/dbconfig/20220502-035240-ladsgroup.json [production]
03:47 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
03:46 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1127 (T306560)', diff saved to https://phabricator.wikimedia.org/P27235 and previous config saved to /var/cache/conftool/dbconfig/20220502-034657-ladsgroup.json [production]
03:46 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
03:46 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
03:45 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
03:42 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1163', diff saved to https://phabricator.wikimedia.org/P27234 and previous config saved to /var/cache/conftool/dbconfig/20220502-034228-ladsgroup.json [production]
03:41 <ladsgroup@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:787939|Set testwiki to READ NEW for templatelinks migration (T306673)]] (duration: 00m 49s) [production]
03:40 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1112', diff saved to https://phabricator.wikimedia.org/P27233 and previous config saved to /var/cache/conftool/dbconfig/20220502-034009-ladsgroup.json [production]
03:37 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1096:3315', diff saved to https://phabricator.wikimedia.org/P27232 and previous config saved to /var/cache/conftool/dbconfig/20220502-033735-ladsgroup.json [production]
03:31 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1127', diff saved to https://phabricator.wikimedia.org/P27231 and previous config saved to /var/cache/conftool/dbconfig/20220502-033152-ladsgroup.json [production]
03:27 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1163', diff saved to https://phabricator.wikimedia.org/P27230 and previous config saved to /var/cache/conftool/dbconfig/20220502-032723-ladsgroup.json [production]