3751-3800 of 10000 results (57ms)
2022-04-26 ยง
19:28 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on db1161.eqiad.wmnet with reason: Maintenance [production]
19:28 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 10:00:00 on db1161.eqiad.wmnet with reason: Maintenance [production]
19:28 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1096:3315 (T298556)', diff saved to https://phabricator.wikimedia.org/P26605 and previous config saved to /var/cache/conftool/dbconfig/20220426-192828-ladsgroup.json [production]
19:20 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1106', diff saved to https://phabricator.wikimedia.org/P26604 and previous config saved to /var/cache/conftool/dbconfig/20220426-192057-ladsgroup.json [production]
19:13 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1096:3315', diff saved to https://phabricator.wikimedia.org/P26603 and previous config saved to /var/cache/conftool/dbconfig/20220426-191323-ladsgroup.json [production]
19:06 <aqu@deploy1002> Started deploy [analytics/refinery@96a3934]: Regular analytics weekly train [analytics/refinery@96a3934] [production]
19:05 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1106', diff saved to https://phabricator.wikimedia.org/P26602 and previous config saved to /var/cache/conftool/dbconfig/20220426-190552-ladsgroup.json [production]
19:02 <aqu> About to deploy analytics/refinery: Weekly deployment train + Artifacts to 0.1.27 [production]
18:58 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1096:3315', diff saved to https://phabricator.wikimedia.org/P26601 and previous config saved to /var/cache/conftool/dbconfig/20220426-185818-ladsgroup.json [production]
18:50 <cmooney@cumin1001> END (PASS) - Cookbook sre.deploy.python-code (exit_code=0) homer to cumin2002.codfw.wmnet,cumin1001.eqiad.wmnet with reason: Release v0.4.1a - cmooney@cumin1001 [production]
18:50 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1106 (T306560)', diff saved to https://phabricator.wikimedia.org/P26599 and previous config saved to /var/cache/conftool/dbconfig/20220426-185047-ladsgroup.json [production]
18:49 <cmooney@cumin1001> START - Cookbook sre.deploy.python-code homer to cumin2002.codfw.wmnet,cumin1001.eqiad.wmnet with reason: Release v0.4.1a - cmooney@cumin1001 [production]
18:48 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1106 (T306560)', diff saved to https://phabricator.wikimedia.org/P26598 and previous config saved to /var/cache/conftool/dbconfig/20220426-184815-ladsgroup.json [production]
18:48 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on clouddb[1013,1017,1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance [production]
18:48 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 12:00:00 on clouddb[1013,1017,1021].eqiad.wmnet,db1154.eqiad.wmnet with reason: Maintenance [production]
18:48 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1106.eqiad.wmnet with reason: Maintenance [production]
18:48 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1106.eqiad.wmnet with reason: Maintenance [production]
18:47 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1133.eqiad.wmnet with reason: Maintenance [production]
18:47 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1133.eqiad.wmnet with reason: Maintenance [production]
18:47 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1132 (T306560)', diff saved to https://phabricator.wikimedia.org/P26597 and previous config saved to /var/cache/conftool/dbconfig/20220426-184729-ladsgroup.json [production]
18:43 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1096:3315 (T298556)', diff saved to https://phabricator.wikimedia.org/P26596 and previous config saved to /var/cache/conftool/dbconfig/20220426-184313-ladsgroup.json [production]
18:41 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1096:3315 (T298556)', diff saved to https://phabricator.wikimedia.org/P26595 and previous config saved to /var/cache/conftool/dbconfig/20220426-184058-ladsgroup.json [production]
18:40 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on db1096.eqiad.wmnet with reason: Maintenance [production]
18:40 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 10:00:00 on db1096.eqiad.wmnet with reason: Maintenance [production]
18:37 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
18:37 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
18:37 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
18:37 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
18:32 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1132', diff saved to https://phabricator.wikimedia.org/P26594 and previous config saved to /var/cache/conftool/dbconfig/20220426-183224-ladsgroup.json [production]
18:17 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1132', diff saved to https://phabricator.wikimedia.org/P26593 and previous config saved to /var/cache/conftool/dbconfig/20220426-181719-ladsgroup.json [production]
18:07 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
18:07 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
18:07 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
18:07 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
18:06 <brennen@deploy1002> rebuilt and synchronized wikiversions files: group0 wikis to 1.39.0-wmf.9 refs T305215 [production]
18:02 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1132 (T306560)', diff saved to https://phabricator.wikimedia.org/P26592 and previous config saved to /var/cache/conftool/dbconfig/20220426-180214-ladsgroup.json [production]
17:59 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1132 (T306560)', diff saved to https://phabricator.wikimedia.org/P26591 and previous config saved to /var/cache/conftool/dbconfig/20220426-175941-ladsgroup.json [production]
17:59 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1132.eqiad.wmnet with reason: Maintenance [production]
17:59 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1132.eqiad.wmnet with reason: Maintenance [production]
17:59 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1105:3311 (T306560)', diff saved to https://phabricator.wikimedia.org/P26590 and previous config saved to /var/cache/conftool/dbconfig/20220426-175933-ladsgroup.json [production]
17:55 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1130 (T298556)', diff saved to https://phabricator.wikimedia.org/P26589 and previous config saved to /var/cache/conftool/dbconfig/20220426-175536-ladsgroup.json [production]
17:54 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1130 (T298556)', diff saved to https://phabricator.wikimedia.org/P26588 and previous config saved to /var/cache/conftool/dbconfig/20220426-175424-ladsgroup.json [production]
17:54 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on db1130.eqiad.wmnet with reason: Maintenance [production]
17:54 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 10:00:00 on db1130.eqiad.wmnet with reason: Maintenance [production]
17:54 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on dbstore1003.eqiad.wmnet with reason: Maintenance [production]
17:54 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 10:00:00 on dbstore1003.eqiad.wmnet with reason: Maintenance [production]
17:53 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 20:00:00 on 8 hosts with reason: Maintenance [production]
17:53 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 20:00:00 on 8 hosts with reason: Maintenance [production]
17:53 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 10:00:00 on db2123.codfw.wmnet with reason: Maintenance [production]
17:53 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 10:00:00 on db2123.codfw.wmnet with reason: Maintenance [production]