3251-3300 of 10000 results (58ms)
2022-05-04 ยง
21:39 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1096.eqiad.wmnet with reason: Maintenance [production]
21:39 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1131 (T307525)', diff saved to https://phabricator.wikimedia.org/P27541 and previous config saved to /var/cache/conftool/dbconfig/20220504-213908-ladsgroup.json [production]
21:33 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1179', diff saved to https://phabricator.wikimedia.org/P27540 and previous config saved to /var/cache/conftool/dbconfig/20220504-213358-ladsgroup.json [production]
21:27 <cjming> end of UTC late backport & config window [production]
21:25 <cjming@deploy1002> Synchronized php-1.39.0-wmf.10/skins/Vector/resources/skins.vector.styles/components: Backport: [[gerrit:789263|Fix TOC fadeout placement (T306893)]] (duration: 00m 51s) [production]
21:24 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1131', diff saved to https://phabricator.wikimedia.org/P27539 and previous config saved to /var/cache/conftool/dbconfig/20220504-212401-ladsgroup.json [production]
21:21 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
21:20 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
21:20 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
21:19 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
21:18 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1179 (T307525)', diff saved to https://phabricator.wikimedia.org/P27538 and previous config saved to /var/cache/conftool/dbconfig/20220504-211853-ladsgroup.json [production]
21:16 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1171.eqiad.wmnet with reason: Maintenance [production]
21:16 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1171.eqiad.wmnet with reason: Maintenance [production]
21:16 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1101:3317 (T307525)', diff saved to https://phabricator.wikimedia.org/P27537 and previous config saved to /var/cache/conftool/dbconfig/20220504-211607-ladsgroup.json [production]
21:14 <cjming@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:788819|labswiki: Enable extension SubPageList3 (T304181)]] (duration: 00m 51s) [production]
21:11 <tgr> running extensions/GrowthExperiments/maintenance/changeWikiConfig.php for T306792 [production]
21:08 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1131', diff saved to https://phabricator.wikimedia.org/P27536 and previous config saved to /var/cache/conftool/dbconfig/20220504-210854-ladsgroup.json [production]
21:04 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
21:03 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
21:03 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
21:02 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
21:01 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1101:3317', diff saved to https://phabricator.wikimedia.org/P27535 and previous config saved to /var/cache/conftool/dbconfig/20220504-210101-ladsgroup.json [production]
20:55 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1179 (T307525)', diff saved to https://phabricator.wikimedia.org/P27534 and previous config saved to /var/cache/conftool/dbconfig/20220504-205526-ladsgroup.json [production]
20:55 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1179.eqiad.wmnet with reason: Maintenance [production]
20:55 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1179.eqiad.wmnet with reason: Maintenance [production]
20:55 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1175 (T307525)', diff saved to https://phabricator.wikimedia.org/P27533 and previous config saved to /var/cache/conftool/dbconfig/20220504-205518-ladsgroup.json [production]
20:53 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1131 (T307525)', diff saved to https://phabricator.wikimedia.org/P27532 and previous config saved to /var/cache/conftool/dbconfig/20220504-205349-ladsgroup.json [production]
20:45 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1101:3317', diff saved to https://phabricator.wikimedia.org/P27531 and previous config saved to /var/cache/conftool/dbconfig/20220504-204556-ladsgroup.json [production]
20:44 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1131 (T307525)', diff saved to https://phabricator.wikimedia.org/P27530 and previous config saved to /var/cache/conftool/dbconfig/20220504-204427-ladsgroup.json [production]
20:44 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1131.eqiad.wmnet with reason: Maintenance [production]
20:44 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1131.eqiad.wmnet with reason: Maintenance [production]
20:44 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1165 (T307525)', diff saved to https://phabricator.wikimedia.org/P27529 and previous config saved to /var/cache/conftool/dbconfig/20220504-204419-ladsgroup.json [production]
20:40 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1175', diff saved to https://phabricator.wikimedia.org/P27528 and previous config saved to /var/cache/conftool/dbconfig/20220504-204013-ladsgroup.json [production]
20:32 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
20:31 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
20:31 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
20:30 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1101:3317 (T307525)', diff saved to https://phabricator.wikimedia.org/P27527 and previous config saved to /var/cache/conftool/dbconfig/20220504-203051-ladsgroup.json [production]
20:30 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
20:29 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1165', diff saved to https://phabricator.wikimedia.org/P27526 and previous config saved to /var/cache/conftool/dbconfig/20220504-202914-ladsgroup.json [production]
20:27 <cjming@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:788776|Duplicate eswiki Growth campaign config to itwiki]] (duration: 00m 51s) [production]
20:26 <razzi@cumin1001> END (PASS) - Cookbook sre.kafka.reboot-workers (exit_code=0) for Kafka jumbo-eqiad cluster: Reboot kafka nodes [production]
20:25 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1175', diff saved to https://phabricator.wikimedia.org/P27525 and previous config saved to /var/cache/conftool/dbconfig/20220504-202508-ladsgroup.json [production]
20:20 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
20:19 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
20:19 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
20:18 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
20:15 <cjming@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:788872|Enable IPInfo instrumentation on all wikis (T296480)]] (duration: 00m 56s) [production]
20:14 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1165', diff saved to https://phabricator.wikimedia.org/P27524 and previous config saved to /var/cache/conftool/dbconfig/20220504-201409-ladsgroup.json [production]
20:13 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
20:10 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1175 (T307525)', diff saved to https://phabricator.wikimedia.org/P27523 and previous config saved to /var/cache/conftool/dbconfig/20220504-201002-ladsgroup.json [production]