1601-1650 of 10000 results (46ms)
2022-02-22 ยง
10:16 <kormat@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1110.eqiad.wmnet with reason: Maintenance [production]
10:16 <kormat@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1110.eqiad.wmnet with reason: Maintenance [production]
10:16 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1099:3311 (T302185)', diff saved to https://phabricator.wikimedia.org/P21269 and previous config saved to /var/cache/conftool/dbconfig/20220222-101604-ladsgroup.json [production]
10:12 <filippo@cumin1001> END (PASS) - Cookbook sre.hosts.reboot-single (exit_code=0) for host prometheus1006.eqiad.wmnet [production]
10:07 <elukey@cumin1001> START - Cookbook sre.hosts.reimage for host ml-serve1001.eqiad.wmnet with OS bullseye [production]
10:02 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1099.eqiad.wmnet with OS bullseye [production]
10:00 <filippo@cumin1001> START - Cookbook sre.hosts.reboot-single for host prometheus1006.eqiad.wmnet [production]
09:52 <XioNoX> restarting cr2-drmrs for software upgrade [production]
09:48 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1099.eqiad.wmnet with reason: host reimage [production]
09:47 <aqu@deploy1002> Finished deploy [analytics/refinery@ed5c9f9] (hadoop-test): Migrate aqs/hourly to Airflow TEST [analytics/refinery@ed5c9f9] (duration: 00m 03s) [production]
09:47 <aqu@deploy1002> Started deploy [analytics/refinery@ed5c9f9] (hadoop-test): Migrate aqs/hourly to Airflow TEST [analytics/refinery@ed5c9f9] [production]
09:47 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1129 (T300381)', diff saved to https://phabricator.wikimedia.org/P21268 and previous config saved to /var/cache/conftool/dbconfig/20220222-094740-marostegui.json [production]
09:45 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 2:00:00 on db1099.eqiad.wmnet with reason: host reimage [production]
09:43 <jayme@cumin1001> END (PASS) - Cookbook sre.dns.netbox (exit_code=0) [production]
09:38 <aqu> Deploying analytics/refinery on hadoop-test only. [production]
09:38 <jayme@cumin1001> START - Cookbook sre.dns.netbox [production]
09:36 <ladsgroup@cumin1001> START - Cookbook sre.hosts.reimage for host db1099.eqiad.wmnet with OS bullseye [production]
09:32 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1129', diff saved to https://phabricator.wikimedia.org/P21267 and previous config saved to /var/cache/conftool/dbconfig/20220222-093235-marostegui.json [production]
09:17 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1129', diff saved to https://phabricator.wikimedia.org/P21266 and previous config saved to /var/cache/conftool/dbconfig/20220222-091730-marostegui.json [production]
09:05 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
09:04 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
09:04 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
09:03 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
09:02 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1129 (T300381)', diff saved to https://phabricator.wikimedia.org/P21265 and previous config saved to /var/cache/conftool/dbconfig/20220222-090226-marostegui.json [production]
08:58 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1099:3318 (T302185)', diff saved to https://phabricator.wikimedia.org/P21264 and previous config saved to /var/cache/conftool/dbconfig/20220222-085835-ladsgroup.json [production]
08:57 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1129 (T300381)', diff saved to https://phabricator.wikimedia.org/P21263 and previous config saved to /var/cache/conftool/dbconfig/20220222-085752-marostegui.json [production]
08:57 <marostegui@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 6:00:00 on db1129.eqiad.wmnet with reason: Maintenance [production]
08:57 <marostegui@cumin1001> START - Cookbook sre.hosts.downtime for 6:00:00 on db1129.eqiad.wmnet with reason: Maintenance [production]
08:56 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Depooling db1099:3311 (T302185)', diff saved to https://phabricator.wikimedia.org/P21262 and previous config saved to /var/cache/conftool/dbconfig/20220222-085653-ladsgroup.json [production]
08:56 <ladsgroup@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 1 day, 0:00:00 on db1099.eqiad.wmnet with reason: Maintenance [production]
08:56 <ladsgroup@cumin1001> START - Cookbook sre.hosts.downtime for 1 day, 0:00:00 on db1099.eqiad.wmnet with reason: Maintenance [production]
08:55 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1172 (T302185)', diff saved to https://phabricator.wikimedia.org/P21261 and previous config saved to /var/cache/conftool/dbconfig/20220222-085536-ladsgroup.json [production]
08:55 <aqu@deploy1002> Finished deploy [airflow-dags/analytics_test@17a70a0]: Add aqs hourly (duration: 00m 08s) [production]
08:55 <aqu@deploy1002> Started deploy [airflow-dags/analytics_test@17a70a0]: Add aqs hourly [production]
08:40 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P21260 and previous config saved to /var/cache/conftool/dbconfig/20220222-084031-ladsgroup.json [production]
08:35 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1156 (T300381)', diff saved to https://phabricator.wikimedia.org/P21259 and previous config saved to /var/cache/conftool/dbconfig/20220222-083534-marostegui.json [production]
08:25 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1172', diff saved to https://phabricator.wikimedia.org/P21258 and previous config saved to /var/cache/conftool/dbconfig/20220222-082527-ladsgroup.json [production]
08:23 <mwdebug-deploy@deploy1002> helmfile [codfw] DONE helmfile.d/services/mwdebug: apply [production]
08:22 <mwdebug-deploy@deploy1002> helmfile [codfw] START helmfile.d/services/mwdebug: apply [production]
08:22 <mwdebug-deploy@deploy1002> helmfile [eqiad] DONE helmfile.d/services/mwdebug: apply [production]
08:21 <taavi> UTC morning deploys done [production]
08:20 <taavi@deploy1002> Synchronized php-1.38.0-wmf.22/extensions/VisualEditor/modules/ve-mw/init/targets/ve.init.mw.DesktopArticleTarget.js: Backport: Revert: [[gerrit:764396|Don't suppress teardown prompt when pressing escape (T302096)]] (duration: 00m 49s) [production]
08:20 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1156', diff saved to https://phabricator.wikimedia.org/P21257 and previous config saved to /var/cache/conftool/dbconfig/20220222-082029-marostegui.json [production]
08:19 <mwdebug-deploy@deploy1002> helmfile [eqiad] START helmfile.d/services/mwdebug: apply [production]
08:10 <ladsgroup@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1172 (T302185)', diff saved to https://phabricator.wikimedia.org/P21256 and previous config saved to /var/cache/conftool/dbconfig/20220222-081022-ladsgroup.json [production]
08:05 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1156', diff saved to https://phabricator.wikimedia.org/P21255 and previous config saved to /var/cache/conftool/dbconfig/20220222-080525-marostegui.json [production]
07:51 <kevinbazira@deploy1002> helmfile [ml-serve-codfw] Ran 'sync' command on namespace 'revscoring-editquality' for release 'main' . [production]
07:50 <marostegui@cumin1001> dbctl commit (dc=all): 'Repooling after maintenance db1156 (T300381)', diff saved to https://phabricator.wikimedia.org/P21254 and previous config saved to /var/cache/conftool/dbconfig/20220222-075020-marostegui.json [production]
07:49 <kevinbazira@deploy1002> helmfile [ml-serve-eqiad] Ran 'sync' command on namespace 'revscoring-editquality' for release 'main' . [production]
07:41 <marostegui@cumin1001> dbctl commit (dc=all): 'Depooling db1156 (T300381)', diff saved to https://phabricator.wikimedia.org/P21253 and previous config saved to /var/cache/conftool/dbconfig/20220222-074106-marostegui.json [production]