7951-8000 of 10000 results (81ms)
2020-03-03 ยง
14:52 <marostegui@cumin1001> dbctl commit (dc=all): 'Slowly repool db1096:3315 and db1096:3316 after reimage to buster T246604', diff saved to https://phabricator.wikimedia.org/P10591 and previous config saved to /var/cache/conftool/dbconfig/20200303-145230-marostegui.json [production]
14:44 <vgutierrez@cumin2001> END (PASS) - Cookbook sre.hosts.decommission (exit_code=0) [production]
14:43 <vgutierrez@cumin2001> START - Cookbook sre.hosts.decommission [production]
14:43 <vgutierrez> running the decommission cookbook against lvs2001.codfw.wmnet - T246779 [production]
14:42 <vgutierrez> replace lvs2001 with lvs2007 - T196560 [production]
14:41 <addshore> START warm cache for db1111 & db1126 for Q20-25 million T219123 (pass 2) [production]
14:29 <vgutierrez> update puppet compiler facts [production]
14:28 <ottomata> postponing LVS for eventgate-analytics-external unti tomorrow [production]
14:13 <ottomata> beginning procedure to add LVS and discovery for eventgate-analytics-external - T233629 [production]
14:12 <elukey@cumin1001> START - Cookbook sre.elasticsearch.rolling-restart [production]
14:11 <liw@deploy1001> Started scap: testwiki to php-1.35.0-wmf.22 and rebuild l10n cache [production]
14:08 <liw@deploy1001> Pruned MediaWiki: 1.35.0-wmf.20 (duration: 15m 54s) [production]
14:08 <vgutierrez@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
14:06 <vgutierrez@cumin1001> START - Cookbook sre.hosts.downtime [production]
13:55 <zpapierski@deploy1001> Finished deploy [wdqs/wdqs@8da3ae6]: (no justification provided) (duration: 10m 48s) [production]
13:54 <vgutierrez> reimage lvs1016 with buster - T245984 [production]
13:52 <vgutierrez@cumin1001> END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) [production]
13:50 <vgutierrez@cumin1001> START - Cookbook sre.hosts.downtime [production]
13:45 <zpapierski@deploy1001> Started deploy [wdqs/wdqs@8da3ae6]: (no justification provided) [production]
13:42 <zpapierski@deploy1001> Finished deploy [wdqs/wdqs@8da3ae6]: (no justification provided) (duration: 00m 26s) [production]
13:41 <zpapierski@deploy1001> Started deploy [wdqs/wdqs@8da3ae6]: (no justification provided) [production]
13:34 <vgutierrez> Re-enable BGP in lvs3005 - T245984 [production]
13:19 <akosiaris@deploy1001> helmfile [STAGING] Ran 'apply' command on namespace 'changeprop' for release 'staging' . [production]
13:14 <akosiaris@deploy1001> helmfile [STAGING] Ran 'apply' command on namespace 'changeprop' for release 'staging' . [production]
13:13 <akosiaris@deploy1001> helmfile [STAGING] Ran 'apply' command on namespace 'changeprop' for release 'staging' . [production]
13:13 <akosiaris@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'changeprop' for release 'staging' . [production]
13:12 <akosiaris@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'changeprop' for release 'staging' . [production]
13:12 <akosiaris@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'changeprop' for release 'staging' . [production]
13:12 <akosiaris@deploy1001> helmfile [STAGING] Ran 'apply' command on namespace 'changeprop' for release 'staging' . [production]
13:11 <akosiaris@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'changeprop' for release 'staging' . [production]
13:10 <akosiaris@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'changeprop' for release 'staging' . [production]
13:02 <akosiaris@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'changeprop' for release 'staging' . [production]
13:01 <akosiaris@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'changeprop' for release 'staging' . [production]
13:00 <akosiaris@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'changeprop' for release 'staging' . [production]
12:48 <hnowlan@deploy1001> helmfile [STAGING] Ran 'sync' command on namespace 'changeprop' for release 'staging' . [production]
12:45 <addshore> START warm cache for db1111 & db1126 for Q20-25 million T219123 (pass 1) [production]
12:38 <addshore@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Reading up to Q20M for the new term store everywhere (was Q15M) + warm db1126 & db1111 caches (T219123) cache bust (duration: 00m 56s) [production]
12:37 <addshore@deploy1001> Synchronized wmf-config/InitialiseSettings.php: Reading up to Q20M for the new term store everywhere (was Q15M) + warm db1126 & db1111 caches (T219123) (duration: 00m 55s) [production]
12:35 <addshore> END warm cache for db1111 & db1126 for Q15-20 million T219123 (pass 3) (finished it early) [production]
12:19 <urbanecm@deploy1001> Synchronized wmf-config/throttle.php: SWAT: 7b48737: Throttle rule for Czech Wikigap (T246356) (duration: 00m 56s) [production]
12:14 <urbanecm@deploy1001> Synchronized wmf-config/interwiki.php: Update interwiki cache (duration: 02m 10s) [production]
12:12 <urbanecm@deploy1001> update-interwiki-cache aborted: Update interwiki cache (duration: 00m 01s) [production]
12:12 <urbanecm@deploy1001> update-interwiki-cache aborted: Update interwiki cache (duration: 00m 00s) [production]
12:10 <kartik@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit|575974|ContentTranslation: Add URL campaign for WikiGapFinder (T246335)]], take II (duration: 00m 56s) [production]
12:09 <kartik@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit|575974|ContentTranslation: Add URL campaign for WikiGapFinder (T246335)]] (duration: 00m 56s) [production]
12:06 <addshore> START warm cache for db1111 & db1126 for Q15-20 million T219123 (pass 3) [production]
12:03 <liw> cutting branch for 1.35.0-wmf.22 train [production]
11:54 <jbond42> disable puppet in order to add netbox hiera backend [production]
11:09 <moritzm> installing Java security updates on an-airflow, an-launcher and an-presto* [production]
11:08 <addshore> START warm cache for db1111 & db1126 for Q15-20 million T219123 (pass 2) [production]