3401-3450 of 8586 results (16ms)
2020-12-07 §
12:27 <effie> rollour scap 3.16.0-1 to canaries - T268634 [production]
12:26 <effie> rollour scap 3.16.0-1 to canaries [production]
2020-12-02 §
17:11 <effie> uploading scap 3.16.0-1 [production]
2020-12-01 §
15:10 <hashar@deploy1001> Finished scap: (no justification provided) (duration: 44m 20s) [production]
14:26 <hashar@deploy1001> Started scap: (no justification provided) [production]
13:10 <hashar@deploy1001> Started scap: testwikis wikis to 1.36.0-wmf.20 [production]
2020-11-30 §
21:51 <mutante> parse2001 - scap pull [production]
2020-11-25 §
22:55 <mutante> mwdebug1003 - scap pull - which rsyncs from deploy1001 and runs php-fpm restart check script (T245757) [production]
01:07 <tgr@deploy1001> Finished scap: Backport: [[gerrit:643156|GrowthExperiments: Add Russian aliases (T268519)]] (duration: 32m 09s) [production]
00:35 <tgr@deploy1001> Started scap: Backport: [[gerrit:643156|GrowthExperiments: Add Russian aliases (T268519)]] [production]
2020-11-24 §
14:34 <liw> finished testing Scap on Beta cluster in prep for https://phabricator.wikimedia.org/T268634 [production]
13:01 <liw> done testing Scap release candidate on beta (failed: disk full on deploy01) [production]
12:30 <liw> testing upcoming Scap release on beta [production]
2020-11-23 §
21:26 <mutante> mwdebug1003 - scap pull because <+icinga-wm> PROBLEM - Ensure local MW versions match expected deployment on mwdebug1003 is CRITICAL [production]
12:32 <Urbanecm> Run scap pull at mwdebug1003 [production]
2020-11-20 §
13:34 <liw> finished trying to test scap on beta cluster [production]
13:12 <liw> testing upcoming Scap release on beta [production]
2020-11-18 §
21:47 <mutante> mwdebug1003 - scap pull - T267248 [production]
2020-11-17 §
20:31 <dancy@deploy1001> Finished scap: testwikis wikis to 1.36.0-wmf.18 (duration: 39m 37s) [production]
19:52 <dancy@deploy1001> Started scap: testwikis wikis to 1.36.0-wmf.18 [production]
2020-11-12 §
03:53 <Krinkle> krinkle@deploy1001 I've locked scap deployments. See https://phabricator.wikimedia.org/T267668#6619762 [production]
2020-11-10 §
21:23 <brennen> testing some scap operations, modified to use ssh -n for debugging T223287 [production]
21:02 <brennen@deploy1001> Finished scap: Backport: [[gerrit:640487|language: Honor $wgTranslateNumerals, even if PHP does digit translation(T267614)]] and [[gerrit:640488|Downgrade the severity of the non-numeric argument to formatNum warnings (T267370, T267587)]] (duration: 34m 46s) [production]
20:27 <brennen@deploy1001> Started scap: Backport: [[gerrit:640487|language: Honor $wgTranslateNumerals, even if PHP does digit translation(T267614)]] and [[gerrit:640488|Downgrade the severity of the non-numeric argument to formatNum warnings (T267370, T267587)]] [production]
2020-11-06 §
00:52 <brennen@deploy1001> Finished scap: Synchronizing to pick up i18n for [[gerrit:639505]]. Will resume moving train to group1 on Monday morning (US) (T263182) (duration: 69m 02s) [production]
2020-11-05 §
23:44 <brennen@deploy1001> Started scap: Synchronizing to pick up i18n for [[gerrit:639505]]. Will resume moving train to group1 on Monday morning (US) (T263182) [production]
2020-11-04 §
19:59 <brennen@deploy1001> Finished scap: testwikis wikis to 1.36.0-wmf.16 (duration: 62m 44s) [production]
18:57 <brennen@deploy1001> Started scap: testwikis wikis to 1.36.0-wmf.16 [production]
12:11 <Urbanecm> Run scap pull at snapshot1010 manually [production]
12:09 <Urbanecm> scap-sync file returned `snapshot1010.eqiad.wmnet returned [255]: Host key verification failed.` [production]
2020-10-30 §
20:59 <mutante> mw1267,mw1268 - scap pull and repool - back to prod - T266164 [production]
20:56 <mutante> mw1267,mw1268 - scap pull [production]
18:48 <cdanis> the above scap began (and mostly finished) several minutes ago but is hanging on a couple hosts down for maintenance [production]
18:47 <cdanis> ✔️ cdanis@deploy1001.eqiad.wmnet /srv/mediawiki-staging 🕝☕ scap sync-file wmf-config/InitialiseSettings.php 'lower frwiki featured feeds limit 1a41ef634 T266865' [production]
2020-10-29 §
22:32 <mutante> mw1269 rsyncd/ferm for scap proxy was enabled - mw1268 rsyncd/ferm for scan proxy was removed - deploy1001 scap-proxies dsh group was adjusted [production]
22:21 <mutante> replacing scap proxy for rack A7 eqiad because mw1268 needs to move physically (T266164) [production]
2020-10-28 §
22:54 <mutante> scandium - scap pull after reinstalling OS [production]
2020-10-22 §
21:56 <mutante> deploy1002 - scap pull and added to mediawiki-installation "dsh" group - will be part of scap trains but just like any appserver (T265963) [production]
2020-10-20 §
13:11 <liw@deploy1001> Finished scap: testwikis wikis to 1.36.0-wmf.14 (duration: 58m 03s) [production]
12:13 <liw@deploy1001> Started scap: testwikis wikis to 1.36.0-wmf.14 [production]
2020-10-13 §
19:40 <dduvall@deploy1001> Finished scap: testwikis wikis to 1.36.0-wmf.13 (duration: 40m 51s) [production]
19:00 <dduvall@deploy1001> Started scap: testwikis wikis to 1.36.0-wmf.13 [production]
2020-10-07 §
18:29 <ryankemper> Above tests are as expected, syncing changes everywhere: `scap sync-file wmf-config/ProductionServices.php 'Config: [[gerrit:632683|cloudelastic: envoy sits in front now (T263073)]]'` [production]
18:27 <ryankemper> `scap pull`ed onto `mwdebug2001`; talking to cloudelastic via mediawiki from codfw has the expected decrease in latency due to the tls connection pooling [production]
18:24 <ryankemper> `scap pull`ed onto `mwdebug1002`. Talking to cloudelastic on localhost (which routes thru envoy), 6105 is `cloudelastic-chi-eqiad`, 6106 is `cloudelastic-omega-eqiad`, and 6107 is `cloudelastic-psi-eqiad` as expected [production]
2020-09-30 §
21:00 <twentyafterfour@deploy1001> scap failed: average error rate on 5/6 canaries increased by 10x (rerun with --force to override this check, see https://logstash.wikimedia.org/goto/e474f13ffac6b8c3bf919c4aeafc8c9b for details) [production]
2020-09-29 §
21:47 <twentyafterfour@deploy1001> Finished scap: testwikis wikis to 1.36.0-wmf.10 (duration: 13m 45s) [production]
21:34 <twentyafterfour@deploy1001> Started scap: testwikis wikis to 1.36.0-wmf.10 [production]
20:45 <twentyafterfour@deploy1001> Finished scap: testwikis to 1.36.0-wmf.11 refs T263177 (duration: 69m 57s) [production]
19:35 <twentyafterfour@deploy1001> Started scap: testwikis to 1.36.0-wmf.11 refs T263177 [production]