V130_DEV_2 - Automatic Filters / Ranking Filters Ignored in Build for 'Selected Timeframe' data and jForex Engine

A bit more info:

v129.302 using MT4 engine project R920 GBPUSD versus v130.448 using jForex Engine.

Same project in V130.448 using Jforex Engine (iso MT4 engine in v129.302), the higher precision backtest rejects 40% (iso 0.02% in V129.302 MT4 Engine). Turning off this crosscheck in V130.448 using Jforex engine results in the strange looking equity charts that are produced at a very high rate.

Build stats from v129.302 using MT4 engine for project R920 GBPUSD (most relevant in bold)

Strategies generated: 3810116, Time per strategy: 4 ms., Time per accepted strategy: 3 s., Accepted: 5237, Rejected: 3804879, Strategies per hour: 855458, Accepted strategies per hour: 1176

Time per strategy details:

Main test: Count: 3251633, Avg. time: 0.2065 s., Total time: 671564.1130 s., % of all: 98.63 %

Higher backtest precision: Count: 6525, Avg. time: 1.4302 s., Total time: 9332.3080 s., % of all: 1.37 %

Rejected details:

Global filter:  Trades Symmetry[Main data] > 10.00, Count: 306, % of all: 0.01 %

Global filter:  Ret/DD Ratio[Main data, IS] > 3.50, Count: 3419, % of all: 0.09 %

Automatic filter: too little trades, Count: 203195, % of all: 5.34 %

Initial population filter: Trades Symmetry[Main data, IS] > 10.00, Count: 64289, % of all: 1.69 %

Cross Check filter in 'Higher backtest precision':  Net profit (Higher backtest precis.)[Cross Check  data] >= 90% of Net profit[Main data], Count: 853, % of all: 0.02 %

Automatic filter: outlier trade - one exceptionally big trade, Count: 81, % of all: 0.0 %

Global filter:  Profit factor[Main data, OOS] > 1.30, Count: 76209, % of all: 2.0 %

Global filter:  # of trades[Main data, IS] > 210.00, Count: 961702, % of all: 25.28 %

Global filter:  # of trades[Main data, OOS] > 90.00, Count: 73160, % of all: 1.92 %

initial population, Count: 201301, % of all: 5.29 %

Global filter:  Profit factor[Main data, IS] > 1.30, Count: 1864383, % of all: 49.0 %

Automatic filter: too many trades closing at the same bar, Count: 100762, % of all: 2.65 %

Cross Check filter in 'Higher backtest precision':  Max DD % (Higher backtest precis.)[Cross Check  data] < 110% of Max DD %[Main data], Count: 435, % of all: 0.01 %

Automatic filter: no trades, Count: 238917, % of all: 6.28 %

Global filter:  Ret/DD Ratio[Main data, OOS] > 1.50, Count: 339, % of all: 0.01 %

Automatic filter: too many ambiguous trades, Count: 15528, % of all: 0.41 %

Accepted details:

In databank, Count: 5000, % of all: 95.47 %

Replaced with better strategy, Count: 91, % of all: 1.74 %

Strategy is too similar or the same as some other strategy in databank, Count: 146, % of all: 2.79 %

My 'hunch' was the higher rejection rates on the Higher precision backtests (M1 Tick sim data) is behaving (correctly) as a filter for the M1 Selected TF builds (which don't seem to be getting filtered somehow in the build process).

It doesn't 'appear' to do it when building on M1 tick sim data, which is why I thought perhaps M1 Selected TF data in the build  might have something to do with it . But I can't be sure / haven't managed to isolate it completely yet.

This is an established strategy, running demo and forward test, regular optimization works fine/as expected on 1 min tick sim and down, on both jforex and mt4 engine,  but selected tf doesnt work with jforex engine.

I am more convinced of a compatibility issue between selected tf and the jforex engine. They don't seem to play nice together. :)

Pic = identical optimization runs, changing only 1 min tick sim vs selected tf

Build 130.457 dev 3

For Robustness testing or Build, only 1 min tick sim data or better should be used with the jForex Engine.

For anyone that needs a workaround until build 131: build and develop with the mt4 engine with realistic gaps turned on, then add an extra step to compare the results of the MT4 Engine with the Jforex Engine on 1min tick sim data or better. In general, they match up pretty good (90% of the time). Then (obviously) double check the behavior/mechanics on the jForex platform itself in a backtest - they match up about 60/70% of the time with the SQX/jForex Engine results (when they don't it is usually down to a building block that has yet to be translated across - there are lots, so this is understandable - just turn them off in the build when you discover one).

Caveats and workarounds aside, it works well enough in its present state for me to use the results in live forward tests.

We summed up a few things we found when solving these bugs. There are some differences in JForex engine that are not so easy to solve, but they are possible to be overcomed.