Build-122_Results-Differences between builder and retester at the same settings: e.g. AUDCAD

Hi,

Unfortunately I can not explain why I get different results in the results of Builder and Retester.
Why is that?

Here is an example for audcad:
I received 19 results in the builder. I passed these results to the retester with the same settings. The result of the retester is: 11 of 19 Results are failed! How can that be, the settings are the same and the results differ?

The database for the test is Tickdata to audcad from 2006 to 2019.

Attachments

Database_audcad.jpg

(25.58 KiB)

audcad_15M_003_1TStrategy 113131.sqx

(221.27 KiB)

Retester-results.jpg

(124.19 KiB)

audcad_15m_Strategy 1131311T_003.sqx

(221.69 KiB)

audcad_15M_003_1TStrategy 116163.sqx

(245.93 KiB)

audcad_15m_Strategy 1161631T_003.sqx

(246.75 KiB)

audcad_15m_Strategy 1121231T_003.sqx

(192.64 KiB)

Builder-results.jpg

(128.97 KiB)

Retester_Settings_audcad_M15_003_1T.cfx

(3.41 KiB)

Builder_Settings_audcad_M15_003_1T.cfx

(24.65 KiB)

audcad_15M_003_1TStrategy 112123.sqx

(192.16 KiB)

Votes 0
Project StrategyQuant X
Type Bug
Status Refused
Priority Normal

History

Diwi

14.07.2019 14:06

Task created

Mark Fric

17.07.2019 13:54

Status changed from New to Refused

ok, the explanation is simple - you use the same settings and data, but you also use Monte Carlo cross checks.

Monte Carlo method is random, it will generate different simulations every time you run it. YOu also use a very liittle number of simulations - only 10 or 30.

That way there is a big chance that totally different conditions will be generated every time you run it.

This is why it could have failed in Retester when it passed in Builder.

One solution would be to use much more simulations - at least 100, possibly 500 or 1000 in MOnte Carlo methods. Ths will make it less random and more statistically significant.

Diwi

17.07.2019 20:32

Hello Marc,

ok, that sounds logical to me. I will do the tests with more accuracy. I forgot that the Monte Carlo method is random.

Ah, thanks for that memory of my gray cells ;-)

Votes: 0