Monte Carlo retest method gives different results in SQ Build 118 than SQ Build 117.

I tested the same strategy again in the latest version of SQ.
The results are completely different than in the previous version of the program.
Attachments
2 MonteCarlo Equity SQBuild118.png
(550.52 KiB)
1 MonteCarlo Equity SQBuild117.png
(635.86 KiB)
Cross checks_setting.png
(159.25 KiB)
Cross checks_setting3.png
(173.74 KiB)
Strategy_config.png
(125.44 KiB)
Cross checks_setting2.png
(175.45 KiB)
  • Votes 0
  • Project StrategyQuant X
  • Type Bug
  • Status Fixed
  • Priority Normal

History

MR
#1

MarekRogowski

03.02.2019 14:25

Task created

m
#2

mabi

04.02.2019 09:49
Have You compared them using selected timeframe only.  In 118 a major fix was applied to montecarlo.
m
#3

mabi

04.02.2019 09:49
Have You compared them using selected timeframe only.  In 118 a major fix was applied to montecarlo.
Kc
#4

coensio

04.02.2019 11:32
Not only the results are very different but there is also a huge difference (decrease) in computation speed between 117 and 118. See sq4_4122. Which one can be trusted?
MR
#5

MarekRogowski

04.02.2019 21:53

Attachment 116126.png added

Attachment 211190.png added

Attachment 241119.png added

Attachment settings_Data.png added

I tested the same time interval (H1). On the same currency pair.
Below are the time frame settings and several test results on other strategies.
The results are very strange.
g
#6

geektrader

04.02.2019 23:16
There was a big bug in #117 with the Monte Carlo simulations (most of them been wrong and hence too optimistic). #118 is giving accurate results now as it seems. Of course this also means that strategies you´ve tested with MC in 117 will now fail in 118 (if they are bad). There is nothing wrong with that unfortunately, the results of 118 are correct as it seems.
MR
#7

MarekRogowski

07.02.2019 07:23

It seems that MC tests in 118 work properly.

The settings were too harsh because the strategies did not pass the tests or the strategies were weak. I also made a mistake in the Data, Test parameters settings. In this case, "Selected timeframe only (fasted)" should be selected. Thank you all for help in finding the cause.
s
#8

Schutten

08.02.2019 19:58

I have the same problem, also with H1 timeframe.

About the remark: " selected timeframe should be selected" Could you explain this?

m
#9

mabi

09.02.2019 00:40
It also depends on the logic of the strategy. Some backtest results is very different in SQx 118 compared with earlier versions.
h
#10

hankeys

09.02.2019 09:16
sure, some building blocks are precision independent - for example everything DAILY - lowdaily will still remain lowdaily, even with 1M or selected TF only


HIGHEST XXX will be the same, etc. etc.


some indicators with high periods can behave the same, because averaging will cut off the differencies

s
#11

Schutten

09.02.2019 14:29

Correct me if I am wrong but the ultimate way of testing would be to use tick data right?

If this is the best way then good models would pass the Retest method as well with tickdata because this would resemble live trading, however there is a big difference in 117 and 118. All my models don't survive version 118 however they make money during real live trading. So I think there is something wrong with version 118.

Results are not bad with retesting but really realy bad just like the pictures above send in.....

TT
#12

Tamas

23.08.2019 08:52

Status changed from New to Fixed


Votes: 0

Drop files to upload

or

choose files

Max size: 5MB

Not allowed: exe, msi, application, reg, php, js, htaccess, htpasswd, gitignore

...
Wait please