CRUCIAL - backtest mismatch

after some time i have made some comparation between 1M precision backtest from SQX and tick backtest from Tick Data Suite, because from my real account trading some strategies doesnt trade as i see in the backtest.



After some tests i found some strategies, where i see very big difference between backtests, and mostly this strategies has some OHLC values - i am afraid, that with these buidling blocks we have a major problem


for example these strategies has in the logic entry on OPEN WEEKLY, HIGH DAILY, CLOSE DAILY


second question: is intention, that OPEN (OHLC) buidling block is negated to the OPEN again, shouldnt be CLOSE on sell side? buidling blocks OPEN and CLOSE (candle, +daily, +weekly, +monthly) arent negated at all

Attachments
  • Votes +12
  • Project StrategyQuant X
  • Type Bug
  • Status Fixed
  • Priority Normal

History

h
#1

hankeys

22.06.2020 07:50

Task created

h
#2

hankeys

22.06.2020 07:50

Attachment EJ_30_203025158_S_OW_CF_SQX-MM.sqx added

Attachment EJ_30_203025158_S_OW_CF_SQX-MM_EQ___BTvsBT.png added

EJ_30_203025158_S_OW_CF_SQX-MM_EQ___BTvsBT.png
(31.93 KiB)
EJ_30_203025158_S_OW_CF_SQX-MM.sqx
(65.65 KiB)
h
#3

hankeys

22.06.2020 07:50
Voted for this task.
h
#4

hankeys

22.06.2020 07:51

Attachment EJ_H1_200113126_S_HD_CF_SQX-MM.sqx added

Attachment EJ_H1_200113126_S_HD_CF_SQX-MM_EQ___BTvsBT.png added

EJ_H1_200113126_S_HD_CF_SQX-MM_EQ___BTvsBT.png
(31.50 KiB)
EJ_H1_200113126_S_HD_CF_SQX-MM.sqx
(51.22 KiB)
h
#5

hankeys

22.06.2020 07:52

Attachment EJ_H4_200412188_S_CD_CF_SQX-MM_EQ___BTvsBT.png added

Attachment EJ_H4_200412188_S_CD_CF_SQX-MM.sqx added

EJ_H4_200412188_S_CD_CF_SQX-MM.sqx
(67.10 KiB)
EJ_H4_200412188_S_CD_CF_SQX-MM_EQ___BTvsBT.png
(31.77 KiB)
h
#6

hankeys

22.06.2020 07:54
it seems, that stategies are holding their direction, but number of trades is very different in some posted examples and sometimes the RDD values are very different


i think there is some problem with OHLC building blocks - MQL4 and MQL5 code is the same

m
#7

mabi

22.06.2020 09:13
Voted for this task.
m
#8

mabi

22.06.2020 09:35

Attachment slippage.jpg added

Attachment slippage.jpg deleted

Attachment spread.jpg added

spread.jpg
(204.41 KiB)
Test them without spread and the diff is gone and H4 trade instead alot more  atleast in 129 dev 3. 
m
#9

mabi

22.06.2020 10:14
Seems spread logic is backwards thought since the larger spread the easier you get filled and not the other-way around which seems to be the case with SQ.
Dw
#10

Diwi

22.06.2020 11:53
Voted for this task.
E
#11

Emmanuel

22.06.2020 16:42
Voted for this task.
h
#12

hankeys

23.06.2020 07:10

Attachment EJ_30_203025158_S_OW_CF_SQX-MM_EQ___BTvsBT.png added

Attachment EJ_H1_200113126_S_HD_CF_SQX-MM_EQ___BTvsBT.png added

Attachment EJ_H4_200412188_S_CD_CF_SQX-MM_EQ___BTvsBT.png added

EJ_H1_200113126_S_HD_CF_SQX-MM_EQ___BTvsBT.png
(31.96 KiB)
EJ_H4_200412188_S_CD_CF_SQX-MM_EQ___BTvsBT.png
(32.13 KiB)
EJ_30_203025158_S_OW_CF_SQX-MM_EQ___BTvsBT.png
(32.35 KiB)
MABI is right - https://roadmap.strategyquant.com/tasks/sq4_6524


in the MQL code, there is MISSING "MIN. DISTANCE" at all - because in my strategies i have set the min. distance to 5 pips


if i remove the min. distance setting, the backtest is similar


PLEASE RESOLVE ASAP - our trading is DIFFERENT on real accounts :(

OS
#13

trader4711

23.06.2020 07:40
Voted for this task.
b
#14

beppil

23.06.2020 11:24
Voted for this task.
PH
#15

PhoebeHe

23.06.2020 14:16
Voted for this task.
t
#16

Simon

23.06.2020 14:25
Voted for this task.
MA
#17

mrcubl

23.06.2020 14:42
Voted for this task.
m
#18

MT

23.06.2020 14:45
Voted for this task.
o
#19

Enric

23.06.2020 20:00

I remember I had this very same discussion on the Forum with someone who disagreed me. Indeed SOME strats don't fit backtest vs reality. That's a fact.

I never found out where was the problem, but there's some bug somewhere. I thought it was about any indicator, anyway I didn't invest too much time on that. I follow each and everyone of the strats I have live. Those ones that don't match with backtest I discard them immediatly.

Hopefully, if this is a bug. It was the root of the problem and gets fixed. I left behind very nice strats because of that.

Voted!

o
#20

Enric

23.06.2020 20:00
Voted for this task.
PO
#21

traderpetr

23.06.2020 21:32
Voted for this task.
TB
#22

Tomas Brynda

24.06.2020 10:43

Status changed from New to Refused

Hi guys,

the min. distance setting is there to match the settings and behaviour of a given symbol in real trading platforms.

You should set it in a way that it reflects the settings you have on your broker's symbol. It was not intended to use it as some filtering option that eliminates orders too close to current price in MQL code.

If you think it would be useful to use it the way that it doesn't execute orders too close to current price even if the real symbol's settings in MT would allow the orders, please add a feature request for this.


Regarding the negations - Yes it was inteded that Open is negated as Open instead of Close. We will check the negation settings of the other price blocks.


Best regards,

Tomas

h
#23

hankeys

24.06.2020 11:16
anglicky už to psát nehodlám - nechápu jak se tady kolem takové základní věci vůbec máme bavit...je to furt jakási teorie, ale na reálných účtech jsme v praxi


v SQX je nějaké nastavení, v MQL kodu to není implementované

tudíž, když ho použiju v SQX, tak očekávám že moje obchodování to bude reflektovat a je jedno, jestli jsem to nastavení použil kvůli omezení brokera, nebo kvůli nějaké moji idee, aby vstup do obchodu nebyl moc blízko aktuální ceny a tudíž fakticky nemám STOP příkaz, ale spíše MARKET a že takové strategie jsou snad nemusím vysvětlovat, nebo tady nikdo fakticky neobchoduje?


rozdíly ve strategiích nejsou malé a co teď s tím mám dělat, všechny strategie vymazat a začít generovat znovu a vyhodit tak třeba půl roku práce?

o
#24

Enric

24.06.2020 12:59

Come on! We have a field that eliminates orders from Backtest, but on live does nothing. That's insane

TB
#25

Tomas Brynda

24.06.2020 13:17
I will reply in English so that others can understand too.


In our MQL code there is no implementation of min. distance, because it is already there. It is a symbol's property in MT. 

Even in SQ3 the min. distance was not meant to be limiting order execution. It is there so you could make a reliable backtest with the settings matching your trading platform.

If you export strategy from SQ3 and make a backtest in MT, an info message will be logged into journal in case of different min. distance between SQ and MT's chart. 

It is there just to inform you that you have used other min. distance value than the MT's symbol has preset and the result may differ because of that.


We have preserved this functionality in SQX just as it was in old SQ.


I don't think your half year work is useless if you used different min distance setting in your generation process. You should just try to make a reliable retest with the right settings matching the settings in your target platform if you want to get the most reliable results in SQ.

h
#26

hankeys

24.06.2020 13:29
and whats this integer in the SQ3 MQL code?


int MinDistanceOfStopFromPrice = 5.0;

TB
#27

Tomas Brynda

24.06.2020 13:47

Status changed from Refused to In progress

Ok, I must apologize for my mistake.

I was going through SQ3's MQL code finding the occurences of MinDistanceOfStopFromPrice, but haven't noticed this value is then carried into eaStopDifferenceNumber variable which actually limits the orders.

If I take a closer look at the code, I can see it's actually really filtering the orders. I suppose we should make this work the same way in SQX.


I will discuss this with Mark and I think we will implement this kind of logic in release 130.

I will let you know the final decision.


Thank you both for your input.


Best regards,

Tomas

h
#28

hankeys

24.06.2020 15:04

Attachment UJ_15_171589173_S_As_CF_SQX-MM_EQ___BTvsBT.png added

UJ_15_171589173_S_As_CF_SQX-MM_EQ___BTvsBT.png
(26.22 KiB)
nice one, B129 will sound better, because we are already trading and getting different results


1 extreme comparation - RED line - min. distance 5, BLUE line - min. distance 0


i make comparation of all my new 500 strats using min. distance 5 and must tell that difference is almost in every strategy, mostly slight, but sometimes not little high

m
#29

mabi

25.06.2020 03:06
50 % of some types of strategies are gone in my retests with  distance to price =0 especially the simple ones that works on many markets and trade very little suddently becomes hughe loosers when they trade alot. 
h
#30

hankeys

25.06.2020 10:13

Attachment UJ_15_171589173_S_As_CF_SQX-MM.sqx added

UJ_15_171589173_S_As_CF_SQX-MM.sqx
(51.56 KiB)
but i am not getting this HUGE difference between min. distance set to 0 or 5 pips


min. distance 5 - number of trades 279

min. distance 0 - number of trades 5318


yes, the strategy is placing pending orders to the ASK/BID buidling block, so this should be very near the current price, but still i am not getting how only 5 pips for pending order distance could make this huge diff


am i missing something? or there is something strange with SQX backtester?


i would assume that 5 pip diff in pending order distance will now have impact for almost every strategy - because our pedning orders are mostly not so near the current price...


please check the logic of min. distance in SQX backtester, my head doesnt get it...

TB
#31

Tomas Brynda

25.06.2020 11:28

Status changed from In progress to Fixed

The implementation of min. distance application in MQL code is finished, it will be included in the upcoming build.

You will be able to make a retest of your strategies and export new MQL codes which will respect the min distance setting.


The huge difference is only for USDJPY, for EURUSD it doesn't make difference. I think it is symbol related a depends on a ticksize and the structure of the market.


Best regards,

Tomas

h
#32

hankeys

25.06.2020 11:34
but whats behind the USDJPY - if its the setting in pips, what could make the diff between USDJPY and EURUSD pair - it still 5 pips and those markets doesnt have different volatility or range...


OK, we will see in the upcoming fix what we will get


THX

m
#33

mabi

25.06.2020 17:05
 I think it is symbol related a depends on a ticksize and the structure of the market....  No it is type of strategy related.  I lost 50 % of Eu with the same criteria for pass with 0 distance to price against 5 pip distance to price. Then i also trade 30 other instruments .


When will this be implemented in is not in 129 Dev 4


h
#34

hankeys

25.06.2020 18:59
please send us the fix, so we can test it...thx
TB
#35

Tomas Brynda

26.06.2020 10:07

Attachment FixedTemplates.zip added

FixedTemplates.zip
(43.83 KiB)
Unfortunately I can't send you the whole fix, because it also contains Java source code which needs to be compiled inside the app. 


I attached a zip file containing updated templates, you should replace the files inside extend/Code directory.

I hope it will work for you, I'm not 100% sure. It should add this line into the MQL code:


double sqMinDistance = 0.0; //Stop orders min distance from current price


You can manually change the value inside strategy's MQL code to test it. 


The full fix will be available in the next dev build

h
#36

hankeys

26.06.2020 10:31
thx, ill try it
h
#37

hankeys

26.06.2020 12:22
looks good and backtest now match, how long for the next dev?


because in the dev4 this fix is not there


i need to make very quick changes in the real accounts

TB
#38

Tomas Brynda

26.06.2020 12:31
Great, good to hear that. 

We plan to release the new version during next week

m
#39

mabi

26.06.2020 14:11
Excellent !  

Votes: +12

Drop files to upload

or

choose files

Max size: 5MB

Not allowed: exe, msi, application, reg, php, js, htaccess, htpasswd, gitignore

...
Wait please