129.302 SQX produces MultiTF MC strategies with drastically different backtests than MultiCharts

I have been working with MultiTF strategies in SQX for the past three months and I have finally been able to test them in MC to try and verify their results. Unfortunately, none of the backtests are accurate at all. I can't say I expected them to work because the strategies have "unreal" performance and that was definitely a red flag. However, it is clear that there is a big difference between the engine in MC and the engine in SQX for testing MultiTF strategies. All of these strategies managed to pass my robustness tests and did not show anything particularly wrong with them which makes me believe it is an error in some of the indicators compiled into MC or it is the engine itself. Either way, this is a serious problem because these strategies were favored in the build process and I can't train to find good strategies if ones like these are what I have to select from. 



I decoupled all of the strategies into the long and short side so it might be easier to identify which parameters are causing the issues. The strategies are in the zip file. 

Attachments
Screen Shot 2020-10-06 at 105011 PM.png
(1.23 MiB)
Screen Shot 2020-10-06 at 105239 PM.png
(1.11 MiB)
Screen Shot 2020-10-06 at 105304 PM.png
(1.07 MiB)
Screen Shot 2020-10-06 at 104917 PM.png
(1.09 MiB)
Screen Shot 2020-10-06 at 104931 PM.png
(1.16 MiB)
Screen Shot 2020-10-06 at 104849 PM.png
(1.12 MiB)
Screen Shot 2020-10-06 at 104940 PM.png
(1.20 MiB)
Screen Shot 2020-10-06 at 105119 PM.png
(1.03 MiB)
Screen Shot 2020-10-06 at 105020 PM.png
(1.22 MiB)
Screen Shot 2020-10-06 at 104859 PM.png
(1.17 MiB)
Screen Shot 2020-10-06 at 105509 PM.png
(1.11 MiB)
Screen Shot 2020-10-06 at 104952 PM.png
(1.37 MiB)
Screen Shot 2020-10-06 at 105543 PM.png
(1.03 MiB)
Screen Shot 2020-10-06 at 104833 PM.png
(976.58 KiB)
MultiTF.zip
(4.18 MiB)
Screen Shot 2020-10-06 at 105000 PM.png
(1.33 MiB)
  • Votes +1
  • Project StrategyQuant X
  • Type Bug
  • Status Fixed
  • Priority Normal

History

KL
#1

kainc301

07.10.2020 05:14

Task created

KL
#2

kainc301

08.10.2020 22:34

Attachment MultiTF Futures Short.cfx added

Attachment Futures_Long_MultiTF.cfx added

Futures_Long_MultiTF.cfx
(28.20 KiB)
MultiTF Futures Short.cfx
(28.13 KiB)
b
#3

beppil

18.10.2020 10:57
Voted for this task.
TB
#4

Tomas Brynda

04.03.2021 20:09

Status changed from New to Fixed

Attachment results.zip added

results.zip
(3.83 MiB)
Hi Kainc301,

we have made some progress here as well.

I backtested your strategies again with the latest code and now I was able to get very similar results.

Here are a few points:


  • First of all I ignored strategies containing Volume block, because they probably won't work very well. I have bad experience with those strategies as they are very sensitive to data
  • I changed M30 timeframes to H1 because when using multi TF strategies in MC, main chart should have the lowest TF of all charts in order to work properly
  • In MC I was using the exact same data as in SQ (exported from SQ -> imported to MC) and using the exact same session everywhere
  • Testing this kind of strategies is very time consuming, so I tested all strategies using the same data setup - ES H1 / NQ H1 / QR H1 / YM D1 

I attach the results so you can check. I haven't included tradelists as they have many thousands of trades. 

In case of YM_1H_L the results are still not very good. It is most probably caused by the presence of LowDaily is rising condition in exit signal. 

Daily blocks don't work well in conjuction with is rising/ is falling. It is problematic because of EasyLanguage syntax. We know about this issue and it is on our list to do.


I think the multiTF backtests are much more reliable now. You should be able to get good results comparing backtests in B131.


Best regards,

Tomas



KL
#5

kainc301

04.03.2021 21:08
Thank you, Tomas. A couple of things to note.


1. Please specify in the documentation that the base timeframe should be the lowest in order to work properly. I previously thought it could serve as an advantage but I was not sure if it was causing a "look-ahead bias" of some sort. I figured it wouldn't but if this is a limitation of Easy language, it would be great to know in the docs. 

2. Can you provide any documentation we have on importing data into Multicharts from SQX? I don't believe I did that 

3. Thank you for the heads up about daily blocks in conjunction with rising/falling. But is there anywhere (like a forum thread) that has a list of different indicators we should not use together? Any means to avoid these types of these edge cases ahead of time would be great


And by the way, you're the best Tomas! This was driving me crazy and I haven't been building anything since the summer because of this so I appreciate the investigation.  

TB
#6

Tomas Brynda

05.03.2021 08:05
  1. Ok, we will add it. You should always use the lowest timeframe on main chart, because otherwise the study is called multiple times on each bar (depending on the lowest TF subchart) and it causes problems.
  2. We don't have a documentation for this. You just export SQ's data from Data Manager and import them into custom ASCII symbol in MC. But this was just a quick way for me to ensure I will have matching data in both platforms. It's completely fine to do it according to our article here https://strategyquant.com/doc/strategyquant/reliable-backtesting-in-tradestation-multicharts/ (export MC data and import it into SQ)
  3. Maybe we should publish some article with good practises. Basically I have these findings:
  • Don't use Volume and AverageVolume blocks in strategies. It is too data/broker sensitive  
  • Be careful when using Fractals indicator. It outputs zero values which is not good for open price and SL/PT calculations
  • Is rising/ is falling is currently not working well in conjuction with Daily/Weekly/Monthly blocks in easylanguage source code. It doesn't apply shift correctly. We will focus on this in further releases
TB
#7

Tomas Brynda

05.03.2021 08:59
Here is a link to a new brief article https://strategyquant.com/doc/strategyquant/best-practices-for-multi-tf-strategies-backtesting-and-trading/

We will be updating it if needed.


Best regards,

Tomas


Votes: +1

Drop files to upload

or

choose files

Max size: 5MB

Not allowed: exe, msi, application, reg, php, js, htaccess, htpasswd, gitignore

...
Wait please