<- Back to Tasks

Walk Forward Matrix doesn't work in SQX 113

Hi, 

it seems that SQX 113 doesn't calculate properly the equity and the parameters in the Out of Sample periods. 
You can see very well this from the equity chart. As you can see in the attached image, in the In Sample period, the optimized strategy has an equity a lot different from the original equity. Instead, in the out of Sample periods the optimized strategies have the same equity as the original one. The two equities should have different shapes, even in the out of sample periods. It is statistically impossible that they are so similar.
I tried with a lot of strategies and with a lot of different configurations, in all the tests there is this problem.


In this way the results are wrong and Walk Forward it is totally unuseful. 



I am sure you can promptly solve this problem
ask me If you need more information


thankyou very much
Michele  




  


Attachments
2018-10-29_2235.png
2018-10-29_2235.png
(725.20 KiB)
  • Votes +14
  • Project StrategyQuant X
  • Type Bug
  • Status Fixed
  • Priority Normal

History

mp
#1

Michele

30.10.2018 00:07

Task created

mp
#2

Michele

30.10.2018 00:08
Voted for this task.
b
#3

bentra

30.10.2018 00:31
Voted for this task.
b
#4

bentra

30.10.2018 00:37
Something is amiss you're right, the in sample portions can not be so different from each other while OOS is identical.

Interestingly, if you change the default maximum optimizations to something like 1000 from 100, you start to notice small differences is the OOS portions....
m
#5

mabi

30.10.2018 02:03
Voted for this task.
b
#6

bentra

30.10.2018 02:12
nope I was wrong, I accidentally had "trades" selected instead of "time" and it gave the illusion of difference. Even with 1k max optimizations, still the same like in your screen shot.
mp
#7

Michele

30.10.2018 10:11
yes, in the "time" mode you can see that there is no difference between the two equities
o
#8

Enric

31.10.2018 06:52
Voted for this task.
h
#9

hankeys

31.10.2018 07:57
Voted for this task.
k
#10

Karish

31.10.2018 08:31
Voted for this task.
JF
#11

Honzik

31.10.2018 09:09
Voted for this task.
RL
#12

rickliao

31.10.2018 14:50
Voted for this task.
IH
#13

clonex / Ivan Hudec

31.10.2018 17:08
Voted for this task.
l
#14

ludvick

01.11.2018 11:19
Voted for this task.
DR
#15

mentaledge

02.11.2018 19:57
Voted for this task.
m
#16

mabi

03.11.2018 22:40

Attachment 1.jpg added

Also these WF special comparisons is incorrect calculated WF Special percentage of profitable runs  358,8 ?  trades 10.4 and they are not related to the test that is done.
mp
#17

Michele

04.11.2018 12:17

Yes mabi, it seems that the entire process WF doesn't work. Both equity and parameters are not well calculated. 

We are awaiting for the solution for this important task.

WF works fine in SQ3. So i think they are doing it. 

 

YJ
#18

extreme

04.11.2018 16:09
Voted for this task.
Dw
#19

Diwi

06.11.2018 17:58
Voted for this task.
MF
#20

Mark Fric

07.11.2018 15:06
we are making soem fixes here, but I don't see that equity of OOS parts matches original strategy as in the very first screenshot of this task.


Does anybody else experience this problem? If yes, could you attach your strategy and Optimizer config?

mp
#21

Michele

07.11.2018 17:06

Attachment 2018-11-07_1623.png added

Attachment Strategy 518621.sqx added

Attachment WFA Config.cfx added

Ok Mark, i attach another image of the results of another random strategy. I even attach this strategy and the Optimizer configuration file.

I can't send to you the WFM results file because is bigger than 2MB, but if you need it i could send you it in another way.  

In the equity chart select the "Time" mode to see the same equities. 


I tried with a lot of different strategies, this problem always happen. 

If you need more example let me know.


thanks

Michele

N
#22

nathan

09.11.2018 16:43
Voted for this task.
b
#23

bentra

10.11.2018 12:12

Attachment Capture.png added

Actually, the small differences in some equity curves can be explained by trades being cut early, otherwise OOS and IS are always the same in the equity charts. In fact I challenge anyone to post a strategy where the walk forward has a different curve for IS than OOS that cannot be explained by early termination of some trades (because new segment begins)

We are talking about the SHAPE of the equity curve being identical from the point where the first OOS segment begins till the end of the entire sample.  Any strategy, any walkforward except as noted above about a few differences explainable by mid trade terminations (or late entries / missed trades) at or near segment borders.

It appears the bug is caused from the code accidentally using the original settings instead of the newly optimized settings for each OOS segment.
b
#24

bentra

15.11.2018 13:20

Attachment Capture_sample.png added

Attachment Optimize_sample.cfx added

Attachment 20181115_sampleStrategy 048885.sqx added

Attachment 20181115_sampleWF Optimization - Strategy 048885.sqx added

Here is a simple quick sample which produces very different in sample equity line but extremely similar OOS line clearly identifying something is wrong. If the in sample lines are very different from each other then so too should the out of sample lines be very different from each other.

I'm still requesting a screenshot of anyone that can produce a walk forward equity line where the OOS of original and OOS of walk forward a clearly using different variable settings because I have yet to produce this or see this!
MF
#25

Mark Fric

15.11.2018 21:14

Status changed from New to Fixed

I found and identified the problem. There really was a bug that the OOS part was tested using original parameters, not the best optimization result found in IS part.


I apologize for this serious eror.

Votes: +14

Drop files to upload

or

choose files

Max size: 5MB

Not allowed: exe, msi, application, reg, php, js, htaccess, htpasswd, gitignore

...
Wait please