[Build138][Major Bug, Fix & Feature Suggestion] Old Bug that still not been fixed (Makes the Walk-Forward, SeqOpt & SPP results WORTHLESS..)

[Build138][Major Bug, Fix & Feature Suggestion] Old Bug that still not been fixed (Makes the Walk-Forward, SeqOpt & SPP results WORTHLESS..)

In an older Task here:
https://roadmap.strategyquant.com/tasks/sq4_9989
i addressed a bug or a missing optional feature that should be available to the SQX user,

This is what i'm talking about with a specific example for Walk-Forward:
When performing a Walk-Forward Optimization, there should be a feature just like with TradeStation as we all know,
that will allow us to chose if we want to use a "Genetic Optimization" or an "Exhaustive Optimization".

Currently with SQX it is set to 'Genetic Optimization' by Default, with no possibility to be changed, and its sucks, ill explain why..,
When performing a Walk-Forward Optimization we MUST use "Exhaustive Optimization" for the sole reason that the concept of the Walk-Forward Optimization method is to Repeat the process in the future in order to keep the strategy adaptable to newer market conditions,
in order to do that we must have the same Settings, Data + Newly added data,
We CANNOT use the Genetic Algorithm in order to perform the Walk-Forward Optimization method because it would NOT be repeated, it will always start at random and evolve,
hence every time we will Optimize or Re-optimize our strategy it will have different results, which is INCORRECT.
We need to use all available optimization combinations ("Exhaustive Optimization") in order to make any conclusions from the Walk-Forward Optimization results
and if we plan to re-optimize out strategy periodically and follow the Walk-Forward Optimization methodology we must use "Exhaustive Optimization" when re-performing this optimization in the future or just want to re-test the strategy again.

You can try it for yourself..,
Build a strategy, perform a WF test on it,
now re-test that same strategy using exactly the same settings and data (send to the retester),
perform WF test on that same exact strategy again,
check out the results, they are different then the previous WF results you had before that,
try to perform another WF test on it,
annnnnd the results you have now is totally different,
do it again, and again and again, each time you will have different results, that officially makes the WF optimization test simply W O R T H L E S S.

As a matter effect, i tried to replicate this process on SeqOpt & on SPP, guess what?, same thing happens, the results change from test to test, these tests are all just a waste of time i'm sorry, but sadly it is the truth..,
If we want to use SeqOpt we want to do this across all available combinations with (min,max,steps, etc..) and not have a random final result with each test we re-run..
if we want to use SPP, again.., we want to do this across all available combinations with (min,max,steps, etc..) and not have a random final result with each test we re-run..

i hope i explained it all well, and you see the problem, Because it is a problem, and should be addressed.



Addressing and Fixing the problem:
I attached some illustrations on how to simply and easily fix this problem,
In the attached image you can see that i brought the attention to TradeStation,
they offer 2 "Optimization Methods" which are "Genetic Optimization" or "Exhaustive Optimization".

Below that, i manipulated the settings menu of each CrossCheck Test settings window, now it has "Optimization Method" drop-down menu, which will contain 2 options, just like in TradeStation - "Genetic Optimization" or "Exhaustive Optimization",
the user can choose if he wants to keep using the "Genetic Optimization" (for some odd unknown reason..) or "Exhaustive Optimization" which is the correct way to go with when using these robustness tests in my opinion and not only my opinion,
thats basically it, this will fix the problem.

Another thing that i want to address is the areas i show in RED - "Maximum tests",
Notice that SeqOpt is the only CrossCheck that does not have that option out of all the rest CrossChecks for some reason,
the "Maximum tests" section is unnessesery, we already got "Value Distribution (UP/DOWN/MAX STEPS)" section that will do the job of limiting the ranges,
there is no real reason to have the "Maximum tests" section,
notice how in SeqOpt it was done right,
the strategy's parameters are limited by UP % & DOWN % and then by MAX STEPS, thats it, simple, we dont need anything else no more, it will cover everything needed to perform all needed iterations we asked for,
the "Maximum tests" section should have been removed after the "Value Distribution (UP/DOWN/MAX STEPS)" section was introduced a few Builds back, but for some odd reason it is still there, messing with the "Value Distribution (UP/DOWN/MAX STEPS)" most likely...., this must be removed.

In Summery:
PLEASE FIX THIS PROBLEM,
• Allow the USER to have an option to use "Genetic Optimization" or "Exhaustive Optimization".
• REMOVE the "Maximum tests" section field, this is should have been removed after the "Value Distribution (UP/DOWN/MAX STEPS)" section was introduced a few Builds back, but for some odd reason it is still there, SeqOpt is the only CrossCheck test that does not got it and the parameters distribution work well.

---
Another thing is if SQX uses a Genetic Algorithm when optimizing the strategy which actual settings does it use?,
is it the Main settings from the "Genetic options" Tab? or are there some other genetic settings that are working behind the scenes?, still unsure about that question, kindly provide some info on the subject.



Attachments
Bug Fix.png
(501.54 KiB)
  • Votes +14
  • Project StrategyQuant X
  • Type Bug
  • Status Fixed
  • Priority Normal

History

k
#1

Karish

17.03.2024 23:44

Task created

k
#2

Karish

17.03.2024 23:45
Voted for this task.
t
#3

tnickel

17.03.2024 23:54
Voted for this task.
CG
#4

Chris G

18.03.2024 01:01
Voted for this task.
TD
#5

tony.davo93@gmail.com

18.03.2024 02:13
Voted for this task.
TM
#6

Toni

18.03.2024 12:20
Voted for this task.
A
#7

Asoea

18.03.2024 16:54
Voted for this task.
IH
#8

clonex / Ivan Hudec

20.03.2024 10:56
Voted for this task.
TR
#9

Tim

21.03.2024 01:12
Voted for this task.
BT
#10

EndUser

22.03.2024 21:41
Voted for this task.
DP
#11

danielpavone@gmail.com

27.03.2024 17:28
Voted for this task.
m
#12

mabi

27.03.2024 22:16
Voted for this task.
RL
#13

rickliao

31.03.2024 11:37
Voted for this task.
y
#14

ytu

02.04.2024 04:40
Voted for this task.
b
#15

blonado

14.04.2024 15:30
Voted for this task.
TT
#16

Tamas

18.04.2024 22:41

Status changed from New to Fixed

Attachment image-0.png added

image-0.png
(222.41 KiB)
We have added the option to select 'Exhaustive (all combinations)' within the Max tests dropdown menu. A screenshot is attached
k
#17

Karish

18.04.2024 22:57
Awesome!, Thanks, waiting for the next beta release to test it out! :)

Something i was noticing is that if i was using even Millions of "Maximum Tests" with 50% UP/DOWN and 5 STEPS, is that for some reason it was generating a huge amount of permutations,
rather than what i was expecting,

i wonder if it would be fixed now with this "Exhaustive" option.


I also wonder if WF optimization will now use all possible permutations with the "Exhaustive" method,
that mean if i will re-optimize the strategy again and again with the same settings as before i did it a few minutes ago, will it show me the same results, or will it show me a different result (genetically optimized),
this is super important.

thanks.

Votes: +14

Drop files to upload

or

choose files

Max size: 5MB

Not allowed: exe, msi, application, reg, php, js, htaccess, htpasswd, gitignore

...
Wait please