[Build138][Major Bug, Fix & Feature Suggestion] Old Bug that still not been fixed (Makes the Walk-Forward, SeqOpt & SPP results WORTHLESS..)
In an older Task here:
https://roadmap.strategyquant.com/tasks/sq4_9989i addressed a bug or a missing optional feature that should be available to the SQX user,
This is what i'm talking about with a specific example for Walk-Forward:
When performing a Walk-Forward Optimization, there should be a feature just like with TradeStation as we all know,
that will allow us to chose if we want to use a "Genetic Optimization" or an "Exhaustive Optimization".
Currently with SQX it is set to 'Genetic Optimization' by Default, with no possibility to be changed, and its sucks, ill explain why..,
When performing a Walk-Forward Optimization we MUST use "Exhaustive Optimization" for the sole reason that the concept of the Walk-Forward Optimization method is to Repeat the process in the future in order to keep the strategy adaptable to newer market conditions,
in order to do that we must have the same Settings, Data + Newly added data,
We CANNOT use the Genetic Algorithm in order to perform the Walk-Forward Optimization method because it would NOT be repeated, it will always start at random and evolve,
hence every time we will Optimize or Re-optimize our strategy it will have different results, which is INCORRECT.
We need to use all available optimization combinations ("Exhaustive Optimization") in order to make any conclusions from the Walk-Forward Optimization results
and if we plan to re-optimize out strategy periodically and follow the Walk-Forward Optimization methodology we must use "Exhaustive Optimization" when re-performing this optimization in the future or just want to re-test the strategy again.
You can try it for yourself..,
Build a strategy, perform a WF test on it,
now re-test that same strategy using exactly the same settings and data (send to the retester),
perform WF test on that same exact strategy again,
check out the results, they are different then the previous WF results you had before that,
try to perform another WF test on it,
annnnnd the results you have now is totally different,
do it again, and again and again, each time you will have different results, that officially makes the WF optimization test simply W O R T H L E S S.
As a matter effect, i tried to replicate this process on SeqOpt & on SPP, guess what?, same thing happens, the results change from test to test, these tests are all just a waste of time i'm sorry, but sadly it is the truth..,
If we want to use SeqOpt we want to do this across all available combinations with (min,max,steps, etc..) and not have a random final result with each test we re-run..
if we want to use SPP, again.., we want to do this across all available combinations with (min,max,steps, etc..) and not have a random final result with each test we re-run..
i hope i explained it all well, and you see the problem, Because it is a problem, and should be addressed.
Addressing and Fixing the problem:
I attached some illustrations on how to simply and easily fix this problem,
In the attached image you can see that i brought the attention to TradeStation,
they offer 2 "Optimization Methods" which are "Genetic Optimization" or "Exhaustive Optimization".
Below that, i manipulated the settings menu of each CrossCheck Test settings window, now it has "Optimization Method" drop-down menu, which will contain 2 options, just like in TradeStation - "Genetic Optimization" or "Exhaustive Optimization",
the user can choose if he wants to keep using the "Genetic Optimization" (for some odd unknown reason..) or "Exhaustive Optimization" which is the correct way to go with when using these robustness tests in my opinion and not only my opinion,
thats basically it, this will fix the problem.
Another thing that i want to address is the areas i show in RED - "Maximum tests",
Notice that SeqOpt is the only CrossCheck that does not have that option out of all the rest CrossChecks for some reason,
the "Maximum tests" section is unnessesery, we already got "Value Distribution (UP/DOWN/MAX STEPS)" section that will do the job of limiting the ranges,
there is no real reason to have the "Maximum tests" section,
notice how in SeqOpt it was done right,
the strategy's parameters are limited by UP % & DOWN % and then by MAX STEPS, thats it, simple, we dont need anything else no more, it will cover everything needed to perform all needed iterations we asked for,
the "Maximum tests" section should have been removed after the "Value Distribution (UP/DOWN/MAX STEPS)" section was introduced a few Builds back, but for some odd reason it is still there, messing with the "Value Distribution (UP/DOWN/MAX STEPS)" most likely...., this must be removed.
In Summery:
PLEASE FIX THIS PROBLEM,• Allow the USER to have an option to use "Genetic Optimization" or "Exhaustive Optimization".
• REMOVE the "Maximum tests" section field, this is should have been removed after the "Value Distribution (UP/DOWN/MAX STEPS)" section was introduced a few Builds back, but for some odd reason it is still there, SeqOpt is the only CrossCheck test that does not got it and the parameters distribution work well.
---
Another thing is if SQX uses a Genetic Algorithm when optimizing the strategy which actual settings does it use?,
is it the Main settings from the "Genetic options" Tab? or are there some other genetic settings that are working behind the scenes?, still unsure about that question, kindly provide some info on the subject.