Strategies that Passes "SeqOpt" after parameters changed shows very different results after a Retest.

Attached Screenshot of one of many strategies that got the same problem,

Strategy's SQX file is also attached.

Thanks.. waiting for any reply that confirms if that is a bug or not..

From what @Hankeys from Discord "SPP - basic thing doenst work - used params from SPP making totally different backtest"

Please note that one in too..

What you see in databank in Builder is a result of the first (main) backtest that was performed on an originally generated strategy BEFORE the seq. optimization was performed.

So the main backtest was made on original set of parameters.

If you chose to apply params to strategy in seq. opt. crosscheck config the parameters are changed after the crosscheck, and so when you retest this strategy in Retester the results will be different - because the strategy already has new optimized parameters.

I'm not sure what exactly should be done here - perhaps we should replace the main backtest results with the result of backtest after seq opt change of parameters was applied.

Or, we could accept that it behaves like this, that it is not a bug.

What would you prefer?

I agree with @hankeys about that the SeqOpt feature should respect the "Trading Options" values.

We should checkout the new Beta or the Official release of the new 134 build with the fixed SeqOpt to really make sure it works as it should,

Cause right now on the current build something very odd happens,

How it should really work:
User performs SeqOpt on an Original strategy (*With auto-apply new parameters),
The strategy should show us the updated results not of the Original strategy but rather the results of the UPDATED strategy and it should save the results into that strategy,

like if we take that updated strategy into a re-tester or any other place in SQX the results of the UPDATED strategy should be taken into account,

Currently i dont know why, SQX still takes into account the Original results of the strategy EVEN THOUGH we clearly performed SeqOpt and it did apply the new parameters and showed us the updated results, and after all that if we take that updated strategy into the re-tester and perform a retest it suddenly shows us the results of the Original strategy as if the SeqOpt and the new parameters have not been performed at all...............

And thats a problem.

What you see in databank in Builder is a result of the first (main) backtest that was performed on an originally generated strategy BEFORE the seq. optimization was performed.

So the main backtest was made on original set of parameters.

If you chose to apply params to strategy in seq. opt. crosscheck config the parameters are changed after the crosscheck, and so when you retest this strategy in Retester the results will be different - because the strategy already has new optimized parameters.

I'm not sure what exactly should be done here - perhaps we should replace the main backtest results with the result of backtest after seq opt change of parameters was applied.

Or, we could accept that it behaves like this, that it is not a bug.

Seq opt in Builder or Retester should be used only as cross check, cross checks should verify only strategy robustness, but they should not modify the original strategy.

Replacing the main result with result after sequential optimization is possible, but it will bring other problems - all cross checks that were performed before Seq. Opt. crosscheck were performed on the original strategy with original parameters. So they are no longer valid if you change parameters in Seq. optimization crosscheck.

To repeat - I think strategy generated in Builder backtest process should be immutable, it shouldn't change in the middle of cross checks.

If you want to perform seq opt on a strategy you should do it later, in step after build.

You can make a simple custom project with a few steps for this, where you configure it the way you want.

As for respecting Trading options - it works as same as all other optimizations. Min/Max SL/PT options are respected during backtest, but they don't influence how your parameters are set in optimization.

What Seq opt filtering mean - strategy passes if a given % of parameters has stable areas. If parameter has no stable area it means that the results fluctuate very much with every step in the parameter value, which is probably not good. We did not measure yet how good/bad this is as a robustness test, it is something I plan to do. 

Look at the doc for better explanation: https://strategyquant.com/doc/strategyquant/sequential-optimization/#how-the-stable-value-of-a-parameter-is-chosen

With all the respect @Mark you are wrong on that last reply.

This is non-sense,
All my project depends heavy on SeqOpt and that problem to be fixed,

This is the best way around it:

User use SeqOpt and prompted with 2 choices like it already is right now on build 133 = Auto-apply new parameters (ON/OFF),

• If ON, this WILL change the parameters of the strategy regardless of other things, SQX from now on ALWAYS take into consideration those new parameters and throw out the Original values!.

• if OFF, this WILL NOT change the parameters of the strategy, and the SeqOpt feature will act as a Test rather than a Test & Act.

This will solve the problem.

I dont see a reason why a user that use SeqOpt IN ORDER to find new parameters for his strategy (using Auto-apply new parameters==ON) will take that new strategy with those new updated parameters and make other tests in SQX and SQX will calculate everything based on the Original parameters of that same strategy again..., this is just non-sense!,

Hence if a user chose to use "Auto-apply new parameters=ON" it should do what it should do. (perform as a test & action to change parameters.)

and if the user chose to use "Auto-apply new parameters=OFF" it should do what it should do. (perform just as an test.)

My whole project depends on this fix, and the whole feature purpose of SeqOpt was initially to change parameters of the strategy and continue the development with that new updated strategy parameters in consideration on other steps and tasks.

Don't you see what's the problem? Crosschecks are executed one after another, the process is as follows:

1. Strategy is generated and backtested (first, main backtest)

2. (optional) What If simulations crosscheck

3. (optional) MC trades manipulation crosscheck

4. (optional) Higher backtest crosscheck

5. (optional) Additional markets crosscheck

6. (optional) MC Retest crosscheck

7. (optional) Seq. Opt. crosscheck <- here you'll change the strategy parameters in Apply, so points 1.-6. will be no longer valid, because the strategy has changed.
Even if we'll replace the main backtest (point 1.), the rest of them (2.-6.) will be still invalid.

I don't see what kind of project you have that relies on this, but I'd say you are using it incorrectly. You can use Seq. optimization also in Optimizer.

It is simple to create a custom project workflow where you'll first build your strategies, then optimize them, then optionally retest and reevaluate them again.

If you need help with this kind of project I can help you, but I really don't think it should change the way you want in crosscheck.

I'm looking forward to see what others think about this.

3. (optional) What If simulations crosscheck

4. (optional) MC trades manipulation crosscheck

5. (optional) Higher backtest crosscheck

6. (optional) Additional markets crosscheck

7. (optional) MC Retest crosscheck

In simple words, we created a strategy, we used SeqOpt we used Auto Apply new parameters, now we got a new strategy with new parameters that we want to continue those following steps with it:

3. (optional) What If simulations crosscheck

4. (optional) MC trades manipulation crosscheck

5. (optional) Higher backtest crosscheck

6. (optional) Additional markets crosscheck

7. (optional) MC Retest crosscheck

BUT!, the problem is that SQX will IGNORE the new parameters EVEN THOUGH the new parameters clearly applied to the strategy (Check "Source Code" Tab you will see those new parameters),
so SQX will IGNORE the new parameters and keep on using the Original parameters rather than the new parameters of that strategy,

so all those following tasks:

3. (optional) What If simulations crosscheck

4. (optional) MC trades manipulation crosscheck

5. (optional) Higher backtest crosscheck

6. (optional) Additional markets crosscheck

7. (optional) MC Retest crosscheck

Are IRRELEVANT!

If you want to do it in the order you explained then why don't you do it in custom project?

Task 1 - Build, no crosschecks

Task 2 - Optimize with Seq Opt

Task 3 - Retest, apply all the remaining crosschecks on strategies with Seq Opt optimized parameters

You have no option to change the order of crosschecks in Builder, and I don't think it is a good idea to put Seq. Opt. As a relatively slow crosscheck to the first place.

Again, Like i mentioned here on post #24: https://roadmap.strategyquant.com/tasks/sq4_8212/edit#comment-94132

The very simple solution to this problem should be a selection that will consider the following:
• Apply new parameters to the strategy.
• DO NOT apply new parameters to the strategy.

3. (optional) What If simulations crosscheck   **Taking the Parameters into account rather than the Original Parameters.

4. (optional) MC trades manipulation crosscheck   **Taking the Parameters into account rather than the Original Parameters.

5. (optional) Higher backtest crosscheck   **Taking the Parameters into account rather than the Original Parameters.

6. (optional) Additional markets crosscheck   **Taking the Parameters into account rather than the Original Parameters.

7. (optional) MC Retest crosscheck   **Taking the Parameters into account rather than the Original Parameters.

8. (optional) Retests etc...........   **Taking the Parameters into account rather than the Original Parameters.

-------------------------------

If the user will select ("DO NOT apply new parameters to the strategy.") the following will happen:
1. Strategy is generated and backtested (first, main backtest)
2. (optional) Seq. Opt. crosscheck

3. (optional) What If simulations crosscheck   **Original Parameters.

4. (optional) MC trades manipulation crosscheck    **Original Parameters.

5. (optional) Higher backtest crosscheck     **Original Parameters.

6. (optional) Additional markets crosscheck    **Original Parameters.

7. (optional) MC Retest crosscheck    **Original Parameters.

8. (optional) Retests etc...........    **Original Parameters.

Simple as that, It should like so already in this current 133 build,
Problem solved........

Waiting for your confirmation Mark, Thanks.

I got you, but the order of crosschecks is the way I wrote it, it cannot be changed. It is ordered from fastest to slowest.

If you want to do it in the order you explained then why don't you do it in custom project?

Task 1 - Build, no crosschecks

Task 2 - Optimize with Seq Opt

Task 3 - Retest, apply all the remaining crosschecks on strategies with Seq Opt optimized parameters

You have no option to change the order of crosschecks in Builder, and I don't think it is a good idea to put Seq. Opt. As a relatively slow crosscheck to the first place.

Ill check it again and give you my feedback, i remember i tried many ways, give me few minutes..

Please check out the following (Attached..)

I dont know what causing this issue......

Here is another example, Judge by the Name of the screenshots.

Why does it shows different results, what actually changed from taking a strategy that passed SeqOpt with new parameters, taking the same strategy with the same data, period, TF, and just re-test it, and waaaammm its just went all crazy wrong and different. What the hell..., i dont know what's going on behind the scenes there in SQX but something is deffently not right, Hence i cannot continue my project like that, because i just take that same strategy after SeqOpt and just perform a re-test, doing nothing more than that, and its all go crazy on me

Then i just use Automatic retest.

Essentially the sequential optimization was not worked as supposed, it always optimized one parameter against the original strategy, instead of progressively updating strategy parameters one by one after each optimization step.

Karish, thank you for pointing this out and for your persistence on the matter.