<- Back to Tasks

SQX B138: Optimized results and retested results are not the same

I think I found a severe error:


When I test an optimized strategy in retester and use the same config settings as the optimized strategy (I'm applying strategy config), then the strategy results are not the same. They should be. This is quite frustrating as this means that the optimizer and/or the retester cannot be trusted! Or maybe I am missing something?

Attached are strategies and config files.

In the attached screenshot, look at e.g. net profit. They are very different. The net profit from Optimizer tab is $4728.7. The net profit from Retester tab is $3990. They should be identical.

Thanks.

Regards

Attachments
Retest.cfx
Retest.cfx
(25.88 KiB)
Optimize.cfx
Optimize.cfx
(25.57 KiB)
Strategy 25117 Builder.sqx
Strategy 25117 Builder.sqx
(106.79 KiB)
Strategy 25117 - Sequential optimized.sqx
Strategy 25117 - Sequential optimized.sqx
(75.86 KiB)
Build.cfx
Build.cfx
(22.96 KiB)
image-0.png
image-0.png
(19.80 KiB)
Strategy 25117 - Sequential optimized Retester.sqx
Strategy 25117 - Sequential optimized Retester.sqx
(110.51 KiB)
  • Votes +1
  • Project StrategyQuant X
  • Type Bug
  • Status New
  • Priority Normal

History

d
#1

Hamselv

19.12.2023 10:52

Task created

TT
#2

Tamas

19.12.2023 11:30

Attachment image-0.png added

Attachment image-1.png added

image-1.png
image-1.png
(599.70 KiB)
image-0.png
image-0.png
(531.42 KiB)
Hello, 

yes, the backtests should match. 

It works fine for me, screenshots attached.


Tamas

d
#3

Hamselv

19.12.2023 13:13
Hi Tamas. Please try this:


1. Load strategy called "Strategy 25117 - Sequential optimized" into Retester. Net Profit should show $4728.7.

2. Go to "Strategy config" tab and press the button "Apply strategy config" button.

3. Run the Retester.

4. Net Profit is now $3990. Why has it changed? Shouldn't it still be $4728.7 as before?


The strategy is created straight out of the box from the "Getting started" tab, so data and all settings are default that comes with SQX.


Regards

d
#4

Hamselv

19.12.2023 13:33
Let me add:


I just tried again with a fresh unzip of SQX:


1. Created a forex strategy from the "Getting started" tab without changing any settings.

2. Then I did a sequential optimization changing "Value distribution" to Up=20, Down=20 & steps=4.

3. Retested the optimized strategy using "Apply strategy config".


The Net Profit changes after retesting. Shouldn't is be unchanged?


Regards

d
#5

Hamselv

12.01.2024 01:47

Attachment image-0.png added

Attachment Strategy 36102.sqx added

Attachment Strategy 36102 - Sequential optimized.sqx added

Attachment Retested - Strategy 36102 - Sequential optimized.sqx added

image-0.png
image-0.png
(26.53 KiB)
Strategy 36102.sqx
Strategy 36102.sqx
(73.42 KiB)
Strategy 36102 - Sequential optimized.sqx
Strategy 36102 - Sequential optimized.sqx
(81.00 KiB)
Retested - Strategy 36102 - Sequential optimized.sqx
Retested - Strategy 36102 - Sequential optimized.sqx
(78.43 KiB)
Let me try again to explain the problem in a simple way: The attached screenshot shows a strategy that is first being optimized (sequential) and afterwards retested using the same strategy config. When retested the Net profit is not the same anymore which is strange (green boxes in the screenshot). All optimization parameters used are the default settings. It looks like the sequential optimizer is somehow buggy/broken.


Regards



Votes: +1

Drop files to upload

or

choose files

Max size: 5MB

Not allowed: exe, msi, application, reg, php, js, htaccess, htpasswd, gitignore

...
Wait please