<- Back to Tasks

129 dev 6 Optimizer Produces Inaccurate Results

I don't know if I'm doing anything stupid but I ran an optimization for a strategy I made for ES on the MultiCharts engine and the original result I got was a net profit of 25k. When I re-ran this same optimization in the retester, I got a result that was substantially less. I don't understand why this would happen other than it being a bug. I am attaching the original strategy along with the optimization. 
Attachments
Strategy 527119.sqx
Strategy 527119.sqx
(50.46 KiB)
Strategy 527119 - Optimization 036570.sqx
Strategy 527119 - Optimization 036570.sqx
(52.27 KiB)
  • Votes 0
  • Project StrategyQuant X
  • Type Bug
  • Status Refused
  • Priority Normal

History

KL
#1

kainc301

12.07.2020 05:08

Task created

KL
#2

kainc301

12.07.2020 05:10

Attachment Screen Shot 2020-07-11 at 110743 PM.png added

Attachment Screen Shot 2020-07-11 at 110728 PM.png added

Screen Shot 2020-07-11 at 110728 PM.png
Screen Shot 2020-07-11 at 110728 PM.png
(593.24 KiB)
Screen Shot 2020-07-11 at 110743 PM.png
Screen Shot 2020-07-11 at 110743 PM.png
(592.99 KiB)
Here are my results for the original optimization and then the retest , using the full range of data both times.
KL
#3

kainc301

12.07.2020 05:36

Attachment Screen Shot 2020-07-11 at 113303 PM.png added

Attachment Screen Shot 2020-07-11 at 113410 PM.png added

Screen Shot 2020-07-11 at 113410 PM.png
Screen Shot 2020-07-11 at 113410 PM.png
(167.61 KiB)
Screen Shot 2020-07-11 at 113303 PM.png
Screen Shot 2020-07-11 at 113303 PM.png
(178.55 KiB)
I think that a potential culprit is that the trade options are optimized to nonsense values. I am optimizing using the recommended settings. For some reason both exit trades on friday and limit time signals is set to false and the time values given for the time ranges are nonsensical. 
KL
#4

kainc301

16.07.2020 12:18
Please also check if this happens in the Crosscheck "Opt Profile / System Param Permutation" I currently don't see a away to check the optimizations of the strategies from that test and if that cross check is having the same issue then it is killing a lot of good strategies by generating incorrect results from the optimization.
MF
#5

Mark Fric

16.07.2020 13:52

Status changed from New to Refused

Attachment diffs.png added

diffs.png
diffs.png
(107.04 KiB)
You have several differences in the backtesting setting between these two strategies - see the screenshot in red.


Last settings is from one strategy and Project config from another one.


I recommend you to use Results -> Strategy config and button Apply from strategy when you load some strategy to optimizer, to exactly match your setting from last build or retest. 

KL
#6

kainc301

16.07.2020 14:12
You are right Mark. Applying the settings worked. I did not notice that the settings did not carry over in the original test which skewed the results. Sorry for the user error. Thank you for looking into this.
Votes: 0

Drop files to upload

or

choose files

Max size: 5MB

Not allowed: exe, msi, application, reg, php, js, htaccess, htpasswd, gitignore

...
Wait please