<- Back to Tasks

[Build 136] The backtest results have changed dramatically.

1.When many of the strategies I developed on version 136dev5 were backtested again in the 136RC3, the results  changed dramatically. The performance of the vast majority of strategies has become abysmal.All the settings are exactly the same, the same data, and I can't find any reason.I don't know if this has happened to anyone else.  For your convenience, I've chosen four of gold's strategies to upload. data is from SQ futures (that is from barchart)

2.When I did the retest, I didn't turn on the filter, as you can see in the attachment, but the number of strategies in the backtest results was significantly reduced(retest 4 strategies, The results of the backtest only fed back 2 strategies.), and I tried it on a variety of other emerging market futures, and there are this problem too.

Attachments
overview_136dev5test.jpg
overview_136dev5test.jpg
(911.72 KiB)
GC finaltest.jpg
GC finaltest.jpg
(760.07 KiB)
overview-136RC3test.jpg
overview-136RC3test.jpg
(838.71 KiB)
delete failed.jpg
delete failed.jpg
(766.57 KiB)
  • Votes +4
  • Project StrategyQuant X
  • Type Bug
  • Status Fixed
  • Priority Normal

History

b
#1

binhsir

05.12.2022 12:38

Task created

b
#2

binhsir

05.12.2022 12:39
Voted for this task.
CG
#3

Chris G

05.12.2022 15:29
Voted for this task.
b
#4

bentra

05.12.2022 16:41
Is this problem exclusive to WF/WFM? 

Can you just compare a normal backrest from 135 to 136 rcx? How does it compare?
E
#5

Emmanuel

05.12.2022 18:57
Voted for this task.
k
#6

Karish

05.12.2022 22:21
Voted for this task.
b
#7

binhsir

06.12.2022 01:05
to Bentra:

 WFM strategy contain normal and simple retest report too, that is mean that normal backtest report between 136dev5 and 136RC3 is big difference. Maybe I made a mistake, but I didn't find the reason, I don't know if anyone else found the similar phenomenon.

MF
#8

Mark Fric

06.12.2022 11:32

Status changed from New to Fixed

we found the problem, it was a bug introduced in the latest RC release.
b
#9

binhsir

06.12.2022 14:49
Great, Looking forward to bug fixes soon and I can't wait to use the new version.
b
#10

binhsir

07.12.2022 15:53

Attachment WF Matrix - Strategy 322196.sqx deleted

b
#11

binhsir

07.12.2022 15:53

Attachment WF Matrix - Strategy 314172.sqx deleted

b
#12

binhsir

07.12.2022 15:53

Attachment WF Matrix - Strategy 115174.sqx deleted

b
#13

binhsir

07.12.2022 15:53

Attachment WF Matrix - Strategy 318170.sqx deleted

b
#14

binhsir

22.12.2022 07:42
The second problem I fed back has not been fixed in 136 final version. When backtesting a batch of strategies, the number of resulting strategies is significantly reduced.Even though I didn't set the filter.This is a very serious mistake.
TT
#15

Tamas

22.12.2022 08:52

Status changed from Fixed to Waiting for information

Hello, we are not able to reproduce the bug. 


Can you please attach the problematic strategies + custom project file ? 


Delete failed strategies works also ok, maybe the strategies were deleted by another task not the one from the printscreen ? 



b
#16

binhsir

23.12.2022 03:34

Attachment WF Matrix - Strategy 115174.sqx added

Attachment WF Matrix - Strategy 314172.sqx added

Attachment WF Matrix - Strategy 318170.sqx added

Attachment WF Matrix - Strategy 322196.sqx added

Attachment Custom strategies GC.cfx added

WF Matrix - Strategy 314172.sqx
WF Matrix - Strategy 314172.sqx
(2.19 MiB)
WF Matrix - Strategy 322196.sqx
WF Matrix - Strategy 322196.sqx
(2.06 MiB)
WF Matrix - Strategy 115174.sqx
WF Matrix - Strategy 115174.sqx
(2.19 MiB)
Custom strategies GC.cfx
Custom strategies GC.cfx
(4.01 KiB)
WF Matrix - Strategy 318170.sqx
WF Matrix - Strategy 318170.sqx
(863.02 KiB)
Sure, i reupload these 4 gold strategy. It is easily to reproduced. If you retest 200 strategies, you will easily reproduced.

I just use one task, run this task(final task) only 

b
#17

binhsir

23.12.2022 03:42

Attachment retest.jpg added

retest.jpg
retest.jpg
(758.83 KiB)
simple reproduced with another method, just retest some strategies in same databank, you can see same result. strategies reduction is not related to filter.
TT
#18

Tamas

23.12.2022 08:50

Attachment image-0.png added

Attachment image-1.png added

image-0.png
image-0.png
(462.91 KiB)
image-1.png
image-1.png
(507.70 KiB)
Hello, I was able to reproduce the problem but it is not a bug.


Some of your strategies contain Custom Blocks from statoasis. To get it work you have to import the missing blocks.

It should be enough to open the strategies in AW Editor. CB will be added automatically.


The problem is that fuzzy logic strategies are not supported in AW SQX136 final. We will update installation zip today to fix it.

b
#19

binhsir

24.12.2022 09:01
That's fantastic. I added a custom block, and it worked. which file in user floder contain custom block information, So that I can make sure it's updated when I migrate to a new version.

Or I have to re-import custom block with .xml file one by one?

TT
#20

Tamas

06.02.2023 13:21

Status changed from Waiting for information to Fixed

Xml file is located in SQX installation folder in \user\settings\customBlocks.xml
Votes: +4

Drop files to upload

or

choose files

Max size: 5MB

Not allowed: exe, msi, application, reg, php, js, htaccess, htpasswd, gitignore

...
Wait please