SQX 136 2022.12.23 [SEVERE BUG]: SQX builds invalid strategies that look ok but are not ok (stockpicker engine)

Hi again.

I have written about this bug before in the thread here: https://roadmap.strategyquant.com/tasks/sq4_9886

I realized that the thread has become confusing, so here I have summed up the problem with a simple example:

The attached strategies, they all open and close trades at the last trading day of the month (I used a template for that).

The strategies use a stock group consisting of only 6 stocks (for simplicity). I have attached the data for 2 stock groups. They are identical except that one stock group has data for 2 more month in it.

So here is the problem: When I retest the strategies using each of the 2 stock groups, then I get different results. That is weird. The results should be exactly the same. I use the same start and end day for retesting. Screenshots that show the problem are attached. I have verified that the data is identical except the extra 2 month.

Using the attached files you should be able to reproduce this bug if you want to. If not, please let me know.

Kind regards

Attachments
TestGroup end 20221230.zip
(105.73 KiB)
Strategies.zip
(238.43 KiB)
TestGroup end 20221230.png
(152.50 KiB)
TestGroup end 20221031.zip
(102.69 KiB)
TestGroup end 20221031.png
(154.10 KiB)
  • Votes +2
  • Project StrategyQuant X
  • Type Bug
  • Status Fixed
  • Priority Normal

History

d
#1

Hamselv

27.01.2023 12:52

Task created

d
#2

Hamselv

27.01.2023 12:54
Voted for this task.
TT
#3

Tamas

27.01.2023 20:39

Status changed from New to In progress

Attachment image-0.png added

image-0.png
(390.90 KiB)

Ok, I see why the difference. Entry/exit signals are not evaluating on the very last bar.


I will fix that in build 137 dev2

d
#4

Hamselv

27.01.2023 23:32

Attachment Trade list with 3 extra trades.png added

Attachment Trade list.png added

Attachment image-0.png added

Trade list.png
(214.11 KiB)
Trade list with 3 extra trades.png
(217.80 KiB)
image-0.png
(460.86 KiB)
Nice, but I think the problem is deeper than that. Here is why:


Attached are 2 screenshots of trading lists for strategy 0.173560. As you can see there are 3 extra trades using the stock group 2022.12.30. The extra trades are in the middle of the trading period, not near the end of the test. The number of trades should be the same for both stock groups, so the 3 extra trades are somewhat a mystery.

Do you see it and do you agree please??


Another minor bug is that the trades labelled "EndTest" should be labelled "Exit Signal" instead. The reason is that the exit date is the same as the end test date (see attached screenshot).

Regards

E
#5

Emmanuel

30.01.2023 09:48
Voted for this task.
TT
#6

Tamas

30.01.2023 11:45

Status changed from In progress to Fixed

Attachment image-0.png added

Attachment image-1.png added

Attachment image-2.png added

Attachment image-3.png added

image-1.png
(81.19 KiB)
image-2.png
(228.14 KiB)
image-3.png
(1.96 MiB)
image-0.png
(52.52 KiB)

It looks that you have modified the template file from ebook and changed Position Score to RandomCondition.

Position score can by only RandomValue, we will limit that in AW.


Second problem is that Indicator MAVP has 2 periods, Min and Max. Builder generated greater Min period than Max which is the source of the 3 extra trades. 

Talib indicators returns 0 values for some periods.


The last one problem is related to the very last bar on end of the period.


All of these bugs will be fixed in the next 137 dev2 on next Monday. 6.2.2023


Thanks for your effort to test stockpicker engine :)

d
#7

Hamselv

30.01.2023 12:25
Hi Tamas.


Nice, thank you. Yes you are correct, I used the ebook template file and modified it.


I'm worried that the MAVP indicator is not the only indicator that has a problem. Can you check all the indicators please?


Why can position score only be RandomValue please? I think it makes sense that the position score can be a random indicator, and not just a random value (I assume that random value is just a random number).


Kind regards

TT
#8

Tamas

30.01.2023 13:39

Attachment image-0.png added

image-0.png
(313.44 KiB)
It looks only MAVP has min/max period. List of all available indicators with definitions are here internal\web\SQWIZARD\branding\global\config.xml

Search for talib_ prefix. But of course we will double check that.


RandomValue is any block except comparisons.

d
#9

Hamselv

30.01.2023 16:30
Very nice, thanks! I checked the xml file for talib_prefixes that contain more than one period in the display attribute. I could find 10 indicators including MAVP. They are listed below. I guess there are more as I didn't look for periods in the Param key elements.


Is it possible to check every indicator in the TA library before the 137 dev2 release, or will it taker longer please?



        <Item key="talib_ADOSC" name="(ADOSC) Chaikin A/D Oscillator" display="ADOSC(@Chart@#FastPeriod#,#SlowPeriod#)[#Shift#]" returnType="number" categoryType="indicator">

        <Item key="talib_APO" name="(APO) Absolute Price Oscillator" display="APO(@Chart@#FastPeriod#,#SlowPeriod#,#MAType#)[#Shift#]" returnType="number" categoryType="indicator">

        <Item key="talib_MACD" name="(MACD) Moving Average Convergence/Divergence" display="MACD(@Chart@#FastPeriod#,#SlowPeriod#,#SignalPeriod#)[#Shift#]" returnType="number" categoryType="indicator">

        <Item key="talib_MACDEXT" name="(MACDEXT) MACD with controllable MA type" display="MACDEXT(@Chart@#FastPeriod#,#FastMAType#,#SlowPeriod#,#SlowMAType#,#SignalPeriod#,#SignalMAType#)[#Shift#]" returnType="number" categoryType="indicator">

        <Item key="talib_MAVP" name="(MAVP) Moving Average with Variable Period" display="MAVP(@Chart@#MinPeriod#,#MaxPeriod#,#MAType#)[#Shift#]" returnType="number" categoryType="indicator">

        <Item key="talib_PPO" name="(PPO) Percentage Price Oscillator" display="PPO(@Chart@#FastPeriod#,#SlowPeriod#,#MAType#)[#Shift#]" returnType="number" categoryType="indicator">

        <Item key="talib_STOCH" name="(STOCH) Stochastic" display="STOCH(@Chart@#FastK_Period#,#SlowK_Period#,#SlowK_MAType#,#SlowD_Period#,#SlowD_MAType#)[#Shift#]" returnType="number" categoryType="indicator">

        <Item key="talib_STOCHF" name="(STOCHF) Stochastic Fast" display="STOCHF(@Chart@#FastK_Period#,#FastD_Period#,#FastD_MAType#)[#Shift#]" returnType="number" categoryType="indicator">

        <Item key="talib_STOCHRSI" name="(STOCHRSI) Stochastic Relative Strength Index" display="STOCHRSI(@Chart@#TimePeriod#,#FastK_Period#,#FastD_Period#,#FastD_MAType#)[#Shift#]" returnType="number" categoryType="indicator">

        <Item key="talib_ULTOSC" name="(ULTOSC) Ultimate Oscillator" display="ULTOSC(@Chart@#TimePeriod1#,#TimePeriod2#,#TimePeriod3#)[#Shift#]" returnType="number" categoryType="indicator">


TT
#10

Tamas

31.01.2023 07:49

Status changed from Fixed to In progress

Yes, we will do our best to fix it in beta2
d
#11

Hamselv

31.01.2023 09:13
Great, thank you :-)
MF
#12

Mark Fric

16.02.2023 15:31

Status changed from In progress to Fixed


Votes: +2

Drop files to upload

or

choose files

Max size: 5MB

Not allowed: exe, msi, application, reg, php, js, htaccess, htpasswd, gitignore

...
Wait please