Replace similar strategies in databank

It would be EXTREMELY useful to have a feature in the build options that replaced a strategy in a databank with a better strategy, instead of having stupid amounts of similar strategies being placed in a databank from the build phase!



I know this is an option for Genetic Evolution, but it should be an option to check existing DB strategies to remove and replace existing strategies with better strategies when this happens.


I also know that I could potentially set DB max to 20 strategies and set Total strategies (that passed filters) to 100 strategies and have this eliminate what is perceived as worse strategies and give me more variety based on my Strategy Quality Ranking (fitness) settings. However, with this method, I run the risk of eliminating strategies that may not be the best from a fitness perspective (and get deleted) that may have done very well in the robustness tests (MC Spread, Parameters and WFM), so this is not a great way of overcoming this problem either.



Example attached where I have a very large amount of similar strategies from the builder task in my project + project attached.


Thanks :)

Attachments
project.cfx
(38.91 KiB)
Replace Similar.png
(46.30 KiB)
  • Votes +7
  • Project StrategyQuant X
  • Type Feature
  • Status Refused
  • Priority Normal

History

JK
#1

Insanity82007

09.04.2020 12:23

Task created

JK
#2

Insanity82007

09.04.2020 12:23
Voted for this task.
JK
#3

Insanity82007

09.04.2020 12:24

Type changed from Bug to Feature

JK
#4

Insanity82007

09.04.2020 12:27
PS that builder task was only trying to build 20 strategies, so finding 8 x DeMarker + PSAR strategies out of 20 is REALLY problematic!
Gl
#5

Gui

09.04.2020 12:31
Voted for this task.
JK
#6

Insanity82007

09.04.2020 12:32

Attachment More of the same problem.png added

Attachment More of the same problem.png deleted

Attachment More of the same problem.png added

6 / 15 results are duplicated in this image
JK
#7

Insanity82007

09.04.2020 12:34

Attachment More of the same problem 2.png added

5 / 9 in this one
JK
#8

Insanity82007

09.04.2020 13:07

Attachment More of the same problem 3.png added

19 out of 65 !!!
JK
#9

Insanity82007

09.04.2020 13:08

Attachment genetic settings.png added

Genetic settings trying to create as much diversity as possible based on reading help and e-mail discussions, but still not working that well. 
JK
#10

Insanity82007

09.04.2020 13:14

Attachment More of the same problem 4.png added

This is literally about 5 minutes later from my 19 / 65 post and now I'm up to 44 / 94 with the same entry criteria!
JK
#11

Insanity82007

10.04.2020 04:01
If this does get included, can we please have 3 options:

  1. Keep all strategies (do not delete similar strategies)
  2. Keep only strategies that have the same entry and exit criteria, but different variables and pick the best one for each 
  3. Keep only the best strategy for strategies that have the same entry and exit criteria


The "best" strategy will be based on Fitness.


For option 2:

  1. Example keep: If 2 strategies have the same entry and exit criteria (databank columns), but each have different other variables, then keep both
  2. Example replace: If 2 strategies have the same entry and exit criteria (databank columns) and have the same variables for everything else


For option 3:

  1. Only keep the best strategy where displayed entry and exit criteria are the same (from databank columns)
o
#12

Enric

10.04.2020 10:17

Hi,

I think this could be a good new feature. But should be an easy option to activate or deactivate. I mean, do not overcomplicate it with setup and parameters just check the correlation %. Simply; i) add a boolean on the Builder to be able to On/Off this option, ii) New field where you can setup which is the maximum % correlation allowed.

o
#13

Enric

10.04.2020 10:17
Voted for this task.
h
#14

hankeys

10.04.2020 12:43
discussed here many times - https://roadmap.strategyquant.com/tasks/sq4_5945https://roadmap.strategyquant.com/tasks/sq4_3998


i dont think so, that its reasonable to solve the similarity only on used building blocks...different parameters and mostly different used SL ant PT values means that even the strategy is using "same" building blocks could be totally diferent


for me the similarity is more equatation of correlation or RDD vs number of trades


and for now we have bigger problems - because there is not solved the simplifier of strategies - because we can see building blocks in the strategy which is doing nothing - https://roadmap.strategyquant.com/tasks/sq4_3807

JK
#15

Insanity82007

10.04.2020 15:33

Attachment Example of strategy with same text.png added

All this needs is a quick text comparison, which is EXTREMELY fast in any database solution.


These 2 strategies (attached) are exactly the same strategy (one strategy). Both have exactly the same variables, but different parameters. I only need one of these strategies to remain in my databank, not both

because I will optimise these parameters during WFM assuming this strategy passes all robustness tests

JK
#16

Insanity82007

10.04.2020 15:38

Subject changed from Replace similar strategies in databank to Replace similar strategies in databank (Psuedo Text comparison and Fitness rank to make decision for add / replace)

m
#17

mabi

10.04.2020 15:49

Subject changed from Replace similar strategies in databank (Psuedo Text comparison and Fitness rank to make decision for add / replace) to Replace similar strategies in databank

This is good but it would need to be done in crosschecks ofcourse. Real diversified strategies have diffrent entry codes but there are ways to trade when just off setting the entry points actually works great especially in trending markets which is the only time breakout strategies makes money  using the same entry so totally removing it when building is not an option and also impossible since would be to slow but as a crosscheck would be great.
m
#18

mabi

10.04.2020 15:50
Voted for this task.
JK
#19

Insanity82007

10.04.2020 23:39
Agreed mabi. A correlation crosscheck would be the best
BS
#20

BobS

14.04.2020 15:32
Voted for this task.
m
#21

Mariano

16.04.2020 14:20
Voted for this task.
b
#22

beppil

20.04.2020 12:37
Voted for this task.
MF
#23

Mark Fric

19.02.2021 10:02

Status changed from New to Refused

there already is check for similar strategies, but the similarity is evaluated by the similarity of trading results, not by the used indicators. 


Small chanes in parameters can create strategy that trades very differently.

h
#24

hankeys

20.02.2021 10:50
yes, this could be true, but we are still waiting for some simple correlation duplicities remover, which we still dont have in SQX


fit to existing portfolio cant be used for our purposes


Votes: +7

Drop files to upload

or

choose files

Max size: 5MB

Not allowed: exe, msi, application, reg, php, js, htaccess, htpasswd, gitignore

...
Wait please