Genetic Option : Initial population



1/   Would it possible to select a databank for initial population ?


For exemple:


If you have the results of a Build task in a "Results Build" databank, would it be possible in a Genetic option to select this databank "Results Build" as initial population.


Like that , the Genetic process in Build will have more fresh stratégies as the initial population constantly as the Build progress.


(I notice Build process build much more stratégies if you update your initial population regularly)




2/   Would it possible to include in the Genetic option , a percentage of random strategie ?


Like that,  the genetic process can generate X percentage of Random stratégies and the rest from initial population. (Whatever is the size of the initial population)







Attachments
No attachments
  • Votes +2
  • Project StrategyQuant X
  • Type Feature
  • Status New
  • Priority Normal

History

E
#1

Emmanuel

14.07.2020 19:34

Task created

E
#2

Emmanuel

14.07.2020 19:34
Voted for this task.
KL
#3

kainc301

16.07.2020 02:41

I believe this already exists and I do it all the time.


If you go to Genetic options, under Initial Population Generation, select the option "Use strategies from Initial population as evolution start" 


When this is selected, you can paste or load strategies in there that you would like to begin evolving from. The rest of the initial population will be randomly generated


This should handle what you are trying to do. If not, please elaborate

E
#4

Emmanuel

16.07.2020 03:15

Thank you kainc301 for your answer.


1/  If we  have an Initial Population size required of 3000 ,   you select a Generated Decimation Coefficient of 3  with the option "Use strategies from Initial population as evolution start


This mean that your initial population will be 9000.


if you have 15000 stratégies in your Initial Population databank,    


Then :


- SQX will never use the 15000 stratégies, only 9000 of them.

- SQX will never produce random stratégies as the 9000 initial population will be filled with the stratégies in your Initial Population databank



If the initial population is filled with the stratégies of  the Initial Population databank, we still need to add a percentage of random stratégies .


It is not because we give an initial population that we don't want randomness as well.


Otherwise we will  have a lot of similar stratégies  as we have always the same parents as initial population without  randomness.

This can be a source of  overfitting  stratégies.


2/ if we was able  to select a databank as initial population databank ,  it would not be necessary to copy and paste the stratégies all the time from the results databank to the "Initial population databank"  in the Build Task

Furthermore, we may have to stop and restart the task for SQX to read this new stratégies in the Initial population databank. (we don't have this information in the documentation)


h
#5

hankeys

16.07.2020 07:33
using too much initial population leads nowhere and i dont trust your opinion that adding "final" strategies again as initial population is a good idea


it will lead only to 2 things - overoptimising and duplicates

?
#6

anonymous

16.07.2020 15:49

Thank you hankeys for your answer.



Here is what I found :


In my experience , If I don't have any inital population , I can produce only 60 Stratégies per hour. (However, this Build is necessary to have a good diversify initial population)

 

If I have around 18000 stratégies as Initial population, I produce around 3000 stratégies per hour without initializing the initial population

 

If I update the initial population, and restart the Buil task, I am having around 6000 new stratégies the first hour, then it is decreasing.


 Which is normal, if you have always the same initial population, it take more and more time for SQX to find new strategies from the same parents.


 Here can see stats I noted last night as an example .


https://drive.google.com/file/d/1827lFGr9_pie3HXnDQNJa4BFpa0Sq3Tj/view?usp=sharing


This is why, it would be interesting to speed up the process by getting the initial population directly from a results databank (a chosen databank), like that we would not have to copy and paste all the time from a results databank to the initial population databank.

 

Of course, it is necessary to filter the results databank before using them in as initial population

 












E
#7

Emmanuel

16.07.2020 16:04

Thank you hankeys for your answer.



Here is what I found :


In my experience , If I don't have any inital population , I can produce only 60 Stratégies per hour. (However, this Build is necessary to have a good diversify initial population)

 

If I have around 18000 stratégies as Initial population, I produce around 3000 stratégies per hour without initializing the initial population

 

If I update the initial population, and restart the Buil task, I am having around 6000 new stratégies the first hour, then it is decreasing.


 Which is normal, if you have always the same initial population, it take more and more time for SQX to find new strategies from the same parents.


 Here can see stats I noted last night as an example .


https://drive.google.com/file/d/1827lFGr9_pie3HXnDQNJa4BFpa0Sq3Tj/view?usp=sharing


This is why, it would be interesting to speed up the process by getting the initial population directly from a results databank (a chosen databank), like that we would not have to copy and paste all the time from a results databank to the initial population databank.

 

Of course, it is necessary to filter the results databank before using them in as initial population.


As the strategies are always going in the same databank, I didn’t notice any duplicate. it may be possible, but I didn’t find any duplicate.

In my testing, the correlation of the filtered strategies a under arround .3 in my portfolio.


It don’t seem that I have to many duplicate with this low correlation  number, however this is possible.


Regarding overoptimising, I tested StrategiesQuantX on EURUSD for an OOS3 for 2017 , then 2018, the results are profitable. I am working on 2019.


If I notice any duplicate, or if I see overoptimization , I will report it here for the community.

       


E
#8

Emmanuel

11.11.2022 15:45
Voted for this task.

Votes: +2

Drop files to upload

or

choose files

Max size: 5MB

Not allowed: exe, msi, application, reg, php, js, htaccess, htpasswd, gitignore

...
Wait please