Implementing Neural Networks Into Genetic Evolution

I was looking at the thread attached (hopefully this is coming soon) and I was wondering if it would be possible to implement neural nets into the genetic evolution process itself in addition to using NN as described in the thread. From what I understand the NN that is currently being worked on is meant for the selection of different blocks. Can a NN also be used for the combination of different strategies in each generation of the genetic evolution process? From what I understand, genetic evolution splices params between the most effective strategies with each successive generation. While this can help strategies improve, I have found in SQ3 that this tends to narrow down the current generation to only a few unique strategies after multiple generations. However, if we are logging the performance of different combinations of blocks, it should be theoretically possible to let a NN take care of using that data in order to splice params instead of the current form of genetic evolution that only splices the top performing params. This should also in theory lead to a diverse set of strategies after multiple generations instead of having multiple versions of the same strategy after multiple generations. The NN should be able to edit/add different user selected blocks to each strategy in the databank based on a log of previous successful combinations. 

https://strategyquant.com/forum/topic/deep-learning-ai-neural-networks-in-sq4/



This is all based on a very basic understanding of these underlying mechanisms so if I am totally off base or this reflects the current plan for implementation, feel free to let me know. 

Attachments
No attachments
  • Votes +8
  • Project StrategyQuant X
  • Type Feature
  • Status Archived
  • Priority Normal
  • Assignee Mark Fric
  • Milestone Archived (To be done later)

History

KL
#1

kainc301

26.12.2018 20:06

Task created

KL
#2

kainc301

17.01.2019 16:51

Update: Thanks to Mark and Geektrader, a base level version of what I wanted Neural Nets to accomplish is coming out in 118. 

https://roadmap.strategyquant.com/tasks/sq4_3923

https://roadmap.strategyquant.com/tasks/sq4_3977


Adding Neural Networks would further expand the capability of this 'fresh blood' feature. Mark already said he's working on implementing a Neural Network for strategy training. What I am suggesting is that a second Neural Net is implemented for the strategy generation of "fresh blood' on each subsequent generation. This would allow for better strategies to be added into the databank out of the gate rather than having completely randomly generated strategies become fresh blood. 


My suggestion for anyone considering implementing this in the future is using a Convolutional Neural Network for the training (reduces time complexity and outperforms other models on time series data), and for strategy selection, a Feed Forward Neural Network (for simplicity/ speed) or a Kohonen Neural Network (for better performing strategies) would probably work best. 


Rr
#3

Partizanas

17.01.2019 18:08
Voted for this task.
M
#4

MKquant

17.01.2019 22:06
Voted for this task.
KL
#5

kainc301

18.01.2019 08:27
Voted for this task.
KL
#6

kainc301

18.01.2019 09:44
Also to be clear, when I refer to a log of previous successful combinations, there would need to be a list of strategies the user could store in order for the selector NN to "learn" from. Users should be able to store their favorite strategies in a type of cache. 


By default, I think it should be manual storing in order to use less memory but an automated version of this would essentially be to store strategies that passed x tests/filters etc. Essentially this would become a copy of a databank that a NN could learn from in order to create new strategies and combine blocks in different ways when splicing a learned outcome with randomly generated code. The result should be new code that the NN thinks should produce a profitable result. 


The editing process should be pretty simple and straightforward:

  • Start with the randomly generated strategy, 
  • Analyze its code and compare it to the patterns stored in the cache of good strategies, 
  • NN would learn patterns from strategies in the cache in order to make a decision about what edit it thinks would make the randomly generated strategy
  • NN would make that edit and forward it to the next round of genetic evolution

The second NN (that Mark mentioned was for the training) would then take that strategy and treat it like any other randomly generated strategy that it would train on the given data set. The difference here is that you had a NN putting the first strategy together based on learned patterns in order to start off with a better result. The larger the dataset, the better the NN would be at putting strategies together because it can learn from more patterns, however, the NN would have to be able to deal with the time complexity of a potentially large data set in order not to significantly slow down genetic evolution. If this was made fast and efficient, the resulting strategies at the end of genetic evolution should be much more effective in theory. 

MF
#7

Mark Fric

18.01.2019 10:37

Assignee changed from Mark Fric to Mark Fric

Description changed:

I was looking at the thread attached (hopefully this is coming soon) and I was wondering if it would be possible to implement neural nets into the genetic evolution process itself in addition to using NN as described in the thread. From what I understand the NN that is currently being worked on is meant for the selection of different blocks. Can a NN also be used for the combination of different strategies in each generation of the genetic evolution process? From what I understand, genetic evolution splices params between the most effective strategies with each successive generation. While this can help strategies improve, I have found in SQ3 that this tends to narrow down the current generation to only a few unique strategies after multiple generations. However, if we are logging the performance of different combinations of blocks, it should be theoretically possible to let a NN take care of using that data in order to splice params instead of the current form of genetic evolution that only splices the top performing params. This should also in theory lead to a diverse set of strategies after multiple generations instead of having multiple versions of the same strategy after multiple generations. The NN should be able to edit/add different user selected blocks to each strategy in the databank based on a log of previous successful combinations. 

https://strategyquant.com/forum/topic/deep-learning-ai-neural-networks-in-sq4/



This is all based on a very basic understanding of these underlying mechanisms so if I am totally off base or this reflects the current plan for implementation, feel free to let me know. 

how exactly do you want to use NN during generation of new strategies?  If I understand correctly you mean that strategy generation should be not done completely randomly, but somehow controlled by NN.


But how exactly? Should NN somehow recommend or rank the best block combinations? Do you have some more exact idea how it should work?

?
#8

anonymous

18.01.2019 23:33
@Mark 

I dont have the exact map for the algorithm for the matching part down but the basic idea is that something like this should be doable: 


  • Block combination patterns stored in cache (either manually or automatically) would become the dataset for the NN to learn from. 
  • Current randomly generated strategy code would be indexed to be compared to patterns stored in cache
  • Loop through the blocks of the randomly generated strategy code,
  • If a block in current strategy matches a block in the dataset, a change would be made to the current random strategy based what the what the NN would weight to be a new block associated with profitable strategies containing the random strategies block. And it would just swap them one at a time, one round of evolution to the next. The NN would keep looping through the blocks of the randomly generated strategy to edit as long as the generation continues. Atleast that's how I'd approach it. You'd have to make the blocks dynamically accessible, I believe

The training algorithm would then take that in as any other strrategy in the databank and trains it on the data before the next evolution (if I understand the NN you're already working on.


I dont know if this is helpful or not. I have been studying computer science so I think this is a viable solution but I'll leave that up to the SQ team to determine :) 

KL
#9

kainc301

18.01.2019 23:36
@Mark the last comment was me, I realized I was not logged in.
IH
#10

clonex / Ivan Hudec

21.01.2019 10:25
Voted for this task.
KL
#11

kainc301

27.01.2019 18:37
I believe there is a better solution for the NN learning from a dataset of strategies. Instead of using local datasets, we should move to a "Hive Mind" structure. Much larger dataset for the NN to learn from would mean much more diverse results. Everyone that contributes helps everyone else and their own strategies get better. 


The full description of this addition is here: https://roadmap.strategyquant.com/tasks/sq4_4042


Hopefully, this catches on. Even though it will be a long term project, I believe the end result would be something that no one else could compete with because we'd all keep improving what we ultimately develop individually and collectively.

MG
#12

maegop

03.02.2019 05:12
Voted for this task.
b
#13

beppil

09.05.2019 09:50
Voted for this task.
MF
#14

Mark Fric

16.02.2020 13:52

Status changed from New to Archived

AG
#15

Tukirito

27.02.2020 14:56
Voted for this task.
dv
#16

PEDRO

20.06.2021 08:41
Voted for this task.

Votes: +8

Drop files to upload

or

choose files

Max size: 5MB

Not allowed: exe, msi, application, reg, php, js, htaccess, htpasswd, gitignore

...
Wait please