b130 dev1: Gen builder, with mutation off: some parents are breeding and deleted and others are duplicated. duplication is probably not needed anymore.

Voted for this task.

For clarity here is a zoom in of the strategies list I was trying to highlight. You can see by the strat name a (1) that means it is an exact copy of the strategy with the same name.

I found this in a ref:
"If it is 0%, then the complete new generation of individuals is to be exactly copied from the older population"

But I'm sure that they mean to bring a single copy of each strategy. They do not mean to make a copy and bring the original AND the copy in to the next generation.

Voted for this task.

Voted for this task.

Voted for this task.

Voted for this task.

Have you double checked the pseudo code to make sure they're actually the same?

Then convention seems to be based on island.generation.strategy 

1.0.30 seems to be island 1. generation 0. strategy 30 and when your evolution restarts after stagnation, it may find another strategy with the same name. 

I can't edit my previous post for some reason and it meant to say "The naming convention" instead of "Then convention" 

Attached is an image of some examples of mine where I have the same strategy name, but entry and exit are different for both.

Voted for this task.

@Insanity, it's not possible for it to be from a previous restart. My screenshot is "last generation" tab which we know empties itself after each restart. 

> I have not seen any references which would suggest that strategies should make a copy of itself and both the copy AND the original are brought forward to the next generations. This may be a bug or a 

> misinterpretation of how genetic evolution is supposed to work or it may be an artifact from a programmer trying to fix the previous breeding bug. In theory creating a copy of itself and bringing the original 

> AND the copy forward to next gen requires that even more of the weakest genomes die off to make room. This will reduce diversity and accelerate convergence towards local optima unnecessarily. 

no, it is intentional and exactly how it should work. There are multiple approaches to genetic programming, and it is possible that there will be multiple copies of the best strategies using elitism.

It doesn't make sense to keep the the weakest candidates and hope for some breakthrough. 

Aha Elitism! Excellent, thanks for the info. But maybe elitism is currently set to be a little aggressive. By generation 10 with crossover set to only 4% and mutation to 20% I have mostly the same 1 or 2 strategies filling  the 100 population island.

If I'm using decimation, then it DOES seem to make sense to leave some balance of  the strats from last generation with the elitism copies and the new children. It seems to NOT make sense to turn on "fresh blood" because that will defeat the purpose of the decimation and the elitism functions and use more CPU.

Is there a way to turn down elitism a little? Or could you please run some tests to find a better balance (if necessary) now that the breeding is fixed?

In fact it seems like I would want more breeding and less elitism if I am using stricter initial pop filters and or higher levels of decimation. So I think there's an argument for a new elitism slider bar =)
(Picture a very diverse high quality starting population where the quality is very high across the entire population. I may want to use very low or even a zero mutation which also detracts from the usefulness of elitism. In this case I would want parents mostly just mating and then only deleting strategies if the children are of even better quality. I wouldn't necessarily want to start throwing away such high quality strategies to make room for redundant copies of other slightly higher quality, it seems like it would do more harm than good in the form of reducing diversity.)

Should I open a new ticket for this request?

bendx77 - can you attach your build config here? The one that leads to the duplicate strategies.

I'll look at the cause, it shouldn't behave like this.

"show-elitism" config file is zero mutation and 30 crossover you can see the elitism in action on generation #1. screenshot 72. some are just copies.

"show-convergence" config file is 10% crossover and 30% mutation. You can see by generation #5 we have a single dominant strategy type in a 25 population island. Mostly just variations of the same indicators. Most of the diversity is lost by generation #5 even with a low crossover of 10%. screenshot 75. I do not wish to turn on fresh blood as this would defeat the purpose of elitism function. The combination of fresh blood with high elitism would then be defeating the purpose of decimation and high initial pop filters. Please maybe look at adding an elitism slider bar or just turning it down. Also, while you're there, please maybe look at the frequency of the breeding that specifically produces children extraordinarily similar to one parent like I complained here: https://roadmap.strategyquant.com/tasks/sq4_7258 this could also be contributing to loss of diversity too early if it's happening too much.

The duplicates are not created only by elitism, elitism is set only for a few top strategies. It is a natural property of genetic selection algorithm that a candidate can be chosen multiple times, it is not wrong. But it shouldn't duplicate one strategy over whole population.

I made some improvements in genetic selection algorithm - no strategy should be selected to be more than 20% of the population.

Thanks for looking at this. Perhaps one day this code could be exposed to us, so many variables to experiment with!

Voted for this task.

Voted for this task.