Performance degrades with big databanks

When I load a big databank like 16000 strategies, it starts very well.
But as time goes load time becomes slower and slower.

Similar things happens when I perform WFM robustness test of a big amount of strategies.
For example 1200+ strategies.
GUI becomes very laggy.
CPU usage is high ~90%.
Memory usage is 16-18GB, total memory 64GB, free memory is available.
After more tests I discovered high disc usage during WFM automatic retest, is it OK?

Trying to fix these issues myself I enabled.
Automatic memory cleanup every 30 minutes.
But it doesn't make much difference.

Attachments
No attachments
  • Votes +4
  • Project StrategyQuant X
  • Type Bug
  • Status Fixed
  • Priority Normal

History

KP
#1

CooleRnax

08.01.2020 09:39

Task created

KP
#2

CooleRnax

09.01.2020 09:43

Description changed:

When I load a big databank like 16000 strategies, it starts very well.
But as time goes load time becomes slower and slower.

Similar things happens when I perform WFM robustness test of a big amount of strategies.
For example 1200+ strategies.
GUI becomes very laggy.
CPU usage is high ~90%.
Memory usage is 16-18GB, total memory 64GB, free memory is available.

After more tests I discovered high disc usage during WFM automatic retest, is it OK?


Trying to fix these issues myself I enabled.
Automatic memory cleanup every 30 minutes.
But it doesn't make much difference.

KP
#3

CooleRnax

09.01.2020 10:15
Some more interesting info:

Was running 930 strategies WFM test, completed 90% overnight.

Performance slowed a lot, got tired of waiting.

I stopped WFM test.

Even after stop, CPU usage is at 90%.


t
#4

tnickel

15.01.2020 16:13
Voted for this task.
KP
#5

CooleRnax

16.01.2020 09:50
Voted for this task.
AG
#6

Tukirito

16.02.2020 21:46
Voted for this task.
AG
#7

Tukirito

16.02.2020 21:52
Same thing is happening to me, in my case RAM usage goes to 34GB and stays there even if I stop everything that SQ is doing, I have a databank with 25,000 strategies for the initial population, when the program starts the memory usage is really low but it takes up to 10 minutes to just load the databank, I have an SSD so its not a problem with the drive but with probably how SQ loads things, at least for that part. I have a custom project that moves the best strategies to use for the initial population for the next run and if it runs for more than a few hours the performance slows down a lot.
m
#8

Mariano

05.05.2020 20:44
Voted for this task.
TT
#9

Tamas

13.05.2020 16:06

Status changed from New to Fixed

We made some performance improvements in recent release. Please upgrade to version 128 and check.

Votes: +4

Drop files to upload

or

choose files

Max size: 5MB

Not allowed: exe, msi, application, reg, php, js, htaccess, htpasswd, gitignore

...
Wait please