RC5 freezes at loading into retester

Hi,
I reload 12000 Strategies into the backtester.
But It freezes after a while

Attachments
log_2018_06_29.rar
(143.16 KiB)
setting Q02 EURUSD H1 H4.xml
(133.37 KiB)
  • Votes 0
  • Project StrategyQuant X
  • Type Bug
  • Status Fixed
  • Priority Normal

History

t
#1

tnickel

30.06.2018 01:30

Task created

MF
#2

Mark Fric

02.07.2018 08:01

Status changed from New to Waiting for information

ok, but 12000 strategies is extreme number, I'm not sure SQ should even support that many.


Databank in SQ4 is backed by files, so you don't need to worry about saving strategies and also so that memory can be freed if strategy is not used.

Bud even OS would have problem handling 12000 small files in one folder.


I think this usage is simply too extreme, what exactly is your reaosn for having that many strategies in databank?

I think 1-2000 strategies is maximum that makes sense. You should use advanced filtering to reject "bad" strategies automatically.


h
#3

hankeys

02.07.2018 08:18
but for now filtering by croschecks doesnt work, multiple OOS i cant use, because i cant filter them independently, custom project doesnt work - so we need to build only by simple method (no filtering) and make the retest of the whole bunch of strategies


we need to move theses things up for RC6, its crucial

t
#4

tnickel

02.07.2018 10:47
Hi Mark,

with the SQ 3.8.2 I am handle more than 25000 Strategies with no problem.

I think the Operating System can handle more than 50000 Strategies in one directory or more.


I found many exceptions in the logfile, I think this is problem in the SQ 4.0


I found: https://superuser.com/questions/446282/max-files-per-directory-on-ntfs-vol-vs-fat32

•NTFS ("New Technology File System")

◦Maximum number of files on disk: 4,294,967,295

◦Maximum number of files in a single folder: 4,294,967,295


---

For Stresstests I generate strategies with less filter cirteria. It will find more bugs in the sq. Bugs can be found best if 

I stress the system to the limit.


--

In the SQ 3.8.2 ,my strategy generation produces more than 25000 Strategies for EURUSD.

With same settings I got only 100 Strategies in different currency pairs.


thomas

MF
#5

Mark Fric

04.07.2018 07:04

Status changed from Waiting for information to Fixed

ok, we modified databank, it should handle big number of strategiew better.


We had to give up realtime synchronization with files, because although it could work it slows down everything 10x.


Votes: 0

Drop files to upload

or

choose files

Max size: 5MB

Not allowed: exe, msi, application, reg, php, js, htaccess, htpasswd, gitignore

...
Wait please