Hangs after 1 day 19 hours AND ALL STRATEGIES ARE LOST!

1.  My machine is a dual CPU E5-2670 Xeon with 132GB Ram. After the first day, the response time to mouse clicks had been reduced to 1 - 2 minutes. It died sometime the following day (died means CPU usage minimal and no response to mouse clicks after 5 minutes). The log files are attached.

2.  Even with 20 Islands, the CPU utilization rarely stayed above 95% and the thermal output of the server was significantly less than running 20 concurrent sessions of 3.8.2.

3.  Why doesn't SQ4 save the databank as it is being created or am I looking in the wrong place??? At least 3.8.2 protected the databank in the event of a catastrophic failure.


Attachments
No attachments
  • Votes 0
  • Project StrategyQuant X
  • Type Bug
  • Status Fixed
  • Priority Normal

History

d
#1

Dave

24.08.2018 00:33

Task created

MF
#2

Mark Fric

24.08.2018 08:30

Status changed from New to Waiting for information

do you use Build 108?


There is an optional file-synchronization in databank, you can set it to synchronize every X minutes, so that you'll not lose your work if program crashes.

Otherwise databank is kept in memory for performance reasons.



We'll look at memory problems during long generation, we'll try to solve it in the next build.

d
#3

Dave

24.08.2018 16:33

Yes, this was on build 108.


I will look for the file-synchronization option.


A suggestion is to set the file-synchronization ON as default. Considering that you have customers from all over the world running SQ on everything from laptops to data center quality servers I am willing to bet that, being customers and not IT specialists, they are not prepared for an extended power outage and many may be in countries without reliable power. In my case, living in Florida which is the lightning capital of USA, my computers have individual dedicated data center level UPS protection. The fact that my server draws nearly 500 watts when running at max capacity the battery run time is still limited to no more that 10 minutes before the server performs an automatic shutdown. Even with a UPS initiated orderly shutdown I suspect that any strategies that had been found would be lost.


d
#4

Dave

25.08.2018 04:14

I found the option to store strategies on the hard disk. I do not understand how writing a file to disk could cause a 90% degradation in the generation cycle. If the intent is to maintain a 100% mirror image with memory when the max strategy threshold has been met then I *might* understand. A real simple option is simply to write every single strategy to disk when it is put in the database and do not worry about any strategies that are later deleted.  In other words if my max strategy number is 5 then keep on writing each subsequent strategy to disk even though SQ has already found 5.  Nobody will complain about 25 additional strategies only because they took a weekend holiday.  Likewise, I run SQ with 10,000 allowed strategies and would not be upset if I found 30,000 after a three week period.  Simply stated, don't worry about synchronizing the data, just save it. The user can never recover the lost data but they can always re-analyze the excess data and throw away what they don't want.


Just an idea.

Dave


d
#5

Dave

25.08.2018 04:14

I found the option to store strategies on the hard disk. I do not understand how writing a file to disk could cause a 90% degradation in the generation cycle. If the intent is to maintain a 100% mirror image with memory when the max strategy threshold has been met then I *might* understand. A real simple option is simply to write every single strategy to disk when it is put in the database and do not worry about any strategies that are later deleted.  In other words if my max strategy number is 5 then keep on writing each subsequent strategy to disk even though SQ has already found 5.  Nobody will complain about 25 additional strategies only because they took a weekend holiday.  Likewise, I run SQ with 10,000 allowed strategies and would not be upset if I found 30,000 after a three week period.  Simply stated, don't worry about synchronizing the data, just save it. The user can never recover the lost data but they can always re-analyze the excess data and throw away what they don't want.


Just an idea.

Dave


MF
#6

Mark Fric

27.08.2018 08:04

Status changed from Waiting for information to Fixed

Dave, the problem is not in deleting strategies. Saving strategies to disk is really slow, much slower than just keeping them in memory.


But we added periodic synchronization with file system, it is the best from both worlds. We'll make it turned on by default in the next build.

I believe saving strategies once every hour or two must be sufficient.


Votes: 0

Drop files to upload

or

choose files

Max size: 5MB

Not allowed: exe, msi, application, reg, php, js, htaccess, htpasswd, gitignore

...
Wait please