[Build122] Crashed when generating strategy in Project
My SQX crashed multiple times.
My SQX ram setting is 48 GB. It crashed even when running M1 data. When it crashes, I check my machine memory usage was only around 50%.
I also active debug level and It generates a huge log file that I cannot open it with any of my editors.
The strange thing is when set debugs level to false. It still produces a huge log file!
I attach my SQX setting, project setting and SQX crash log. I cannot attach debug log because the file size is too large.
Please let me know if you need anything.
-
Votes +1
-
Project StrategyQuant X
-
Type Bug
-
Status Fixed
-
Priority Normal
History
ps21
25.08.2019 12:10In the past, My SQX also crashes after running for 1-2 week. It doesn't happen frequently so I did not report this.
But I have made progress. It now can run for days. Without knowing why. I will try to run the project again with debug level on to see why it crash! I will keep you posted!
ps21
30.08.2019 16:35Attachment hs_err_pid17364.log added
Attachment debug.log added
I attached the log file with both parts of the debug log and crash log.
I also encounter a strange problem that SQX use only 1 NUMA core at 100%.
I will try build123 to see if it can solve both problems.
ps21
01.09.2019 08:05Attachment hs_err_pid12076.log added
Attachment sqx.zip added
Here's crash log
ps21
04.09.2019 10:11Please help me with the log file first so I can easier get the log file for you when SQX crash.
And I know now why the log file was huge after debug level option set to false. It about the custom column that I created.
the log file contains a lot of this
23:59:59.999 [Blocking computeThread #18 - Builder_4-Generation 5.21.196] INFO SQStats - Unknown double PctReturnDDRatio=1.47
I have around 5 custom column so I print out 5 times per strategy!
Here is the custom column code:
package SQ.Columns.Databanks; import com.strategyquant.lib.*; import com.strategyquant.datalib.*; import com.strategyquant.tradinglib.*; public class PctReturnDDRatio extends DatabankColumn { public PctReturnDDRatio() { super("PctReturnDDRatio", DatabankColumn.Decimal2Pct, ValueTypes.Maximize, 0, 0, 100); setDependencies("ReturnDDRatio"); setTooltip(L.t("Pct of Return / Drawdown Ratio between MC and main backtest")); } //------------------------------------------------------------------------ /** * This method should return computed value of this new column. You should typically compute it from the list of orders * or from some already computed statistical values (other databank columns). */ @Override public double compute(SQStats stats, StatsTypeCombination combination, OrdersList ordersList, SettingsMap settings, SQStats statsLong, SQStats statsShort) throws Exception { double mainReturnDD = stats.getDouble("ReturnDDRatio"); return mainReturnDD; } //------------------------------------------------------------------------ public String getCrossCheckComputedValue(ResultsGroup results, double crossCheckValue) throws Exception { // Return/DD of configured cross check double ccReturnDD = crossCheckValue; // Return/DD of main result double mainReturnDD = results.portfolio().stats(Directions.Both, PlTypes.Money, SampleTypes.FullSample).getDouble(StatsKey.RETURN_DD_RATIO); double pct = SQUtils.safeDivide(ccReturnDD, mainReturnDD) * 100; return String.format("%.2f", pct); } }
Mark Fric
04.09.2019 10:26I'm glad NUMA problem was solved.
I updated the logging process in build 124 so that it doesn't repeat the same message about unknown value, it should work in Build 124.
ps21
17.09.2019 05:25Attachment Crash 070919.zip added
Attachment Crash 080919.zip added
Attachment Crash 090919.zip added
Attachment Crash 100919.zip added
Attachment Crash 120919.zip added
I also encountered numbers of crashes for the past few weeks. I have attached log files with this comment. One file zip for one crash. In zip file contains a crash log, project log, SQXlog.
It seems that ParGC crash a lot so I try running SQX with G1 GC same problem. I read a post mainly on StackOverflow they said that It maybe JAVA problems and should update the java version and maybe my memory or hardware problem. I scan my machine memory for a whole day using memtest86 still not find any error.
So is it possible that I try a newer JAVA version on the machine? or SQX will run only on JAVA that ship with the program.
Please lets me know if you need anything more on crash information
Mark Fric
17.09.2019 09:28and put it to this folder (back up the existing version).
SQ uses Java 11 from Azul systems, maybe "standard" Java from AdoptOpenJDK would work better.
But in my opinion, the bigger chance is that crashes are caused either by bad memory block, or by runnning out of memory. You hav e48 GB memory for SQ, but it is possible to fill any amount of memory with "bad" configuration.
Maybe try to lower the number of parallel threads, or turn off some expensive cross checks (WF and Optimization profile), maybe also MonteCarlo retests and see if it still crashes.
© Copyright. All rights reserved. ProjectPanel.com
It would really help if you could send at least some part of your big log in NON--debug mode. It doesn't create big log in my tests.
You can use a tool: https://download.cnet.com/Large-File-Viewer/3000-2351_4-76641086.htmlto view big files, you can use it to copy a part of it to another file.