<- Back to Tasks

[B131 Dev2] Max Heap Size is massively ignored

Please see the attached screenshot. Max allow heap size, as set in the config file, is 25GB. In reality, it uses almost 30GB. Might be an internal component that uses external code (like a .DLL)? This only seems to happen if using Custom Projects (and many of them). Anyway, it should still obey the set heap size, which it doesn´t. The internally reported heap usage within Strategyquants memory chart, is just 19GB. So there are 11GB being used that do not come from the JVM directly. In any case, this should be fixed.

Attachments

Untitled.png

(397.61 KiB)

Votes +1
Project StrategyQuant X
Type Bug
Status Refused
Priority Normal

History

geektrader

03.03.2021 01:42

Task created

geektrader

03.03.2021 02:01

Voted for this task.

Mark Fric

08.03.2021 11:34

Status changed from New to Refused

it is because we use also offheap memory in SQ for performance reasons, this is not included in Java heap controlled by -Xmx parameter.

You can see its usage in Offheap memory chart, usually it is below 1 GB, but it could be more

It is used to store raw history data for backtests.

geektrader

08.03.2021 19:44

That explains it then. I don´t like if developers do this, to be honest. The Java heap, especially with Graal, is being managed so efficiently, that using off heap almost always does no good in terms or performance, apart from that it of course will ignore all your memory limits as I´ve experienced.

Mark Fric

09.03.2021 08:36

if it would make no sense we wondln't be doing it. Of course we made tests and for some type of data using direct memory access is the fastest option.

Graal is only one option for SQ.

Votes: +1