132: support processor groups

I have noticed the SQX does not make use of CPU threads that span across multiple processor groups. I’m asking for you to add support for processor groups which will allow me to make use of my 64c/128t CPU (AMD Threadripper 3990X).


 

For a detailed explanation of processor groups look here:

Link 1) https://docs.microsoft.com/en-us/windows/win32/procthread/processor-groups#:~:text=Support%20for%20systems%20that%20have,are%20numbered%20starting%20with%200.&text=The%20operating%20system%20minimizes%20the%20number%20of%20groups%20in%20a%20system.

Link 2) https://techgage.com/article/microsoft-processor-groups-beyond-64-thread/


 

When SQX (more specifically, the javaw.exe process) is loaded, Windows assigns the process to the processor group with the lowest load. Now, by default, applications can only make use of threads that are contained within the same processor group. For example, since I have 128 threads I have 2 processor groups each with 64 threads. This means that after the javaw.exe process has been assigned to a random processor group, it can only utilise 64 (50% in my case) threads because that’s the number of threads assigned to that process group. Effectively, I can only use 50% of my total CPU power in a single SQX instance. This is a real issue and fixing this would result in a huge performance improvement.


 

A few things that I have tried:

  • Disabling hyperthreading. This does not resolve this issue, it simply ignores it. Performance is halved based on SQX benchmarks I have made myself.

  • Process Lasso. Third party software cannot solve the underlying issue. It must be solved internally by your developers.

  • Playing with the CPU settings inside of SQX (manually assigned CPU threads to 128 didn’t allow SQX to use more than 64 groups because it is not looking outside of the processor group it’s assigned to).


 

I have attached 2 pictures which highlight SQX’s CPU usage whilst running a random generation strategy builder. Again, to be clear, this is not an issue with my personal SQX setup, or an issue regarding my windows version (all Windows 10 versions – Pro, Workstation, Server) split threads into processor groups of maximum 64 threads each. SQX needs to be updated and be made “processor group aware” to allow users to utilise all CPU threads they have available. Without this, SQX is simply not making use of the hardware it’s running on (excluding users who have less than 64 threads – in this case it’s fine since there is only 1 processor group).


 

If you need any additional information please let me know. I’m happy to be a “beta tester” for any solutions you would like to test.


 

Many Thanks and Best Regards,


 

Attachments
image-0.png
(192.50 KiB)
image-1.png
(20.07 KiB)
  • Votes +12
  • Project StrategyQuant X
  • Type Bug
  • Status New
  • Priority Normal

History

TV
#1

Tomas Vanek

23.10.2020 18:56

Task created

m
#2

mabi

26.10.2020 12:05
Voted for this task.
BS
#3

BobS

27.10.2020 14:45
Voted for this task.
N
#4

nathan

28.10.2020 09:54

Attachment Task Manager.jpg added

Attachment Task Manager.jpg deleted

Attachment Task Manager.jpg added

Task Manager.jpg
(167.10 KiB)
I agree, this would be nice, but if I remember correctly I think its a limitation of Java.


I'm running a server with 60 cores and had the same issue - 50% utilization.


I got around it by disabling logical cores (hyper threading) at the BIOS level. After that task manager showed 60 cores and 60 logical processors and 100% utilization in build. Before adjusting at the BIOS task manger showed 60 cores and 120 logical cores with 50% utilization.

WH
#5

Stormin_Norman

11.11.2020 10:24
"I got around it by disabling logical cores (hyper threading) at the BIOS level. After that task manager showed 60 cores and 60 logical processors and 100% utilization in build. Before adjusting at the BIOS task manger showed 60 cores and 120 logical cores with 50% utilization."


I just wanted to make it clear that this does not "get around the problem". You're simply avoiding it and missing out on a huge amount of processing power.

g
#6

geektrader

11.11.2020 17:23
Voted for this task.
N
#7

nathan

12.11.2020 11:21

Stormin_Norman

 1 day ago
"I got around it by disabling logical cores (hyper threading) at the BIOS level. After that task manager showed 60 cores and 60 logical processors and 100% utilization in build. Before adjusting at the BIOS task manger showed 60 cores and 120 logical cores with 50% utilization."


I just wanted to make it clear that this does not "get around the problem". You're simply avoiding it and missing out on a huge amount of processing power.



How so? The physical processors are 100% utilized? If I compare it to my 3 other servers with16 cores each, but with hyper threading on on these servers, the strategies generated per hour using identical projects is almost 4 times greater on the 60 physical core server with hyperthreading off, than on the 16 physical core server with hyperthreading on. If am missing something please let me know, would love some more computing power!

k
#8

Karish

11.05.2021 03:52
Voted for this task.
KB
#9

kbtech

08.07.2021 18:22
Voted for this task.
MW
#10

Goon

09.07.2021 03:50
Voted for this task.
E
#11

Emmanuel

24.11.2022 20:12
Voted for this task.
KB
#12

Kevin

03.01.2023 17:23
Voted for this task.
KB
#13

Kevin

03.01.2023 17:38
This is not related to Java as has been commented above. It is related to the way Windows handles cores in processor groups as described in the description of this task.
My temporary solution has been to use Linux instead of Windows, as Linux doesn't have this issue. But SQX has its own problems in Linux, so it's not the ideal solution either. It seems as though the preferred OS version for the dev team is Windows, but this core limitation in Windows has still not been addressed after all these years.
b
#14

binhsir

04.01.2023 07:34
Voted for this task.
TV
#15

Tomas Vanek

25.01.2023 16:43

update, forwarded message from one of our customer


Great news, I just found the answer by simply testing it out on a 112-core machine with Windows Server 2022 in the Google Compute Engine (which allows creating machines with up to 400+ cores and 10TB of RAM) and indeed, with Windows Server 2022 and Windows 11 the 64 core limitation/1 processor group per application is finally gone! Any application can now utilize the full core amount without being NUMA (processor grouping) aware, including JavaVM/GraalVM (and hence SQX) which I've just got to max out all 112 cores at once!


Microsoft also notes this in the blue extra box on this site:


"Starting with Windows 11 and Windows Server 2022, it is no longer the case that applications are constrained by default to a single processor group. Instead, processes and their threads have processor affinities that by default span all processors in the system, across multiple groups on machines with more than 64 processors."


https://learn.microsoft.com/en-us/windows/win32/procthread/processor-groups ;

e
#16

earei

14.02.2023 05:35
Voted for this task.
JT
#17

TiNTa

14.02.2023 22:52
Voted for this task.
MV
#18

mvilla@itcm.es

1 week ago
Voted for this task.

Votes: +12

Drop files to upload

or

choose files

Max size: 5MB

Not allowed: exe, msi, application, reg, php, js, htaccess, htpasswd, gitignore

...
Wait please