<- Back to Tasks

[129Dev6] CPU affinity flag binds to a core instead of processor thread (Windows/Linux)

Hello,

Once affinity flag is enabled blocking thread pool manager reserves a whole core which usually has more than one thread (SMT2). The problem arises that pool manager does not reflect multiple threads per core factor and library fails registration for half of cores which is logical.

On Linux:

it's able to see right topology problem is manifested clearly:

BTW: in example one core is reserved for GUI but it's not reflected in initial affinity map: "No isolated CPUs found, so assuming CPUs 1 to 7 available."


00:10:23.560 [main] DEBUG c.s.g.utils.CoreUsagesEvaluator - Using all cores but reserving:1 cores 00:10:23.561 [main] DEBUG c.s.g.utils.CoreUsagesEvaluator - Total cores available: 8, using: 7 00:10:23.566 [main] DEBUG c.s.g.c.p.MultithreadComputePerformer - Using: 7 cores. 00:10:25.697 [main] DEBUG c.s.gridlib.concurrent.ThreadPool - Preparing thread executor: #0, additional:false 00:10:32.893 [Blocking computeThread #0] DEBUG c.s.gridlib.concurrent.ThreadPool - Binding cpu's core 00:10:32.893 [main] DEBUG c.s.gridlib.concurrent.ThreadPool - Preparing thread executor: #1, additional:false 00:10:32.893 [Blocking computeThread #1] DEBUG c.s.gridlib.concurrent.ThreadPool - Binding cpu's core 00:12:00.352 [main] DEBUG c.s.gridlib.concurrent.ThreadPool - Preparing thread executor: #2, additional:false 00:12:00.353 [Blocking computeThread #2] DEBUG c.s.gridlib.concurrent.ThreadPool - Binding cpu's core 00:12:00.359 [main] DEBUG c.s.gridlib.concurrent.ThreadPool - Preparing thread executor: #3, additional:false 00:12:00.359 [Blocking computeThread #3] DEBUG c.s.gridlib.concurrent.ThreadPool - Binding cpu's core 00:12:00.363 [main] DEBUG c.s.gridlib.concurrent.ThreadPool - Preparing thread executor: #4, additional:false 00:12:00.363 [Blocking computeThread #4] DEBUG c.s.gridlib.concurrent.ThreadPool - Binding cpu's core 00:12:00.367 [main] DEBUG c.s.gridlib.concurrent.ThreadPool - Preparing thread executor: #5, additional:false 00:12:00.367 [Blocking computeThread #5] DEBUG c.s.gridlib.concurrent.ThreadPool - Binding cpu's core 00:12:00.372 [main] DEBUG c.s.gridlib.concurrent.ThreadPool - Preparing thread executor: #6, additional:false 00:12:00.372 [Blocking computeThread #6] DEBUG c.s.gridlib.concurrent.ThreadPool - Binding cpu's core 00:12:00.380 [main] DEBUG c.s.g.c.p.MultithreadComputePerformer - Using: 7 cores. 00:12:00.442 [Blocking computeThread #1] INFO net.openhft.affinity.AffinityLock - No isolated CPUs found, so assuming CPUs 1 to 7 available. 00:12:22.092 [Blocking computeThread #1] INFO net.openhft.affinity.LockInventory - Assigning core 2: cpus 4, 5 to Thread[Blocking computeThread #1,5,main] 00:15:49.847 [Blocking computeThread #0] INFO net.openhft.affinity.LockInventory - Assigning core 1: cpus 2, 3 to Thread[Blocking computeThread #0,5,main] 00:16:45.170 [Blocking computeThread #2] WARN net.openhft.affinity.LockInventory - No reservable Core for Thread[Blocking computeThread #2,5,main] 00:16:45.171 [Blocking computeThread #2] INFO net.openhft.affinity.AffinityLock - Assigning cpu 6 to Thread[Blocking computeThread #2,5,main] 00:16:50.643 [Blocking computeThread #3] WARN net.openhft.affinity.LockInventory - No reservable Core for Thread[Blocking computeThread #3,5,main] 00:16:50.645 [Blocking computeThread #3] INFO net.openhft.affinity.AffinityLock - Assigning cpu 1 to Thread[Blocking computeThread #3,5,main] 00:16:50.651 [Blocking computeThread #5] WARN net.openhft.affinity.LockInventory - No reservable Core for Thread[Blocking computeThread #5,5,main] 00:16:50.653 [Blocking computeThread #5] WARN net.openhft.affinity.LockInventory - No reservable CPU for Thread[Blocking computeThread #5,5,main] 00:16:50.655 [Blocking computeThread #6] WARN net.openhft.affinity.LockInventory - No reservable Core for Thread[Blocking computeThread #6,5,main] 00:16:50.657 [Blocking computeThread #6] WARN net.openhft.affinity.LockInventory - No reservable CPU for Thread[Blocking computeThread #6,5,main] 00:16:50.658 [Blocking computeThread #4] WARN net.openhft.affinity.LockInventory - No reservable Core for Thread[Blocking computeThread #4,5,main] 00:16:50.660 [Blocking computeThread #4] WARN net.openhft.affinity.LockInventory - No reservable CPU for Thread[Blocking computeThread #4,5,main]


Windows: details latter..


Proposal: As quick fix use acquireCore instead of acquireLock
Attachments
No attachments
  • Votes +1
  • Project StrategyQuant X
  • Type Bug
  • Status Fixed
  • Priority Normal

History

DB
#1

Enyx

14.07.2020 08:56

Task created

DB
#2

Enyx

14.07.2020 08:57

Correction..

Proposal: As quick fix use acquireLock  instead of acquireCore ..

DB
#3

Enyx

14.07.2020 09:39

Windows is also showing some problems. Note is a VMware VM which has a disabled HW topology presentation to guest which means OS does not see core/thread mapping but just plain cores.


Anyway still some reservations are not placed as seen below.


09:34:02.028 [main] DEBUG c.s.g.utils.CoreUsagesEvaluator - Using percentage of cores:100%. Total cores:8. Cores for usage:8 09:34:02.028 [main] DEBUG c.s.g.utils.CoreUsagesEvaluator - Total cores available: 8, using: 8 09:34:02.030 [main] DEBUG c.s.g.c.p.MultithreadComputePerformer - Using: 8 cores. 09:34:02.032 [main] DEBUG c.s.gridlib.concurrent.ThreadPool - Preparing thread executor: #0, additional:false 09:34:02.033 [main] DEBUG c.s.gridlib.concurrent.ThreadPool - Preparing thread executor: #1, additional:false 09:34:02.033 [Blocking computeThread #0] DEBUG c.s.gridlib.concurrent.ThreadPool - Binding cpu's core 09:34:02.033 [Blocking computeThread #1] DEBUG c.s.gridlib.concurrent.ThreadPool - Binding cpu's core 09:34:02.033 [main] DEBUG c.s.gridlib.concurrent.ThreadPool - Preparing thread executor: #2, additional:false 09:34:02.033 [Blocking computeThread #2] DEBUG c.s.gridlib.concurrent.ThreadPool - Binding cpu's core 09:34:02.035 [main] DEBUG c.s.gridlib.concurrent.ThreadPool - Preparing thread executor: #3, additional:false 09:34:02.035 [Blocking computeThread #3] DEBUG c.s.gridlib.concurrent.ThreadPool - Binding cpu's core 09:34:02.036 [main] DEBUG c.s.gridlib.concurrent.ThreadPool - Preparing thread executor: #4, additional:false 09:34:02.037 [Blocking computeThread #4] DEBUG c.s.gridlib.concurrent.ThreadPool - Binding cpu's core 09:34:02.038 [main] DEBUG c.s.gridlib.concurrent.ThreadPool - Preparing thread executor: #5, additional:false 09:34:02.039 [main] DEBUG c.s.gridlib.concurrent.ThreadPool - Preparing thread executor: #6, additional:false 09:34:02.039 [Blocking computeThread #6] DEBUG c.s.gridlib.concurrent.ThreadPool - Binding cpu's core 09:34:02.039 [Blocking computeThread #5] DEBUG c.s.gridlib.concurrent.ThreadPool - Binding cpu's core 09:34:02.039 [main] DEBUG c.s.gridlib.concurrent.ThreadPool - Preparing thread executor: #7, additional:false 09:34:02.040 [Blocking computeThread #7] DEBUG c.s.gridlib.concurrent.ThreadPool - Binding cpu's core 09:34:02.044 [main] DEBUG c.s.g.c.p.MultithreadComputePerformer - Using: 8 cores. 09:34:02.046 [Blocking computeThread #0] INFO net.openhft.affinity.AffinityLock - No isolated CPUs found, so assuming CPUs 1 to 7 available. 09:34:02.115 [Blocking computeThread #0] INFO net.openhft.affinity.LockInventory - Assigning core 6: cpus 6 to Thread[Blocking computeThread #0,5,main] 09:34:02.117 [Blocking computeThread #1] INFO net.openhft.affinity.LockInventory - Assigning core 5: cpus 5 to Thread[Blocking computeThread #1,5,main] 09:34:02.117 [Blocking computeThread #7] INFO net.openhft.affinity.LockInventory - Assigning core 4: cpus 4 to Thread[Blocking computeThread #7,5,main] 09:34:02.118 [Blocking computeThread #5] INFO net.openhft.affinity.LockInventory - Assigning core 3: cpus 3 to Thread[Blocking computeThread #5,5,main] 09:34:02.119 [Blocking computeThread #3] INFO net.openhft.affinity.LockInventory - Assigning core 2: cpus 2 to Thread[Blocking computeThread #3,5,main] 09:34:02.120 [Blocking computeThread #6] INFO net.openhft.affinity.LockInventory - Assigning core 1: cpus 1 to Thread[Blocking computeThread #6,5,main] 09:34:02.127 [Blocking computeThread #4] WARN net.openhft.affinity.LockInventory - No reservable Core for Thread[Blocking computeThread #4,5,main] 09:34:02.127 [Blocking computeThread #4] WARN net.openhft.affinity.LockInventory - No reservable CPU for Thread[Blocking computeThread #4,5,main] 09:34:02.128 [Blocking computeThread #2] WARN net.openhft.affinity.LockInventory - No reservable Core for Thread[Blocking computeThread #2,5,main] 09:34:02.128 [Blocking computeThread #2] WARN net.openhft.affinity.LockInventory - No reservable CPU for Thread[Blocking computeThread #2,5,main]



JK
#4

Insanity82007

14.07.2020 12:24
Voted for this task.
PS
#5

Petr Šomek

16.07.2020 17:51

Status changed from New to In progress

MF
#6

Mark Fric

07.04.2021 10:47

Status changed from In progress to Fixed

this should be fixed, it should bind it to thread now
Votes: +1

Drop files to upload

or

choose files

Max size: 5MB

Not allowed: exe, msi, application, reg, php, js, htaccess, htpasswd, gitignore

...
Wait please