Very Low Processor Usage (10%) when running Walk Forwards; 100% usage for all other test types

Perhaps this is normal and has to do with the nature of walk forwards and how they are dealt with by the processor. Anyway, I always 100% CPU usage when running things like builder, retest additional markets etc...but with Walk Forward Analysis I never see more than 10% CPU usage, with the odd quick peak to 25% every 10 minutes or so. Is this normal behavior and something to do with the way WF tasks are fed to the CPU? Should I never expect more than 10% average usage when running WF tests?



See attached. 

Attachments
Low Resource Usage.jpg
(422.33 KiB)
  • Votes +3
  • Project StrategyQuant X
  • Type Bug
  • Status Fixed
  • Priority Normal

History

N
#1

nathan

23.01.2019 18:56

Task created

N
#2

nathan

24.01.2019 16:04

Attachment Last Steps take significantly more time.jpg added

In this example (see attached) of a Matrix Walk Forward (real tick, real spread, simulated), the first ~5200 steps (97%) are completed in around ~8 minutes, with the remaining ~400 steps (3%) taking an additional ~210 minutes.


Average CPU usage during these first 8 minutes is around 90%. After that, whilst completing the remaining ~400 steps CPU usage averages only around 3 to 5%.


Is this normal behavior / to be expected?

a
#3

Ash24FX

26.01.2019 09:49
Voted for this task.
Dw
#4

Diwi

26.01.2019 17:26
Voted for this task.
OS
#5

trader4711

26.01.2019 18:54
Voted for this task.
MF
#6

Mark Fric

27.01.2019 15:52
I'll verify it. The second part (computing WF from simulated results) shouldn't take that long.
N
#7

nathan

28.01.2019 13:49
The second part, computing WF runs, is what takes the longest by far...it accounts for at least 5% or so in the progress bar but takes longer than the first part (optimizations)...it (the second part) only uses around 5% or the CPU, max 10%.


The first part, if it hooks up, will utilise 100% - but it has to hook up, restarting it several times it will usually get it to  hookup, if it doesnt it will trundel along at only 5% CPU usuage, similar to the second part (which I have never seen hookup and use 100%)


Perhaps its the same issue - hooking up - for both parts. I might get it to hook up in the first stage with 100% CPU usuage by restartying it until it does, but I dont have the same workaround option if if fails to hookup to 100% CPU usage in the second half.


In its present form it reneders the Custom Projects null and void. With WF in the process tree, you cannot leave it alone to run for any lenght of time as it will inneviatably run into this bug.

MF
#8

Mark Fric

28.01.2019 16:24

Status changed from New to Fixed

it is improved. I added one more optimization type that is the fastest one. I'll write a short article about it.

Votes: +3

Drop files to upload

or

choose files

Max size: 5MB

Not allowed: exe, msi, application, reg, php, js, htaccess, htpasswd, gitignore

...
Wait please