"Simulated" and "Exact" methods produce identical results - Are these options enabled yet?

Run two walk forward matrix tests on any strategy using real tick, real spread; the first using the "Simulated" method, the second using the "exact" method. The results will be 100% identical.



Surely they should be quite different, with the 'Exact" method giving poorer (but more accurate) results than the simulated method. In any case, they should not produce 100% identical results/numbers across the board.


Are the two methods actually 'live" yet?



Attachments
No attachments
  • Votes +12
  • Project StrategyQuant X
  • Type Bug
  • Status Refused
  • Priority Normal

History

N
#1

nathan

25.01.2019 08:13

Task created

MF
#2

Mark Fric

27.01.2019 15:50

Status changed from New to Refused

you are right, right now they are the same. 

Exact method is not implemented yet, so even if you chose it, it will run simulated WF.


We plan to implement also Exact method soon, but we have more urgent priorities.

N
#3

nathan

28.01.2019 13:53
Any articles/info avaliable online for the exact v simulated...before commiting real money I would much prefer the most precise evalution method for a startegy. Perhaps simulated is better? 'Exact' could imply otherwise .
b
#4

bentra

16.01.2022 12:18
This is misleading, this dropdown still does absolutely nothing. Please either put a note on it that it isn't working yet or put in the function.


b
#5

bentra

16.01.2022 12:19
Voted for this task.
b
#6

bentra

16.01.2022 13:14
Actually, in cases where we are not doing an exhaustive search and genetics kicks in, there is a look ahead bias!

Explanation: 
sim/sim which is the default and only selection that works currently will use the whole sample at once. The genetics kicks in on the whole sample causing more iterations to converge around the higher fitness iterations on the whole sample. So when the best settings are chosen for a specific segment it is more likely to choose one that works well OOS as there will be more of those to choose from. This could be a very small look-ahead bias but it exists and is not good. 

If we could turn off genetic for the optimizer then the distribution would be random instead of skewed towards what works well on the whole sample so that would be nice to alleviate this look-ahead bias. There is a similar bias on SPP caused by the genetic engine so actually, I will open a feature request. 
IH
#7

clonex / Ivan Hudec

16.01.2022 16:22
Voted for this task.
JJ
#8

jjsb41

16.01.2022 17:12
Voted for this task.
o
#9

Enric

16.01.2022 17:46
Voted for this task.
JH
#10

Jabezz

17.01.2022 02:34
Voted for this task.
JH
#11

Jabezz

17.01.2022 02:36
Yeah it would be nice to know these things when they are in a build but are not actually working due to further development being required. As bentra suggested - a note would be a good idea.
KB
#12

kbtech

17.02.2022 02:41
Voted for this task.
m
#13

mabi

17.02.2022 11:40
Ha!  I remember Norm saying on forum that we much use exact methode it is the only one that are useful the other methodes are shit.And it has never worked gofigure !


m
#14

mabi

17.02.2022 11:40
Voted for this task.
CG
#15

Chris G

16.04.2022 18:25
Voted for this task.
IN
#16

altenburg90

18.04.2022 02:03
Voted for this task.
JJ
#17

Jackson

17.11.2022 09:26
Voted for this task.
k
#18

Karish

17.11.2022 10:39
Voted for this task.
JB
#19

0xjoeblack@proton.me

18.11.2022 15:21
Voted for this task.

Votes: +12

Drop files to upload

or

choose files

Max size: 5MB

Not allowed: exe, msi, application, reg, php, js, htaccess, htpasswd, gitignore

...
Wait please