<- Back to Tasks

Randomized Out of Sample

Randomized Out of Sample

More details here:

Screenshot attached



--
Another thing that i forgot to point out!**

With this feature hopefully implemented we would want another thing that was specified inside the article above,
Randomization of OOS crosscheck (unless it can be implemented somehow into the MC retest)


the idea is to have a separated crosscheck test and use the INITIAL VALUES that we have put it at the main page:
• Randomly pick % value
• Limit Deviations to # sections value


and use the data full historical data period we selected (from the screenshot's example):
2003.08.04 ~ 2018.01.01

and use BOTH values to re-randomize the OOS 1000 times and get new 1000 OOS samples again to compare with,
its kinda fits into the MC retest but im not sure if it can be used with the currently built MC retest, because MC retest will only work with the INITIAL OOS that we will have, and it wont be able to look at the full data (2003.08.04 ~ 2018.01.01) and re-randomize the OOS sample again with the 2 INITIAL selected inputs (Randomly pick % value + Limit Deviations to # sections value) or can it?..


Anyway this is a great feature and i hope it could be implemented sooner than later..
Thanks.


Attachments
Randomization of OOS.png
Randomization of OOS.png
(182.63 KiB)
  • Votes +5
  • Project StrategyQuant X
  • Type Feature
  • Status New
  • Priority Normal

History

k
#1

Karish

15.11.2022 11:11

Task created

AA
#2

Alex

15.11.2022 11:14
If this is done, also add # of OOS sections. (30% in 4 sections, 20% in 6 sections)
E
#3

Emmanuel

15.11.2022 11:23
Voted for this task.
k
#4

Karish

15.11.2022 11:33
Voted for this task.
k
#5

Karish

15.11.2022 11:34

Attachment Randomization of OOS new.png added

Randomization of OOS new.png
Randomization of OOS new.png
(233.17 KiB)
Updated the "Vision" Screenshot with Alex's request
HF
#6

Vence_Jo

15.11.2022 11:45
Voted for this task.
CG
#7

Chris G

15.11.2022 11:49
Voted for this task.
k
#8

Karish

15.11.2022 12:09

Attachment Example.png added

Example.png
Example.png
(616.70 KiB)
k
#9

Karish

15.11.2022 12:17

Attachment Randomize Hisorical Bars Order CrossCheck Example Vision.png added

Randomize Hisorical Bars Order CrossCheck Example Vision.png
Randomize Hisorical Bars Order CrossCheck Example Vision.png
(80.08 KiB)
BTW this is very similar and kinda same thing was suggested by me here:

https://roadmap.strategyquant.com/tasks/sq4_9343#comment-102819

https://roadmap.strategyquant.com/tasks/sq4_9089


Putting the Main page's initial setup aside for a minute,

the above suggestion can work on MC re-test so it got me thinking..,

This CAN WORK with the current MC retest engine!,

see my vision in the screenshot attached!




IH
#10

clonex / Ivan Hudec

15.11.2022 13:09
Voted for this task.
k
#11

Karish

15.11.2022 16:31

Attachment Randomize Hisorical Bars Order CrossCheck Example Vision UPDATED.png added

Randomize Hisorical Bars Order CrossCheck Example Vision UPDATED.png
Randomize Hisorical Bars Order CrossCheck Example Vision UPDATED.png
(86.28 KiB)

Added Exact / Resample methods into the menu of options of this MC retest feature,

Just like with-in the MC trades that we can choose Exact / Resample methods

this should be available here as-well,


• Exact means that the test will pick single random Bars in the data only once.

• Resample mean that the test will be allowed to pick single random Bars in the data and use it more than once.

k
#12

Karish

15.11.2022 20:58
It came to my attention at the Discord group that there is a small "bug" that should be fixed before implementing this feature as it should

https://roadmap.strategyquant.com/tasks/sq4_8445

Ok, we will add a special setting for that "Dismiss trades that start in IS period and end in OOS period"

- Tamas

Votes: +5

Drop files to upload

or

choose files

Max size: 5MB

Not allowed: exe, msi, application, reg, php, js, htaccess, htpasswd, gitignore

...
Wait please