Parameter Reactivity and Selection

Perhaps there is already a quick and easy way to do this in SQX, but haven found it yet :)


It would be great if, in the [Optimizer>Optimizer Type>Simple] there was the option to 'check for parameter reactivity', whereby the optimization proceeded one parameter at a time, holding the others constant, in order to ascertain which parameters had the most significant impact on performance (the objective function, or ranking/fitness in SQX) The result could be a simple table with the range of variance in performance per parameter. From here, you could quickly pick the top one or two parameters to optimize. I am not a fan of optimizing more than one or two strategy parameters. From Pardo (below)...



Trading strategies can have quite a range in number of rules and formulae that can accept parameters that might benefit from optimization. It is always best to have the smallest number of optimizable parameters possible, and yet still provide the required flexibility. The greater the number of parameters to be optimized, the more likely it is that the results will be overfitted or optimized incorrectly.


Consequently, it is best to make every effort at the outset to identify the key strategy parameters to be incorporated into the optimization framework. Of course, the key parameters are those that have the most significant impact on performance. If a parameter has little impact on performance, there is no reason to include it in the optimization. Instead, a fixed value or constant should be used for these less significant parameters.


If the relative significance of the model parameters is not known a priori, then an added empirical step must be performed to determine their significance. The easiest way to do this is to scan a relevant parameter range for each strategy variable, one at a time, holding the other strategy variables constant at a known reasonable value. A parameter is important if such a scan shows dramatic performance change. Conversely, a parameter is considered of less importance if such a scan shows little or no change in performance. Of course, such a test also raises the more important question as to whether this variable is needed at all in the strategy. It is always best to simplify a trading strategy to only those parameters, rules, and formulae that are necessary to produce peak performance. So, if the elimination of peripheral rules and parameters does not detract significantly from performance, reduce the strategy to just its necessary components.


Pardo, Robert. The Evaluation and Optimization of Trading Strategies (Wiley Trading) (pp. 217-218). Wiley. Kindle Edition. 

Attachments
No attachments
  • Votes +6
  • Project StrategyQuant X
  • Type Feature
  • Status Archived
  • Priority Normal
  • Assignee None
  • Milestone Archived (To be done later)

History

N
#1

nathan

23.01.2019 12:55

Task created

Rr
#2

Partizanas

23.01.2019 16:28
Voted for this task.
N
#3

nathan

23.01.2019 21:06
Voted for this task.
IH
#4

clonex / Ivan Hudec

25.01.2019 09:15
Voted for this task.
KL
#5

kainc301

22.03.2019 19:59
Voted for this task.
KL
#6

kainc301

22.03.2019 20:10
I believe this will be a very important feature to implement as the implementation of this can help with other things down the road.  For example, this can set precedent for what parameters a neural network should look into modifying first as it establishes priority. 


As such, I recommend creating a "score" (for example, from 1-10) to assign to each parameter as it relates to the reactivity. Parameters that cause massive variance in the strategy can be ranked higher while params that don't cause much variance can be ranked lower. 


I would also recommend params are tested with wider step variance in order to make  this process faster as the goal is just to see an overview of reactivity. For example, if testing a moving average, the step could be 20 instead of a lower number that would be used for optimization such as 5. The outer deviations would have to be more narrow too so in this example we wouldn't do something like test a moving average all the way up to a period of 120 as we would in regular optimization.  

MF
#7

Mark Fric

16.02.2020 13:52

Status changed from New to Archived

m
#8

Mariano

14.04.2020 17:41
Voted for this task.
b
#9

bentra

03.05.2021 23:16
Voted for this task.

Votes: +6

Drop files to upload

or

choose files

Max size: 5MB

Not allowed: exe, msi, application, reg, php, js, htaccess, htpasswd, gitignore

...
Wait please