Status changed from New to Refused
It is not based on OHLC prices but ticks - where each O,H,L,C price is represented as tick. Each tick can be modified with the same probability.
You can make your own data changing algorithm by copying and modifying the default one.
public void modifyData(IRandomGenerator rng, TickEvent tickEvent, double globalATR) {
double dblProbability = ((double) Probability/ 100.0d);
if(rng.probability(dblProbability)) {
// we should change this price
double ask = tickEvent.getAsk();
double bid = tickEvent.getBid();
double spread = ask - bid;
int change;
if(relativeMaxChange <= 0) {
change = rng.nextInt(MaxChange);
} else {
change = rng.nextInt(relativeMaxChange);
}
double dblChange = ((double) change)/ 100.0d;
double priceChange = 2 * globalATR * dblChange;
bid = (rng.nextInt(2) == 0 ? bid + priceChange : bid - priceChange);
tickEvent.setBid(bid);
tickEvent.setAsk(bid + spread);
}
}
how every MC run could have better stats than original strategy?
Status changed from Refused to In progress
We'll check it.
Attachment MC Price 100 100 settings.png added
Attachment MC Price 100 100.png added
Attachment EURGBP MC Price 100 100 Strategy 22356.sqx added
Attachment project.cfx added
With the settings of 100% probability and 100% of ATR I would expect to see similar results to the backtests on additional markets.
Status changed from In progress to Fixed
- you can select % and size of change for Up and Down side separately
- there is optional check to keep gaps (or no gaps) between close and next open
Global ATR is average ATR(14) of the data, the change is defined as % of this ATR.
As for the "strange" results - the attached examples are most probably strategies that benefit fom increased volatility which happens with simulation configured like this.
> With the settings of 100% probability and 100% of ATR I would expect to see similar results to the backtests on additional markets.
I don't see why you'd expect something like this. With change of 100% ATR you'd have much bigger volatility of the data, possibly much bigger than any other real existing symbols.
You are welcome to make your own version, if you'll send it to us we could add it to the official distribution.
Attachment Screenshot 98.png added
/* * Copyright (c) 2017-2018, StrategyQuant - All rights reserved. * * Code in this file was made in a good faith that it is correct and does what it should. * If you found a bug in this code OR you have an improvement suggestion OR you want to include * your own code snippet into our standard library please contact us at: * https://roadmap.strategyquant.com * * This code can be used only within StrategyQuant products. * Every owner of valid (free, trial or commercial) license of any StrategyQuant product * is allowed to freely use, copy, modify or make derivative work of this code without limitations, * to be used in all StrategyQuant products and share his/her modifications or derivative work * with the StrategyQuant community. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, * INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR * PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES * OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. * */ package SQ.MonteCarlo.Retest; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import com.strategyquant.lib.*; import com.strategyquant.datalib.*; import com.strategyquant.datalib.consts.Precisions; import com.strategyquant.tradinglib.*; @ClassConfig(name="Randomize history data", display="Randomize history data v2.0, with max price change #MaxChange# % of PRICE") @Help("<b>Note!</b>If you choose this option then robustness tests will be tested with Selected Timeframe precision, while original strategy will be tested with configured precision.<br/>This might cause difference in results between original strategy and simulations because of different precision.<br/>To avoid this, use Selected Timeframe precision in Setings : Data.") public class RandomizeHistoryData2 extends MonteCarloRetest { public static final Logger Log = LoggerFactory.getLogger(RandomizeHistoryData2.class); //@Parameter(name="Probability change price", defaultValue="20", minValue=1, maxValue=100, step=1) //public int Probability; @Parameter(name="Max change price", defaultValue="10", minValue=1, maxValue=100, step=1) public int MaxChange; private int barsProcessed = 0; private double atrValue = 0; private int atrPeriod = 14; private int relativeMaxChange = -1; //bendedit: we need some persistant variables so we can track price and allow it to wander away from original data private double lastbid=0; private double lastorgbid=0; private long lastticktime=0; //------------------------------------------------------------------------ //------------------------------------------------------------------------ //------------------------------------------------------------------------ public RandomizeHistoryData2() { super(MonteCarloTestTypes.ModifyData); } //------------------------------------------------------------------------ public void modifyData(IRandomGenerator rng, TickEvent tickEvent, double globalATR) { //double dblProbability = ((double) Probability/ 100.0d); //if(rng.probability(dblProbability)) ;{ // bendedit: I don't see a point in probability of change, lets just always change every tick, it will be much smoother now that we turfed ATR from the equation. // we should change this price double ask = tickEvent.getAsk(); double bid = tickEvent.getBid(); double spread = ask - bid; double tickorgchange=0; long currentticktime = tickEvent.getTime(); //bendedit: check if first tick, if not then set original tick change. if (lastticktime <= currentticktime && lastticktime!=0) { //bendedit: calculate the price change from last original tick to the current tick from original data tickorgchange=bid-lastorgbid; //debug("MC price randomizer", "tickorgchange is: "+tickorgchange); //debug("MC price randomizer", "time is: "+currentticktime); //debug("MC price randomizer", "last time is: "+lastticktime); //debug("MC price randomizer", "EOL"); } else { //bendedit: in case of first tick there is no change. lastbid=bid; tickorgchange=0; } //bendedit:set last original data bid in to variable before we change bid, we only need it to calculate the price change next time. lastorgbid=bid; //bendedit:store the ticktime. lastticktime = currentticktime; int change; if(relativeMaxChange <= 0) { change = rng.nextInt(MaxChange); } else { change = rng.nextInt(relativeMaxChange); } double dblChange = ((double) change)/ 10000.0d; //bendedit: Modding every tick and allowing price to move far away from original data therefore we need to use much smaller adjustments. //bendedit: I chose a percent of a percent //bendedit: changed from using ATR to just use tick price change of original data. double priceChange = tickorgchange * dblChange; //bendedit: set the working bid relative to the last bid instead of itself //bid = (rng.nextInt(2) == 0 ? bid + priceChange : bid - priceChange); bid = (rng.nextInt(2) == 0 ? lastbid + tickorgchange + priceChange : lastbid + tickorgchange - priceChange); tickEvent.setBid(bid); tickEvent.setAsk(bid + spread); //bendedit:set last bid in to variable for next time. lastbid = bid; //} } //------------------------------------------------------------------------ public void initSettings(SettingsMap settings) { super.initSettings(settings); // compute relative Max Change according to chart TF and precision ChartSetup chartSetup = (ChartSetup) settings.get(SettingsKeys.BacktestChart); if(chartSetup != null) { int testPrecision = chartSetup.getTestPrecision(); if(testPrecision == Precisions.SelectedTF) { // do nothing for selected TF precision relativeMaxChange = MaxChange; return; } // it is higher precision, set MaxChange to 1/3 of the value // because the change is applied o every minute bar / tick we have to lower the MaxChange // so that the resulting full bar in target timeframe (for example H1) isn't changed too much //bendedit: Hopefully we don't need this adjustment anymore since we aren't using globalATR which is inflated by higher timeframes. deviding by 3 arbitrarily was suspect anyways. If we still need to adjust, I'll hopefully find a better way. //bendedit: Now that we are doing change % instead of atr %, we may not need any further adjustments relative to precision or TF. Turned off this equalizer for now. //relativeMaxChange = MaxChange / 3; relativeMaxChange = MaxChange; if(relativeMaxChange <= 0) { relativeMaxChange = 1; } } } }
Attachment Screenshot 99.png added
Attachment Screenshot 100.png added
Status changed from Fixed to In progress
/* * Copyright (c) 2017-2018, StrategyQuant - All rights reserved. * * Code in this file was made in a good faith that it is correct and does what it should. * If you found a bug in this code OR you have an improvement suggestion OR you want to include * your own code snippet into our standard library please contact us at: * https://roadmap.strategyquant.com * * This code can be used only within StrategyQuant products. * Every owner of valid (free, trial or commercial) license of any StrategyQuant product * is allowed to freely use, copy, modify or make derivative work of this code without limitations, * to be used in all StrategyQuant products and share his/her modifications or derivative work * with the StrategyQuant community. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, * INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR * PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES * OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. * */ package SQ.MonteCarlo.Retest; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import com.strategyquant.lib.*; import com.strategyquant.datalib.*; import com.strategyquant.datalib.consts.Precisions; import com.strategyquant.tradinglib.*; @ClassConfig(name="Randomize history data", display="Randomize history data v2.0, with max change #MaxChange# % of TICK PRICE CHANGES") @Help("<b>Note!</b>If you choose this option then robustness tests will be tested with Selected Timeframe precision, while original strategy will be tested with configured precision.<br/>This might cause difference in results between original strategy and simulations because of different precision.<br/>To avoid this, use Selected Timeframe precision in Setings : Data.") public class RandomizeHistoryData2 extends MonteCarloRetest { public static final Logger Log = LoggerFactory.getLogger(RandomizeHistoryData2.class); //@Parameter(name="Probability change price", defaultValue="20", minValue=1, maxValue=100, step=1) //public int Probability; @Parameter(name="Max change price", defaultValue="10", minValue=1, maxValue=100, step=1) public int MaxChange; private int barsProcessed = 0; private double atrValue = 0; private int atrPeriod = 14; private int relativeMaxChange = -1; //bendedit: we need some persistant variables so we can track price and allow it to wander away from original data private double lastbid=0; private double lastorgbid=0; private long lastticktime=0; //------------------------------------------------------------------------ //------------------------------------------------------------------------ //------------------------------------------------------------------------ public RandomizeHistoryData2() { super(MonteCarloTestTypes.ModifyData); } //------------------------------------------------------------------------ public void modifyData(IRandomGenerator rng, TickEvent tickEvent, double globalATR) { //double dblProbability = ((double) Probability/ 100.0d); //if(rng.probability(dblProbability)) { // bendedit: I don't see a point in probability of change, lets just always change every tick, it will be much smoother now that we turfed ATR from the equation. // we should change this price double ask = tickEvent.getAsk(); double bid = tickEvent.getBid(); double spread = ask - bid; double tickorgchange=0; long currentticktime = tickEvent.getTime(); //bendedit: check if first tick, if not then set original tick change. if (lastticktime <= currentticktime && lastticktime!=0) { //bendedit: calculate the price change from last original tick to the current tick from original data tickorgchange=bid-lastorgbid; //debug("MC price randomizer", "tickorgchange is: "+tickorgchange); //debug("MC price randomizer", "time is: "+currentticktime); //debug("MC price randomizer", "last time is: "+lastticktime); //debug("MC price randomizer", "EOL"); } else { //bendedit: in case of first tick there is no change. lastbid=bid; tickorgchange=0; } //bendedit:set last original data bid in to variable before we change bid, we only need it to calculate the price change next time. lastorgbid=bid; //bendedit:store the ticktime. lastticktime = currentticktime; int change; if(relativeMaxChange <= 0) { change = rng.nextInt(MaxChange); } else { change = rng.nextInt(relativeMaxChange); } double dblChange = ((double) change)/ 10000.0d; //bendedit: Modding every tick and allowing price to move far away from original data therefore we need to use much smaller adjustments. //bendedit: I chose a percent of a percent //bendedit: changed from using ATR to just use tick price change of original data. double priceChange = tickorgchange * dblChange; //bendedit: set the working bid relative to the last bid instead of itself //bid = (rng.nextInt(2) == 0 ? bid + priceChange : bid - priceChange); bid = (rng.nextInt(2) == 0 ? lastbid + tickorgchange + priceChange : lastbid + tickorgchange - priceChange); tickEvent.setBid(bid); tickEvent.setAsk(bid + spread); //bendedit:set last bid in to variable for next time. lastbid = bid; //} } //------------------------------------------------------------------------ public void initSettings(SettingsMap settings) { super.initSettings(settings); // compute relative Max Change according to chart TF and precision ChartSetup chartSetup = (ChartSetup) settings.get(SettingsKeys.BacktestChart); if(chartSetup != null) { int testPrecision = chartSetup.getTestPrecision(); if(testPrecision == Precisions.SelectedTF) { // do nothing for selected TF precision relativeMaxChange = MaxChange; return; } // it is higher precision, set MaxChange to 1/3 of the value // because the change is applied o every minute bar / tick we have to lower the MaxChange // so that the resulting full bar in target timeframe (for example H1) isn't changed too much //bendedit: Hopefully we don't need this adjustment anymore since we aren't using globalATR which is inflated by higher timeframes. deviding by 3 arbitrarily was suspect anyways. If we still need to adjust, I'll hopefully find a better way. //bendedit: Now that we are doing change % instead of atr %, we may not need any further adjustments relative to precision or TF. Turned off this equalizer for now. //relativeMaxChange = MaxChange / 3; relativeMaxChange = MaxChange; if(relativeMaxChange <= 0) { relativeMaxChange = 1; } } } //------------------------------------------------------------------------ /** * Clones this MC retest object * * @return the clone * @throws Exception the exception */ @Override public RandomizeHistoryData2 getClone() throws Exception { RandomizeHistoryData2 mc = new RandomizeHistoryData2(); mc.MaxChange = this.MaxChange; return mc; } }
Please attach the strategy where you see this problem and I'll check it.
/* * Copyright (c) 2017-2018, StrategyQuant - All rights reserved. * * Code in this file was made in a good faith that it is correct and does what it should. * If you found a bug in this code OR you have an improvement suggestion OR you want to include * your own code snippet into our standard library please contact us at: * https://roadmap.strategyquant.com * * This code can be used only within StrategyQuant products. * Every owner of valid (free, trial or commercial) license of any StrategyQuant product * is allowed to freely use, copy, modify or make derivative work of this code without limitations, * to be used in all StrategyQuant products and share his/her modifications or derivative work * with the StrategyQuant community. * * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, * INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR * PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES * OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. * */ package SQ.MonteCarlo.Retest; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import com.strategyquant.lib.*; import com.strategyquant.datalib.*; import com.strategyquant.datalib.consts.Precisions; import com.strategyquant.tradinglib.*; @ClassConfig(name="Randomize history data", display="Randomize history bendx mod v1.1, with max change #MaxChange# % of TICK PRICE CHANGES") @Help("<b>Note!</b>If you choose this option then robustness tests will be tested with Selected Timeframe precision, while original strategy will be tested with configured precision.<br/>This might cause difference in results between original strategy and simulations because of different precision.<br/>To avoid this, use Selected Timeframe precision in Setings : Data.") public class RandomizeHistoryData2 extends MonteCarloRetest { public static final Logger Log = LoggerFactory.getLogger(RandomizeHistoryData2.class); //@Parameter(name="Probability change price", defaultValue="20", minValue=1, maxValue=100, step=1) //public int Probability; @Parameter(name="Max change price", defaultValue="40", minValue=0, maxValue=1000, step=1) public int MaxChange; private int barsProcessed = 0; private double atrValue = 0; private int atrPeriod = 14; private int relativeMaxChange = -1; //bendedit: we need some persistant variables so we can track price and allow it to wander away from original data private double lastbid=0; private double lastorgbid=0; private long lastticktime=0; //digits for symbol for rounding private int SymDigMod; //------------------------------------------------------------------------ //------------------------------------------------------------------------ //------------------------------------------------------------------------ public RandomizeHistoryData2() { super(MonteCarloTestTypes.ModifyData); } //------------------------------------------------------------------------ public void modifyData(IRandomGenerator rng, TickEvent tickEvent, double globalATR) { //double dblProbability = ((double) Probability/ 100.0d); //if(rng.probability(dblProbability)) { // bendedit: I don't see a point in probability of change, lets just always change every tick, it will be much smoother now that we turfed ATR from the equation. // we should change this price double ask = tickEvent.getAsk(); double bid = tickEvent.getBid(); double spread = ask - bid; double tickorgchange=0; long currentticktime = tickEvent.getTime(); //bendedit: check if first tick, if not then set original tick change. if (lastticktime <= currentticktime && lastticktime!=0) { //bendedit: calculate the price change from last original tick to the current tick from original data tickorgchange=bid-lastorgbid; //debug("MC price randomizer", "tickorgchange is: "+tickorgchange); //debug("MC price randomizer", "time is: "+currentticktime); //debug("MC price randomizer", "last time is: "+lastticktime); //debug("MC price randomizer", "EOL"); } else { //bendedit: in case of first tick there is no change. lastbid=bid; tickorgchange=0; } //bendedit:set last original data bid in to variable before we change bid, we only need it to calculate the price change next time. lastorgbid=bid; //bendedit:store the ticktime. lastticktime = currentticktime; int change; if(relativeMaxChange <= 0) { change = rng.nextInt(MaxChange); } else { change = rng.nextInt(relativeMaxChange); } double dblChange = ((double) change)/ 10000.0d; //bendedit: changed from using ATR to just use tick price change of original data. double priceChange = tickorgchange * dblChange; //bendedit: set the working bid relative to the last bid instead of itself //bid = (rng.nextInt(2) == 0 ? bid + priceChange : bid - priceChange); bid = (rng.nextInt(2) == 0 ? lastbid + tickorgchange + priceChange : lastbid + tickorgchange - priceChange); //bendedit:set last bid in to variable for next time undrounded lastbid = bid; //round to proper tick for engine bid = SQUtils.round(bid, SymDigMod); //set modded bid and ask for further processing tickEvent.setBid(bid); tickEvent.setAsk(bid + spread); //} } //------------------------------------------------------------------------ public void initSettings(SettingsMap settings) { super.initSettings(settings); // compute relative Max Change according to chart TF and precision // Get symbolinfo for digits ChartSetup chartSetup = (ChartSetup) settings.get(SettingsKeys.BacktestChart); ChartDef chart = chartSetup.getMainChart(); InstrumentInfo symbolInfo = chart.getSymbolInfo(); SymDigMod = symbolInfo.decimals; // debug("MC price randomizer", "EOL:" + SymDigMod ); if(chartSetup != null) { int testPrecision = chartSetup.getTestPrecision(); if(testPrecision == Precisions.SelectedTF) { // do nothing for selected TF precision relativeMaxChange = MaxChange; return; } // it is higher precision, set MaxChange to 1/3 of the value // because the change is applied o every minute bar / tick we have to lower the MaxChange // so that the resulting full bar in target timeframe (for example H1) isn't changed too much //bendedit: Hopefully we don't need this adjustment anymore since we aren't using globalATR which is inflated by higher timeframes. deviding by 3 arbitrarily was suspect anyways. If we still need to adjust, I'll hopefully find a better way. //bendedit: Now that we are doing change % instead of atr %, we may not need any further adjustments relative to precision or TF. Turned off this equalizer for now. //relativeMaxChange = MaxChange / 3; relativeMaxChange = MaxChange; if(relativeMaxChange <= 0) { relativeMaxChange = 1; } } } //------------------------------------------------------------------------ /** * Clones this MC retest object * * @return the clone * @throws Exception the exception */ @Override public RandomizeHistoryData2 getClone() throws Exception { RandomizeHistoryData2 mc = new RandomizeHistoryData2(); mc.MaxChange = this.MaxChange; return mc; } }
Status changed from In progress to Fixed
I refactored the code a little, but with no changes in functionality.