Hi,
I’ve detected two major failures in the Walk Forward engines of StrategyQuantX. In my view it nearly invalidates any WF analysis in the software. Images attached
Failure 1 - When running Walk Forward optimization or matrix, the software considers the last “In Sample-IS” day as the first “Out of Sample-OOS” day. See image below. This is a major mistake and results in the classic “data snooping”. The in sample period is looking one day into the future (OOS) and then the engine considers that the last IS day (07.01.2010 in the image) is ALSO out of sample, when it is not (because it was used for IS optimization). First I thought that Strategy Quant behaved similarly to Metatrater and ignored the last day of the range, but after checking many IS and OOS ranges, I realized that actually it uses the last IS day also as the first OOS day.
In real life, an in sample period ending on 07.01.2010 would have an OOS starting on the 08.01.2010. This is a very serious error in the software.
Failure 2 - The walk forward engine also uses the last OOS day as the first day for the next OOS segment. See image below. An OOS segment ending on 12.01.2010 would have the next OOS starting on 13.01.2010 but the software duplicates the trading session of 12.01.2010. This means that any trade taken in the last OOS day will happen again in the next OOS segment, and completely destroy the WF analysis. See duplicate trades in attached image
As a result of these error, the software not only creates look ahead bias, but also suffers from data snooping and creates invalid list of trades for OOS segments. As an example of what that could cause, you can see below that absolutely any strategy with few trades will have an excellent performance out of sample on walk forward analysis, simply because the last trade in sample occurs as the first trade out of sample.
Would you look into this?
Best regards
Jose