The Mudder Report 2.0

Where is the hunt going for the next 3 to 6 months?

Hi Hunters,

I hope all is well. This is just a quick update on the goals and current objectives of the hunt.

Please remember that our goal is like yours. We are searching for the holy grail of automated trade strategy. Yes, we release some of our favorite strategies every two weeks, but our goal is not to sell strategies. If that was our goal, we wouldn’t be questioning backtest results. This is why you don’t see the subject of backtest accuracy discussed often. If all you’re trying to do is sell strategies, it undermines your goal.

Another reason you don’t see the subject of backtest accuracy discussed often is because backtests are the best defense we have against losing large sums of money, so there’s a real desire to look the other way, to pretend. You’ll see discussions about overfitting, but not backtest accuracy, presumably because the former can be corrected, but the latter takes time to control. You actually have to understand the limitations of the backtest engine and the impact it has on each individual strategy.

What we’ve had to do is focus on something that is rarely discussed: what can we do as traders to make strategies less prone to backtest error? Put another way, what can we do to aid the simulation? Like the game of tennis, you can’t assume that simply swinging the racket is going to get the ball over the net. So, the value we bring is not in our strategies, but in our approach to strategy formulation. With every strategy published, we learn more about the limitations of the simulation. And now, instead of just hitting the ball, we’re using a variety of different strokes depending on what the situation calls for.

For example, when we conduct an optimization, we never optimize on indicator parameter, because this leads to overfitting. However, we do optimize on data series type (minute based) and instrument because these help to identify the best market to trade the strategy in.

As another example, instead of range, tick or Renko based strategies, we’re using minute based strategies. If it’s not minute based, the simulation has a tendency to lag. And, the longer the minute-based data series, the better, because it reduces the likelihood that the strategy creates false or bad signals. I believe this is also why longer or higher frequency minute-based time series tend to have higher profit factors. Of course the trade-off to longer minute-based time series is fewer trade opportunities, but that can be countered with the use of additional contracts. We’re looking at additional contracts using the same strategy now, but our next step is to create a portfolio that uses the best strategy for each contract.

Let’s take a look at Strategy 49 and 7 as examples.