Is The Holy Grail of Automated Trading Strategy A Moving Target?
"His mother was a mudder; his father was a mudder"
We are a small group of traders and we spend a lot of time thinking about the best way to proceed, but we don’t always agree. For example, I tend to believe that the holy grail of trade strategy is a moving target with chameleon like qualities. What does this mean from a practical perspective? It means that the holy grail of trade strategy may be more about having the ability to determine which strategy is the best strategy to trade in any given week rather than one strategy that performs well at all times. If we were to analogize it to a horse race, it would be the difference between always having to bet on the same horse or being able to bet on the horse with the best chance of winning.
His Mother Was a Mudder…
I’m a big fan of the show Seinfeld. One of my favorite episodes is when Kramer wins a bet made on a horse race. How does he do it? By betting on the ‘mudder’.
What is a mudder horse? A mudder horse is defined by Merriam-Webster as:
1 : a race horse that runs well on a wet or muddy track.
As the joke goes in the following clip, not only was the horse a mudder, but “His mother was a mudder; his father was a mudder.”
Ultimately, Kramer only wins the bet because the race track was muddy and the horse he bet on was known to perform well in rainy conditions. Of course strategies aren’t living beings, but we can imagine that if horses were robots, their programs might run better under certain conditions. So, you can go looking for the program that runs well in every race or you can figure out which programs run well under certain market conditions. The latter is the case I’m making for defining the holy grail of trade strategy as a moving target rather than a static program. As the dynamics of the race change, so should the horse that wins the race. We’ve tried to accomplish this with conditionals, but there may be a need to change horses altogether.
If we continue with this line of reasoning, instead of looking for the holy grail of trade strategy, we should be looking for the holy grail of trade strategy selection. Don’t fret, we aren’t changing focus. We are still searching for the holy grail of trade strategy, which is defined here, but we don’t want to be so rigid in our definition that it deters us from thinking outside of the box. Additionally, it might be helpful to view the hunt from a different perspective — with a different lens. With that in mind, we’ve created a new performance chart that looks at trade performance on a weekly basis.
Consistency Is Still The Goal
One of the main reasons we’re attracted to this hunt is because trading manually can be highly inconsistent. The hope is that automating a strategy will improve consistency. As a result, we have become laser focused on finding consistent strategies and when we say consistent we mean strategies that perform like their backtest. If you’re on the hunt with us, this is your primary goal as well. All the questions you’ve been asking regarding optimization and overfitting are really getting at consistency. All of your concerns about whether or not to trade a strategy are based on the notion that the backtest results may not be indicative of what you can expect in the future. In other words, if you knew that Strategy 33 would perform consistently with backtest results, you would start trading it immediately. Your only concern, the only thing holding you back, is the fear that the strategy’s future performance won’t be consistent with the backtest. As a result, those of us on the hunt spend a great deal of time answering the following two questions:
How can I improve backtest accuracy?
How do I know that a strategy will perform well in the future?
The answer to question 1 can be found here. Once we felt assured about question 1, we shifted focus to question 2.
We’ve said this before and we’ll say it again, the best way to know if a strategy will perform well in the future is to track the performance of the strategy over time. We already track the performance of strategies every other month — to read the latest update click here — now we want to dig a little deeper by looking at weekly performance. In the same way that you might monitor the performance of a race horse from week-to-week, we are going to monitor the performance of our strategies from week-to-week. The process will likely evolve over time, but this is our first run. Here’s the report:
Click here for a link to all strategy descriptions.
Before analyzing the data within the table, let’s discuss how it’s set up.
The table is split into two sides — weekly performance is on the left and annual performance is on the right (in blue).
The annual stats, which are based on data from 8/30/2020 to 8/30/2021, have already been published here. We have pulled some of those stats into this table for ease of comparison. As a side note, we will be updating annual data to 11/1/2020 to 11/1/2021 in the next week or so.
Now, let’s talk about the weekly update (the left side of the chart).
The first column is the strategy.
The second column is the # of weeks the strategy has been profitable or unprofitable using the following notation (profitable / unprofitable). For example, if a strategy has been unprofitable for the last two weeks it would read 0 / 2. Likewise, If a strategy has been profitable over the last two weeks it would read 2 / 0. This is our way of monitoring how many weeks in a row the strategy has been profitable or unprofitable. It is important to note that there is value in finding a strategy that is consistently unprofitable as well. We can easily turn a consistently unprofitable strategy into a consistently profitable strategy just by reversing the order command.
The third column tells you how many trades were made in the week. Some strategies made hundreds of trades, while others made no trades at all.
The fourth column tells us what the net profitability of the strategy is. Some strategies have a negative net profit while others have a positive net profit.
The fourth column is calculated by taking the net difference of column five and six, gross profit and gross loss, respectively. This column gives a little more insight into the reinvestment rate on the strategy as well. For example, a strategy might make $1,000 in one week, but also require a gross loss of -$50K and a gross profit of $51K to achieve that profitability. This a strategy with a low profit factor. We usually only show the profit factor, but we felt like it was important to show the actual dollar amounts for this report.
The last column is profit factor. Profit factor is calculated by dividing gross profit by gross loss. The higher the profit factor, the lower the reinvestment rate is and the better the strategy. If a strategy has a profit factor of 1.10, it means that the ratio of gross profit to loss is 1.10. In other words, the strategy made 1.10 times more money than it lost during the period.
All strategies marked in yellow have been optimized, which is to say they will likely have inconsistent results, but we have included them in the table for good measure.
All strategies marked in green have shown a favorable trend over the past 2+ weeks. If strategies were horses, these are the horses we would bet on.
So, if I were reading Strategy 1 from left to right, I would say that it:
has had 4 unprofitable weeks, back to back
has had 145 trades
lost $1,710 last week, which continued the negative trend
had a gross profit of $26K and a gross loss of $28K
has a profit factor of .94, which means that it lost more money than it made during the week
Which Strategy Shows The Most Consistency?
Now that we’ve reviewed how the report is set up, let’s talk about what the data is telling us. Here’s the same report for your review again so you don’t have to keep scrolling up:
We found two strategies with a discernible 6 week trend: Strategy 3 and Strategy 10. Both of these strategies have been profitable for the last 6 weeks, especially Strategy 10. Strategy 10 also has the highest return on net drawdown since it has one of the lowest drawdown stats given its profitability. The biggest issue with Strategy 10, however, is that it makes a large number of trades every day so it works best for traders with a flat rate commission plan.
One thing we need to be intellectually honest about is that it’s easy to look the other way when the inconsistency works in your favor. The question is: is the strategy performing better due to a change in market conditions or is it just inconsistent? Last week Strategy 10’s profit factor was 1.44, which is markedly better than its annual profit factor of 1.16. Does this mean we can expect a correction in the strategy’s performance soon or is it just a function of market conditions?
Perhaps our biggest surprise is Strategy 8. Last week the strategy had a profit factor of 2.76, but its annual profit factor is only 1.09. Again, either the strategy is performing better due to a change in the market OR perhaps this is a sign that it’s due for a correction back to its norm of 1.09. Will we see another 4 weeks of profitability, as the current trend suggests or will we see a reversal? Again, only time will tell.
What we do know for certain is that only 5 out of the 36 strategies had an unprofitable week. That means that 27 out of 36 strategies were profitable last week. Put yet another way, 75% of the strategies were profitable or flat. Those are fairly good odds if nothing else. That said, last week may have been a good trading week. It could be that only 20% of the strategies perform well this week as opposed to 75% last week. Going back to the ‘mudder horse’ analogy, perhaps some of our strategies are mudders that only perform well in a certain market. If so, our next step (and this will take some time), is to figure out a way to determine what kind of market a strategy performs best in. We may even be able to gauge market conditions by monitoring the performance of strategies. Once again, the only way to tell is to compare from week-to-week, which is what we’ll be doing.
It is also worth noting that the longest week-to-week profit run is 6 weeks and the longest week-to-week loss is also 6 weeks, so perhaps something in the market changed 6 weeks ago? Six weeks ago was also the beginning of the fall trading session. Traders returned from August vacation and started trading just after Labor Day so there was a significant increase in volume.
While Strategy 10 and 8 are our best performers, they weren’t the most consistent. In terms of consistency, Strategy 3 was our best performer. We will be testing out other ways of measuring consistency, but we welcome and look forward to your feedback here. Meanwhile, which strategy do you think will perform the best/worst next week?
If we were to take a guess based on past results it would be the following:
The strategy that is most consistent with annual results: Strategy 3
The strategy with the highest profit factor of any unoptimized strategy with more than 1 trade: Strategy 8
The strategy with the highest number of profitable weeks: Strategy 3 or 10
Will the trend continue or will there be an upset? We’ll see how it plays out next Monday (11/8) when we update the weekly report. If we find consistent performance from week-to-week, the holy grail of trade strategy may be a moving target that toggles back and forth between several strategies. Perhaps the holy grail of trade strategy is a portfolio of strategies? I’m not a big fan of diversification, especially when you have an 75% chance of selecting a profitable strategy, but if you are, it might mean that you want to trade the two most profitable strategies from week-to-week with the two most unprofitable strategies. We are working on a correlation heat map to help with this as well.
As a final note, one thing we’ve noticed is that strategies with less than 500 trades tend to be less consistent on a week-to-week basis. We already knew this to a certain extent, which is why we added a requirement for at least 253 trades per year to the holy grail definition, but looking at the data from a more granular level seems to reinforce or confirm this finding. Or, perhaps we should be looking at consistency from a daily perspective for strategies with a low trade count. With more clarity comes more questions. We hope to get closer to answering some of these questions as this weekly series develops.
If you have any questions, feel free to contact us directly at firstname.lastname@example.org.