The Mudder Report: Q3 Forward Test Update
What a week!
Important: There is no guarantee that ATS strategies will have the same performance in the future. I use backtests and forward tests to compare historical strategy performance. Backtests are based on historical data, not real-time data so the results shared are hypothetical, not real. Forward tests are based on live data, however, they use a simulated account. Any success I have with live trading is untypical. Trading futures is extremely risky. You should only use risk capital to fund live futures accounts and if you do trade live, be prepared to lose your entire account. There are no guarantees that any performance you see here will continue in the future (good or bad)—that’s what makes the hunt for the holy grail so difficult. This is why the best way to trade is with a simulated account on live data. I recommend using ATS strategies in simulated trading until you/we find the holy grail of trade strategy.
Housekeeping:
My apologies for getting this out later than expected. I’ve been traveling and there’s bad weather in the area.
Also, a few of you have asked me if I’m trading any strategies live. I understand the confusion. Hopefully, I’ll have an opportunity to create a timeline in the fall. As a quick review, I used to only publish backtests. Backtests are third fiddle to forward tests, and forward tests are second fiddle to live tests.
Backtests are run on historical data using a simulation. This allows for errors in the historical data file as well as in the way the data file is used (OHLC) to simulate bars. Bars that require more than these four data points are often wrong. Bars that take longer to form (60 minute) are more reliable than bars that form quickly (1 minute). In other words, you can’t rely on backtests: garbage in-garbage out.
Forward tests use live data on a simulated account. So the data is real, not historical. The chart is also live, so the simulation can use more than four data points to create each bar. As a result, I can confidently say that the performance data received from forward tests is better for decision making. That doesn’t mean that the strategy is going to behave the same in the future. It merely gives us a better understanding for how an animal moves.
Knowing what I know now, I would never run a strategy live that I hadn’t forward tested for at least a few months first.
Live Tests are the best, but they are also the riskiest. Unlike the ATS Forward Test, the ATS Live Test was conducted on a live account, not a simulated account. ATS ran our first live test in January of 2024. It ended in June up 196%. It was terrifying at first, but then the process of “letting go” got easier. The experience provided many insights that you can read about in Live Test updates. Needless to say, my goal for the next four months is to find the best strategy or strategies to run in the next Live Test.
I hope this helps those of you that may be new to automated trading and/or ATS. Please let me know if you have any additional questions and click here for a list of FAQs.
Let’s get into it…
What a week guys! After last Monday’s panic selling, the market nearly wiped out its losses for the week. The performance played out in the ATS Forward Test portfolio as well— net profit came in flat to positive at $1,428 on 671 trades.
It all started two weeks ago, actually. After the Fed announced no change, traders that had baked in easing threw a tantrum and markets panicked. If nothing else, we learned that the market is extremely sensitive (and far-reaching) these days. From investors and business owners crying over no-rate cuts to an unwinding of the yen carry trade, the markets are finding it increasingly hard to settle on price. In other words, there is a great deal of uncertainty in the air. Looking at it from a trading perspective, this is part of a pattern: a cooling-off period before the next run-up. Don’t forget, GDP is still at 2.8%.
There are those that say volatility is the price of return, put another way, risk is the price of profit, but I’m convinced that does not have to be the case. After running a live test from January to June of this year using ATS strategies, experiencing all the ups and downs, and the emotions that come along with those ups and down, I’ve changed my fitness test a bit. Instead of a focus on one factor, ie, net profit or profit factor, my fitness test for strategy viability is a combination of three things: max adverse excursion (MAE), profit factor and trade count. Net profit, is still the primary filter, but these are the metrics I use to prioritize. Prioritization is critical when trading multiple strategies, especially if they are highly correlated.
Two weeks ago we looked at:
which strategy variations performed well
which instruments performed well
how well AI-generated strategies performed ag. non-AI generated strategies
This week, I’m going to provide you with three different lists:
a list prioritized by net profit
a list prioritized by profit factor
a list prioritized by MAE
Depending on your goals and account type/size, each of these lists can help to align and prioritize your portfolio with your goals.
We know that almost all strategies in the portfolio perform well when the market goes up. Some do better than others. Likewise, some do better than others when the market goes down. With such a volatile and crazy market, the question is: which strategies were able to do both?
Well, out of the 140+ strategy variations in the Q3 Forward Test, these are the high performers. It’s important to note that the strategies in the Q3 Forward Test were chosen based on their overall performance in the Q1 and Q2 portfolio, which is to say, their performance has already been confirmed by forward tests on some level. We’re getting close…
Top Strategies By Net Profit
Here’s a chart of the top 20 strategies in the Q3 Forward Test so far by net profit…