The Last Mudder Report of 2023: ATS Forward Test (Simulated Live) Update
Forward Tests Are King
Important: There is no guarantee that these strategies will have the same performance in the future. I use backtests to compare historical strategy performance. Backtests are based on historical data, not real-time data so the results shared are hypothetical, not real. There are no guarantees that this performance will continue in the future. Trading futures is extremely risky. If you trade futures live, be prepared to lose your entire account. I recommend using these strategies in simulated trading until you/we find the holy grail of trade strategy.
We haven’t found the holy grail of automated trade strategy yet, but we get closer with every strategy. Click here for links to all strategy descriptions and the most recent performance chart from the backtest.
Forward Tests Are King
If I were to rank the reliability of testing methods on the simulation it would be:
Live Forward Test (live data on a live account)
Simulated Forward Test (live data on a simulated account)
Market replay Simulation (backtested on historical data)
High-order resolution Simulation (backtested on historical data)
Standard Simulation (backtested on historical data)
For more detail on each of these tests, please see this previous Mudder Report.
In general, the more reliable the test, the more time it takes to run. In other words, accuracy takes time. So there’s no way to get around it—the best way to test strategies before going live is with a forward test, using live data on a simulated account. I am fairly certain that one of the main reasons people lose so much money on the hunt is because they skip this critical step. Here’s a post I recently published on how to do it:
ATS was started three years ago, but the ATS forward test didn’t start until a year ago. I regret that. It is the single most important tool in our arsenal. A great way to learn from my error is to start your own forward test today. We started with only a few strategies and then continued to add more throughout the year. Today, I’m going to show you the full year ATS forward test.
Once we get the forward test results, then what? Well, we can see if the strategies really are profitable or if the backtest was faulty in some way. For example:
historical data may be corrupted
the optimization model may be flawed or over-fitted
bar formation frequency may be wrong or highly volatile
For the first time, we’ll be able to compare the one year forward test performance against the backtest performance from 1) the original backtest period, and 2) the forward test period. I think you’ll find the results very interesting, especially since the strategy I’m going to use in the example is the best performing strategy of the year in the forward test.
Now that we have 1 year of forward test data, it’s time to take the test to the next level. It’s time for a live test! A real live test. Not a test on a simulated account or a funded account, but a test on a real live account. Even if it fails, I think we’ll learn a lot from the process.
The live test will begin on January 14, 2024—the day of our 3 year anniversary.
What strategies are we going to use?
I don’t know yet, but it will be based on the results of the forward test. There will also be a six month embargo on the actual strategies being used (and why) as a control, but I will share performance information with all paid subscribers on a weekly basis. ATS Research will also help to validate.
Before showing you the best performing strategies in the ATS 2023 Forward Test, I’d like to ask you a question:
I’ll be posting the Q4, 2023 Performance Report shortly after the new year. To read the Q3 Performance Report, click here. We currently update the report every quarter (every 3 months), but I’m questioning the value of the process so I could really use your input.
Here’s a second question I could use your help with:
Thank you for your input.
In addition to reviewing the forward test results, I’m also going to give you an update on the election day strategy: Strategy 73, and Strategy X.
Now let’s get into the forward test results.
The Forward Test Is Evolving
These are the results of the forward test. The trades are being made on a simulated account, so the results are still hypothetical, but the data driving the simulation is live. Note, this does not cover all weeks of the year. December 15 was the last day of the 50th week of 2023, and as of today we’ve captured 43 of those 50 weeks in the forward test. Also note that strategy variations vary by instrument, data series, parameter setting and contract size. The chart below provides the: 1) strategy, 2) net profit, 3) start date (not all strategies were started at the same time); and, 4) trade count for all strategies in the forward test. I’ll also provide a quick overview of which instruments performed the best and which instruments—across the board—we’re going to stay away from in 2024.
The purpose of the Mudder Report 3.0 was to track real-time performance, not to maximize performance. In 2024, however, I will be tracking six additional forward tests aimed at getting a better understanding for how to maximize performance. Portfolio 1 will be the standard master forward test that we’ve been tracking for the last year, however, Portfolio’s 2 - 7 will focus on a specific category or type of strategy as follows:
Portfolio 1: the standard master forward test based on backtest results from Ninjatrader’s historical data server
Portfolio 2: standard forward test based on backtest results from Tick Data, LLC’s historical data (cleaned)
Portfolio 3: top performing strategies (from 2023 forward test)
Portfolio 4: worst performing strategies after being flipped (from 2023 forward test)
Portfolio 5: AI Generated strategies
Portfolio 6: HWR strategies or scalping strategies
Portfolio 7: Best performing strategy from each instrument (from 2023 forward test)
I will share the performance of these portfolios with you in the Mudder Report.