Automated Trading Strategies: FAQs
Answers to Some Commonly Asked Questions
Where can I find ATS strategies for download? Do you have a list?
Click here for a list of links to all strategies. The download(s) for all strategies is located at the bottom of each strategy description.
Where can I find the latest ATS Forward Test portfolio?
Subscribers can find the latest ATS Forward Test at the bottom of the Strategy Description page.
What’s the first thing I should do once I sign-up?
The first thing you need to do is what you are doing — read the FAQs.
The second thing you should do is look through the Strategy Description page. Review the latest ATS Forward Test Portfolio.
The third thing you should do is start your own forward test using live data on a simulated account. You don’t have to copy what I’m doing. That can be overwhelming. I started with just ten strategies and grew from there. Every quarter it gets easier and the first quarter was the hardest, but you have to start. You just have to push yourself to start and the rest will flow out of that action.
How do I start my own forward test?
One thing that differentiates ATS from other newsletters that focus on automated trading strategies is that I’m actually testing ATS strategies using live data. The account is simulated, but the data is live. You can read more about forward test updates in the Mudder Report. Subscribers can also read a step by step tutorial here:
Which strategies are the best?
That’s hard to say. It changes as we learn more about how the strategy performs and get better at documenting performance. The best place to get the most recent update is the Mudder Report.
Are your strategies compatible with NT7? No. NinjaTrader 8 indicators and strategies are NOT necessarily compatible with NinjaTrader Version 7.
What if the strategy isn’t working? Do you have any troubleshooting tips?
If you have any difficulties duplicating performance results, try the following:
Clear the cache. How do you you do this?
Emily, from NTs scripting support team, provided the following instructions:
In order to start with a clean slate of historical data in your database for testing, please try the following (you may revert back to your previous set of historical data after these steps if you'd like):
Shut down NinjaTrader
Open a file explorer window and navigate to the Documents\NinjaTrader 8\db\ directory
Rename or temporarily move the following folders to another location (such as your desktop):
cache, day, minute, tick
Launch NinjaTrader and connect to only your NinjaTrader connection
Run the backtests:
Use the same session template (this is most important): CME US Index Futures ETH in Trading Hours within the Strategy Analyzer, https://ninjatrader.com/support/forum/forum/ninjatrader-7/platform-technical-support/89018-different-results-for-backtests-with-same-parameters
Be sure to use the US dating convention. Dates are written in the month–day–year order (e.g. October 5, 2022).
If the strategy has an embedded data series, it uses more than one data series to make decisions. In this case, you may need to update the strategy to the current front month in order for it to work properly. Here’s a tutorial using NT8:
If you’re still having difficulty, send an email to AutomatedTradingStrategies@protonmail.com with an in-line screenshot of your strategy analyzer results page.
Have you backtested some of the earlier strategies with more recent volatile market data?
Every performance update is a backtest update that includes most strategies. You can compare the current update with the previous update, to understand the impact of volatility.
Why do you only provide 1 year backtests?
It's a fair question.
The short answer is: If I trusted the data more, I would, but I don't.
The long answer is: Knowing that a strategy performed well for the last 5 years as opposed to the last year would certainly make me 'feel' better about trading it live, but just because you run a 5 year backtest doesn't mean you're going to get the answer to that question. One of our primary concerns is backtest risk or the accuracy of the standard backtest. Standard backtests use historical data, which is inherently flawed and I believe long dated backtests can exacerbate the issue. This is because the data tends to deteriorate the further back you go. Even if the strategy performs better in the past than it does in the current year, I wouldn't trust it. Unfortunately, we've been impaled by a few of our own unicorns already, which is to say, we know how inaccurate a backtest can be (If you follow the newsletter, you know what I'm talking about). To avoid this from happening again, we've become specialists at creating strategies that promote backtest accuracy. We work within the limits of the simulation. One of those limits is historical bar calculation; the other is the historical data itself. We can manage the former by switching to minute based data over range, but the latter is somewhat out of our control. I've tried several data providers and the issue is the same -- audits of longer dated backtest results tend to fail. This is not the case with market replay data, but it is difficult to run strategies in market replay for 6 months, let alone 5 years.
Finally, switching from reasons based on practical application to personal theory, I tend to value the most recent performance of a strategy over its performance two years ago. It's more relevant. I'm also more interested in the degree to which the profit factor changes when we update our performance charts every two months than I am in what the profit factor was 5 years ago. This is why we update the performance of strategies every two months.
Do you have any strategies that will pass funded trader programs?
Click here for a post on this topic:
If you’re looking to pass a funded trader program, this is not the place for you (read: do not purchase a subscription if that’s what you’re looking for). Automated trading strategies require more latitude than more funded programs are willing to allow for. The additional restrictions make an already difficult task that much more difficult. My advice to you is to learn how to trade manually.
Do your strategies work on cryptocurrencies?
In general, one of the most common questions I get pertains to the use of ATS strategies on other assets like crypto. You can read more about that in the post: Do ATS strategies work on cryptocurrencies? At the time, both Strategies 24 and 26 showed promise. I hope to revisit this at some point. Currently, I am testing BTC futures on several strategies in the ATS Forward Test.
What is overfitting and what are you doing about it?
Another commonly asked question is in regards to overfitting. You can read the following posts to learn more about overfitting and what we’ve done to reduce its impact: Overfitting: What is it and what can we do about it and What Are We Doing To Ensure Backtest Accuracy? A few months after writing these posts I received a question regarding strategy strength and what optimization tools we’re using to increase robustness. I answered that question in the post: How Can You Tell If An Automated Trading Strategy Will Perform Well Over Time?
The truth is, however, that the simulation is limited. Until we reach escape velocity, the simulation will always be flawed in some way. Not only are there issues with the simulation, but the data is often flawed, especially with regard to contract expiry. The easiest way to deal with a rock in the road is to go around it. For those of us that like analysis, it can be tempting, like a siren song, to come up with ways of dealing with the rock, but the best way to resolve overfitting issues is to run strategies on a simulated account using live data.
What’s the difference between AI Strategies and ‘normal’ strategies?
They are both ATS strategies, but one was created with the help of generative AI. You will notice that there are two designations of strategies on the performance chart AI Strategies (AI Strategy 4, AI Strategy 9, AI Strategy 21) and normal strategies (Strategy 3, Strategy 4, Strategy 80). The difference between an AI Strategy and a ‘normal’ Strategy is that the AI Strategy is created with the help of generative AI, whereas the ‘normal’ strategy was created by ATS. So AI Strategy 6 is not the same as Strategy 6. You can read more about AI Generated strategies here:
Why are your strategies too simple or complex?
Some subscribers have said that ATS strategies are too complex, while others have said that they lack complexity. To the former I say: I am here to help. Please reach out to me with your questions. To the latter I say: the aim is not complexity, it is a certain level of consistent performance. That’s what updates are all about. We’re all trying to figure out what works and what doesn’t over time. If you have any questions about a strategy, please reach out. It might take a few days, but I try to respond to every email/comment.
Why are the strategies you give away poor performers?
One common critique is that the strategies we give away (Strategies 1 and 5 ) don’t perform well. That’s why we’re giving them away. The goal is to show you how the process works, not give away the best strategies.
Are the markets rigged?
A few months ago I received a question about rigged markets. In particular, he was worried about something referred to as front-running. The truth is, what most people think of as ‘rigged’, traders have been working with for years. There is little room for regulation in free-markets; that’s the rub. You can’t have free-markets without the “free” part. There are no nets, nor should there be. Unfair play is fair play. The best ‘play’ for the retail trader is to identify and use the traps that sharks and whales set. Become a bottom-feeder. Look for a longer post on this shortly.
Remember, micros and e-mini futures are traded through a regulated exchange. Prices are market-driven AND transparent.
To read more about this, check out this post:
Why don’t you use payoff ratio instead of profit factor?
We use profit factor to measure the strength of ATS strategies. Profit factor is calculated:
Gross profit / Gross loss = Profit Factor
Payoff ratio is calculated:
Average winner / Average loser = Payoff Ratio
We prefer profit factor because unlike the average win/loss ratio, also known as the payoff ratio, it tells us immediately if the strategy was profitable or not. The payoff ratio tells you about the average trade, while the profit factor tells you about the total strategy. In order to understand if the strategy was profitable in aggregate, you would need to multiply the payoff ratio by the % of profitable trades or the win rate. The ideal strategy is one that offers a high win rate and a high payoff ratio.
For the sake of the hunt, let’s create two scenarios to reinforce the difference between profit factor and payoff ratio:
Scenario #1: If you have a strategy with a payoff ratio of .92 and a win rate of 63%, it means on average your win/loss ratio is .92 so you’re losing money on every trade, but you win more than you lose, so you’re making a profit.
Scenario #2: If you have a strategy with a payoff ratio of 11.36 and a win rate of 12%, it means on average you make 11x more than you lose on each trade, but you only win 12% of your trades.
The holy grail of trade strategy might have a payoff ratio of 11.36 and a win rate of 63%. We have yet to find the holy grail of trade strategy, but the first scenario profiled above belongs to Strategy 40 and the second scenario belongs to Strategy 41.
Why did you switch to NT8?
Another question I get, is about the switch to NT8. As a primer, it takes a lot of data to mimic all the movements of the market, so backtest software creates methodologies that try to mimic the market without using all the data required to do so. The result: holes in your data. These holes often lead to inaccurate backtest results, especially for bar charts that require more robust calculations. I witnessed these holes first hand when creating backtests on Renko charts in NT7. You can read more about that sad event here.
Ninjatrader was very aware that backtest accuracy was an issue with their software (and with backtest software in general) so they made several changes to the backtest methodology in NT8. Originally I was going to move to NT8 at the beginning of 2022, but the discrepancy between backtest results and market replay data created an urgency. There’s no point in hunting for the holy grail of trade strategy if the maps are leading us down the wrong path.
Unlike NT7, NT8 provides two options to control the granularity of historical order fill processing: Standard and High.
The Standard order fill resolution (the fastest) breaks each historical bar into three virtual bars to mimic the movement of price within each bar's time-frame, which provides more realistic historical data compared to traditional backtesting calculation methods (used in NT7) that only use the open/high/low/close (OHLC) prices of the bar.
The three virtual bars are created based on the following logic:
1. Open price to the High price
2. High price to the Low price
3. Low price to the Close price
Here’s a visual example provided by Ninjatrader:
What happens if the open price of the bar is closer to the low price than the high price?
1. Open price to the Low price
2. Low price to the High price
3. High price to the Close price
What I just reviewed is the Standard or “faster” backtest methodology in NT8. NT8 also allows for a High order fill resolution, which allows us to enter a secondary bar series to be used as the price data to fill orders. This provides us with more granular data (such as 1 tick or 1 second) so that the strategy or indicator contains the individual ticks/seconds in the backtest/historical data. Ultimately, NT8 provides more accurate backtest results than NT7 at both the Standard and High Order Fill resolution level, so either way you’re better off.
How do I learn how to create automated strategies for myself?
My advice is to use the resources provided by Ninjatrader To learn more about their daily training programs, click here. See below for an overview of the kind of training they have and it’s all free. I am not affiliated with Ninjatrader in any way.
What is the difference between Total Investment & Reinvestment?
Read the following post for an answer:
Which strategy do you suggest for a $10,000 investment?
I don’t suggest any one strategy, but if I had to design a process for selection I would say to:
Pick five or more strategies based on profit factor. Use ATS strategies as a pool of strategies to choose from or create your own.
Run each strategy continuously via live simulation on a virtual server. Do this for at least three months. Your starting account balance for each simulated live test should be the max drawdown of the strategy with a buffer. Remember, max drawdown is the most cumulative net profit has ever dropped from the strategy’s high, not from $0.
Study performance from week to week.
Develop a strategy to ‘run the strategy’. Running a strategy is more than just enabling it to run. What happens if you get kicked out of a strategy? What’s your rollover strategy? What happens if the virtual server shuts down? What happens if the strategy falls out of sync? Will you enter the trade immediately or will you wait until the strategy is flat? Do you want a strategy that closes at the end of the session? If not, you need to add the cost of initial margin to your account (to learn more about futures margin click here). My point is that automation comes with its own set of questions.
Compare live performance against backtest results? If it's the same, you may want to try that strategy live on a micro contract, but you still need to be prepared to lose your entire account.
Do your strategies work with Renko?
Yes and no, but mostly no. This is one of the main reasons I made the switch to NT8. Subscribers can click here for the full answer.
What is slippage and how are you accounting for it?
Slippage is when you receive a different price than you wanted on the trade execution. It can also manifest as a missed or skipped trade altogether. This can have a positive or negative impact on net income. One of the main ways to avoid slippage is to make use of limit orders instead of market orders. This is because a limit order should only be filled at your desired price. Also keep in mind that contracts with more volume/contracts traded at each tick tend to have less slippage, especially in highly volatile markets.
Is there a way to discern a choppy, high volume session from a normal, high volume session?
Yes. The main sign for me is longer tails. The longer the tail, the choppier the water. The question is, how can we take advantage of this; that is, is there a strategy that does well in a choppy session?
I reached out to a friend that surfs for help with this. This may sound odd, but the market moves in waves and I’ve found that surfers have a natural proclivity for trading. He said the best thing to do in choppy waves is referred to as “tacking”. He described it as the same movement you use when moving a skateboard - back and fourth movements. Incidentally, this is the same thing you do when you’re on a sunfish boat and you’re sailing against the wind. This translates into short, small trades used with limit orders, stops and take profits. Think of yourself tacking back and forth from one end of the tail to the other. So instead of using a strategy that’s looking for long waves or big price movements, you’ll need a strategy that uses limit orders to get in, longer stops and shorter take profits. The hard part is creating a program that identifies a choppy session.
Can you teach me how to trade?
No, but before you do anything else, read this: Trading Is A Dark And Scary Forest. This tells you a little about my personal journey and what you can expect as a trader. Then you may want to read this: We Are Treasure Hunters Searching For The Holy Grail Of Automated Trade Strategy to find out about my goal with ATS. And finally, if you’re new to trading, I highly recommend simulated trading for at least a year. Conduct your own forward test with ATS strategies or yours.
I also want to suggest doing a search for the company TopStep online. Review all of their videos on Youtube. Use their Youtube channel as a place to learn. You don’t need to buy anything. Just watch the morning call, review all of their videos and sign up for their blog. I believe they also have a free group class on Wednesdays given by Hoag — don’t take any classes unless they’re with Hoag or Eddie.
Treat your search for a teacher with respect. You are looking for someone that can teach you how to learn more from losing than winning. Expect some hazing. This could be one of the most important decisions of your life if you do it right.
What’s your favorite platform?
NinjaTrader, but I’m open.
What can market physics teach us about trading strategy?
The market is based on auction mechanics. This is why prices aren’t random. To read more about this subject, read the following post:
Which type of futures contracts should I use?
This is up to you. I am struggling with this question too.
For a post on what was found when comparing NQ to RTY read the following:
Is this a good time to start investing due to the market volatility?
It is up to you. I could make an argument either way. It's like telling someone that wants to learn how to surf that they should wait until after the storm. Meanwhile, the best waves come in the middle of a storm.
I will continue to update and reference this post. If you don’t see your question here and haven’t received an answer to your question, please respond to this post or email directly at AutomatedTradingStrategies@protonmail.com.
I'm sorry if I missed something obvious, but I cannot see anywhere I've looked -- including the source code of your strat #1 -- are your strategies applied to a single security, or a pool? Could you please either point me to the description of how the strategies are applied to the asset(s)?
Secondly, while back-testing is a reasonable tool, what I would LOVE to see is how the strategies worked when applied to a validation or -- better -- a test period. While I am relatively new to the data science world, I have come to understand that performance degrades from training to validation to test -- depending on the strength of an algorithm.
Lastly, I think it would be appropriate to report on percentage of gains vs absolute dollar amounts. I don't know what the basis is; so how can I assess the net gain performance?
Thanks in advance.
An article referenced under the question, "What is overfitting and what are you doing about it?," is locked behind a paywall. Does this article, "How Can You Tell If An Automated Trading Strategy Will Perform Well Over Time?," explain how Ninjatrader's Walk Forward optimization method is leveraged in ATS development?
Assuming some type of walk forward analysis is utilized, why compare the backtest results of strategies instead of the combined OOS walk forward series for each strategy?