Hidden States FOUND (Part 3)
This HMM approach increased portfolio performance by 750% on out-of-sample-data. The Incubator went from losing $901K to making $5.8M.
Important: There is no guarantee that ATS strategies will have the same performance in the future. I use backtests and forward tests to compare historical strategy performance. Backtests are based on historical data, not real-time data so the results shared are hypothetical, not real. Forward tests are based on live data, however, they use a simulated account. Any success I have with live trading is atypical. Trading futures is extremely risky. You should only use risk capital to fund live futures accounts and if you do trade live, be prepared to lose your entire account. There are no guarantees that any performance you see here will continue in the future. I recommend using ATS strategies in simulated trading until you/we find the holy grail of trading strategy. This is strictly for learning purposes.
We’re not just developing strategies—we’re on a quest for the holy grail of automated trading. Questions? Check the FAQs or feel free to reach out directly: AutomatedTradingStrategies@protonmail.com.
There’s a pivot point in every successful journey where momentum fundamentally shifts. It’s the moment the hero stops running and starts chasing. It’s when Westley and Buttercup emerge from the Fire Swamp. It’s the instant Neo decides to rescue Morpheus, or when Jean Valjean returns to save Javert from the barricade. It’s when Atticus Finch stands guard at the jailhouse door or when Odysseus stops hiding his identity and strings his bow. It is the transition from a retrospective view of the challenges behind you to a prospective vision of the victory ahead.
Needless to say I am inspired. I’ve had an epiphany for our little experiment guys and it’s big. It’s bigger than the day I started forward testing. It’s bigger than the day I started tracking individual strategy performance (by variation). It’s bigger than the day I realized the holy grail was more process than strategy. And now that process, having gone through many iterations, is focused on finding the best strategy to use in the current moment. If you can do this at the portfolio level, well that IS the holy grail.
Currently, I select strategies based on a hybrid FIRE approach, which is retroactive. You can read more about the FIRE (Filter, Isolate, Refine, Expand) method here, but it’s essentially a method that uses ML to create filters around certain conditional elements. It’s good to know, but rarely persistent enough to be reliable—like a map to a store that is constantly changing location. It’s good to know that the store is never open on Friday, but it would be even better to have a schedule or perhaps an alert system that tells you when the store is in your favorite location. You’ll also want to know:
How long the store tends to stay at that location (state persistence)?
If there’s any kind of order or sequence to the change in location (state sequence)?
I’m looking for an algorithm that finds the most profitable hidden, persistent locations. It takes all the trade data I have and not only tells me which strategies are best to trade, but when and how much.
The Question That Started Everything
In Part 1, I introduced Hidden Markov Models (HMM)—the mathematical framework behind Renaissance Technologies’ $66 billion success. In Part 2, we explored Chase Hughes’ behavioral psychology and how neurochemical states drive market participants.
Now comes the follow-up question: What is THE BEST way to use HMM and behavioral psychology in portfolio management?
The final answer was a surprise that came after falling into a deep rabbit hole. Questions surfaced like:
What’s the difference between a state and a condition?
What’s the difference between an observable and hidden state, and which is more persistent?
Do we use HMM on price, or is it better to use HMM on actual trade data?
I won’t bore you with the rabbit hole details, but the end result was a new approach for strategy improvement and portfolio optimization. I’m talking about an approach that took the Incubator portfolio, a portfolio that lost $900K, and turn it into a +750% improvement (+$6.7M swing).
Enough already. Let’s get into it…