Automated Trading Strategy 75: The Genetic Algorithm
Part 2
Important: There is no guarantee that these strategies will have the same performance in the future. I use backtests to compare historical strategy performance. Backtests are based on historical data, not real-time data so the results shared are hypothetical, not real. There are no guarantees that this performance will continue in the future. Trading futures is extremely risky. If you trade futures live, be prepared to lose your entire account. I recommend using these strategies in simulated trading until you/we find the holy grail of trade strategy.
Our goal is to find the holy grail of automated trade strategy. We haven’t found the holy grail yet, but we get closer with every strategy. Click here for the most recent performance chart and links to all strategies. For answers to some FAQs, click here.
This is Part 2 in a two part post for Automated Trading Strategy 75. Click here to read Part 1. In Part I we revisited Ninjatrader’s Generative AI model (AG). In Part 2, we’re going to look at the Genetic Optimization (GO) model. Both are powered by a genetic algorithm.
In Part I, I gave you 9 more strategies created by AG. We also looked at the code behind the best performing AI Generated Strategy in the forward test.
In Part II, I’m going to give you a framework for how to achieve the best results with the GO. We’ll also look at the results of 27 tests and the impact changes in GO properties have on strategy performance.
Let’s get started.
"… if you gaze for long into an abyss, the abyss gazes also into you."
- Nietzsche
(Beyond Good and Evil: Prelude to a Philosophy of the Future (1886), Chapter IV. Apophthegms and Interludes, §146).
I love birds, especially a certain subgroup within the birds of paradise. Described as “truly bizarre”, this is a group that has distinguished itself. From an incredibly talented Black Sicklebill, to the painstaking perfectionism of a certain Western Peroquia, these birds are absolutely obsessed and they have become famous for it. Studied both by humans and other birds, these birds are obsessed with flair, pomp, and the manipulation of one’s environment to attract a mate.
This is my favorite bird. He is a Western Peroquia. Other male birds try to emulate him, but can’t. He has become a legend.
As interesting as Darwin’s theory of evolution is, it is limited (by his own admission). No doubt, in a world of lack, the finch that develops a longer beak will survive, but what about the bird that lives in a world of abundance? In order to procreate, this bird must obsess over the mating dance. It could be explained in many ways, but ultimately, I believe it is the ability to obsess that gives these birds an evolutionary advantage.
I suppose we aren’t that different from birds. Pythagoras wasn’t just a philosopher and math genius, he had OCD and an irrational fear of fava beans. Napoleon’s obsession with licorice turned his teeth black. Christian VII of Denmark’s obsession with his own penis is said to be the reason he wasn’t allowed to wield any real power.
Good or bad, we all tend to develop these obsessions over one thing or another. I believe we have our ancestors to blame. And when I say ancestors, I refer to a group of hominids that found their way to hold the title of modern day Homo Sapien. From the Neanderthals of Wales, to the Denisovans of Siberia, to the hobbits on the Island of Flores, and the giants (X lineage) of West Virginia, all had unique and distinct adaptations to the environment, but they are no longer here. All people living today belong to the species Homo Sapiens.
Why is that?
One could say “coincidence”. Others believe in intelligent design or some hybrid thereof. Idiots like myself may even try to develop a theory about the evolutionary processes behind such a large scale extinction.
Here’s my theory: we are the descendants of the most obsessed, and the most compulsive. According to twin studies, about 45%–65% of OCD symptoms can be attributed to genetic factors. While it has been proven that OCD is not associated with a higher IQ, a myth popularized by Sigmund Freud, there’s a reason the myth exists in the first place.
We come from hominids that obsessed over land, resources, revenge, power and physical differences. It is in our shared mythology as the villain always becomes obsessed with the hero and vice versa. So if two hominids lived on a mountain 300,000 years ago, chances are it was the Homo Sapien that stayed up at night plotting ways to take the land away from the weird looking guy on the other side of the mountain.
Modern day Homo Sapiens are equally obsessed. I believe we beat out all other hominids because of our ability to ‘obsess’, and I wonder if the transhumans of the future will suffer the same fate to some form of obsessed artificial intelligence that views modern day Homo Sapiens as the weirdos on the other side of the mountain, but where did this “obsessive” trait come from?
It could have always been there—dormant— just waiting to be activated. It could be a slight mutation that literally changed the shape of DNA. Perhaps this mutation unleashed the first obsessive/compulsive gene in modern day Homo Sapiens and the rest is history. Who can beat the man or woman that is obsessed with a thing?
Why do I bring any of this up? Ninjatrader has created two different tools to help with our obsession for finding “the holy grail of automated trading strategies” and they both operate off of a genetic algorithm. I’m hoping that by connecting and extending the genetic metaphor to our own evolutionary superiority, it might help to make its properties more tangible and easier to digest.
I’m also hoping that we can help Ninjatrader’s AI Generate to evolve by giving it the ability to add mutation and crossover properties to its capabilities. These properties are already available in the Genetic Optimization (GO) so it shouldn’t be too hard to add these properties to the generative tool (AG) as well. I’ve already asked Ninjatrader to do this, but would appreciate your help in making this a reality. You can add your vote by making a comment here.
Now, let’s discuss the genetic algorithm.
What Is A Genetic Algorithm
Mutation is the engine of evolution in that it generates the genetic variation on which the evolutionary process depends. To understand the evolutionary process we must therefore characterize the rates and patterns of mutation.
Nature is an amazing model so I love that Ninjatrader used a genetic algorithm to fuel both the generative AI tool “AG” as well as the genetic optimizer “GO”, but the tools can be difficult to learn. I hope this post gives you the ability to leapfrog over some of this confusion. Let’s start by learning more about the genetic algorithm.
To learn more about what a genetic algorithm is, I recommend reading the paper: Using Genetic Algorithms to Find Technical Trading Rules. The paper was published some time ago, but it provides a good foundation.
Before getting into each property, let’s talk a bit more about what the genetic algorithm is supposed to do. Here’s an excerpt from Ninjatrader:
The general idea of how the GA solves an optimization problem is analogous to the concept of how evolution via natural selection adapts a species to the environment. In biology, only the strongest individuals will be able to reproduce and pass on their superior genes to the next generation. Assuming each generation can only pass on the strongest genes, after several iterations we would be left with the optimal attributes for the environment. Through this same mechanism, the GA will test a random preset of your parameters. Through multiple generations of testing, the parameters will zero in on an optimum solution.
This is a basic use of the “survival of the fittest” mentality when applied to generative processes used by AI Generate, but it lacks mutation, which could be the literal ‘spark’ behind modern day Homo Sapien’s evolutionary superiority.
Crossover rate is the property that allows the GO to “create” mutations, a property that does not exist in AI Generate. Here’s Ninjatrader’s description:
Crossover is the process in generating offspring that are not 100% identical to their parents. It is done by taking half of the parameter settings from parent A and mixing it with the other half from parent B. Crossover allows GA to test different combinations of parameters and hone in on the optimal solution. Crossover alone however will eventually yield identical offsprings in the population through several generations and so through mutation, some random parameter settings will be interjected in a few of the offsprings to allow for an adaptive quality to the algorithm.
In order to use the GO you also need to understand the Properties , which are shown below:
This is a fascinating model, but there are so many inputs that it’s a bit overwhelming. Let’s start by understanding how to make sense of each input.
In the picture above, the generation size is 25, which means each generation has 25 children. There are also 5 generations. The generation size is the number of combinations to test in each generation.
The convergence threshold refers to the number of duplicate children in one generation, so a threshold of 20 means that if 20 of the 25 children are duplicates that have already been tested, then the optimization will be terminated. This impacts model efficiency more so than function.
There are two ways that a generation of offspring can be created: randomly and