The critical test in applying machine-based learning and other forms of artificial intelligence to trading is whether it can generate alpha, or returns that exceed those that could be attributed solely to market forces.

The evidence is yes, but with a caveat. As with any investment strategy, market impact must be taken into account.

“My experience, both at my current company and through my consulting and teaching, we find cases where it produces alpha,” said Tucker Balch, professor of computer science at Georgia Institute of Technology and founder of Lucena Research, a machine learning-based investment technology company.

“People who are generating alpha aren’t advertising it. It’s often the case that strategies are capacity limited, i.e., if someone else replicated it, that would reduce the capability to make money with that approach.”

Prospecting for Gold
It’s analogous to prospecting for gold or diamonds. The challenge is not only in finding them, but in getting them out of the ground.

Often, the cost of extraction will exceed the value of the find, thereby negating profits.

“In my class, I have students using machine learning to make price forecasts, and we often discover approaches that correctly make future price predictions,” said Balch. “The challenge is they don’t always overcome the cost of trading.”

Trading system providers are hard at work applying the techniques of machine-based learning, as well as other forms of AI, such as neural networks, to making investment decisions.

“Much of the research around AI has been aimed at real-world applications, including developing trading strategies,” said Bruce Bland, head of algorithmic research at trading technology firm Fidessa. “This would enable you to input a set of responses to when a particular data event occurs, and generate trades.”

Can one in effect use artificial intelligence to predict future price movements?

“The obvious statement is you can’t predict the future, no matter how clever the algorithm,” said Bland. “That said, you can use machine learning to give a better idea of what might happen, if not an exact prediction, which can then be used to produce algorithms or other decision logic.”

Within its research lab, Fidessa employs “sophisticated clustering technology to look for patterns in data”, Bland said. “Data in finance has been exploding, and making sense of that is part of the problem,” he added.

Lucena Research provides quantitative analysis and statistical machine learning technology to hedge funds, wealth advisers and advanced individual investors.

Its cloud-based artificial intelligence decision support technology enables short-term investors and traders to find market opportunities and to reduce risk in their portfolio using technical and fundamental quantitative pattern matching.

“Algorithms have predictive value, but the market impact from execution dilutes it,” said Balch at Lucena Research. “I’ve run across many cases where we were able to predict future prices, and while the direction and extent of the move can be correct, the change is not significant enough to turn into a return.”

For example, Balch’s Georgia Tech students discovered a strategy “such that when a certain event occurred, the price would change by 12 basis points”, said Balch. “That prediction was repeatable and correct,” he said. “The problem is that in order to execute it, it costs five basis points to enter the position and another five to exit.”

Filtering Out the Noise
What AI techniques do “that traditional financial engineering cannot do is to make sense of the noise”, said Tim Grant, head of marketing and sales at Benchmark Solutions, a provider of streaming market data for fixed income.

Obtaining as many data sets as possible always helps to generate a better solution.

“Let’s make the simplifying assumption that alpha can be generated by two primary routes,” he said. “The first is by having a timelier or more accurate understanding of where value in assets lie now. The second is having a timelier or more accurate predictor of where asset values may lie in the future. Machine-learning technology, such as Bayesian statistical inference, can be applied to solving both of these problems.”

Commercial applications of AI-based trading software are already on the market.

For example, NeuroDimension, a Florida-based software development company, has embedded neural network technology into its software. A neural network is able to capture and represent complex input/output relationships.

“We’ve developed a number of trading models based on neural networks that not only performed well in the past, but have continued to perform well since the models were created,” said Gary Lynn, chief executive of NeuroDimension.

The motivation for the development of neural network technology stemmed from the desire to develop an artificial system that could perform “intelligent” tasks similar to those performed by the human brain.

Neural networks resemble the human brain in two ways: a neural network acquires knowledge through learning, and a neural network’s knowledge is stored within inter-neuron connection strengths known as synaptic weights.

On the question of whether AI techniques can be used to generate alpha, Vadim Mazalov, research and development specialist at trading systems provider Cyborg Trading Systems and a PhD Student in computer science specializing in machine learning at Western University in London, Ontario, said: “Certainly, but it depends on the selected indicators, the classifier and comprehensive backtesting across various combinations of the parameters, possibly combined with optimization algorithms.”

Before selecting the machine learning paradigm to be used as the backbone, “one needs to have a clear picture of the problem that needs to be solved and take into account important requirements, e.g. efficiency, flexibility, scalability, parallelizability,” he said. “In addition, most of the trading algorithms should allow incremental learning, i.e., to be adaptive to new market patterns.”