It’s tough to make predictions, especially about the future, said baseball legend Yogi Berra.

But that doesn’t stop people from trying, particularly within financial markets, where machine learning trading algorithms are being developed and launched by hedge funds, with a view towards finding practical applications of the large body of theory that exists for artificial intelligence.

Perhaps, not surprisingly, many of those at the leading edge hold advanced degrees in mathematics or computer science. While having a PhD isn’t mandatory, it clearly is an advantage.

“When I learned about machine learning, it occurred to me that it could be useful in financial applications,” said Spencer Greenberg, co-founder of Rebellion Research, a New York-based hedge fund. Greenberg is currently pursuing a doctorate at New York University’s Courant Institute of Mathematical Sciences.

“When we try to make money in stock market, we have no fully formed notions of whether to buy and sell, value, momentum, relative value, etc.,” said Greenberg. “Maybe machine learning can extract investment styles in an automated fashion, and an algorithm can be created to learn that process. I got fascinated with it.”

Rebellion Research employs a machine learning-based system to make predictions about the performance of stocks and other asset classes.

The basic premise is that machines can be programmed by Google to conduct web searches or by Amazon and Netflix to recommend movies and books, so there’s no reason why they shouldn’t be able to be trained to make investment decisions.

“One reason people are skeptical about using artificial intelligence for investing is they think of investing as something that’s too difficult for a human to solve, and therefore too difficult to program,” said Greenberg. “There are lots of theories about how the market works. Our approach is to have machine learning algorithms analyze investing in an automated way.”

The field of knowledge in the area is expanding at a rapid clip.

“For decades, machine-based artificial intelligence techniques have been the core elements of algorithmic trading and computational finance in general,” said Vadim Mazalov, research and development specialist at trading systems provider Cyborg Trading Systems, and a PhD student in computer science specializing in machine learning at Western University in London, Ontario.

Machine Learning
The extensive body of knowledge in the art already contains a variety of models that can be applied on different levels and scales—from high-frequency to systematic trading.

“Over the last five years, we’ve seen enormous advances in automated trading technology,” said Alfred Eskandar, chief executive of trading systems provider Portware. “Advanced front-end solutions have introduced massive efficiencies, reduced operational risk and given traders unprecedented access to global liquidity.”

However, the current generation of execution management systems has taken trade and workflow automation about as far as it can.

The responsibility for a trade’s overall lifecycle—analyzing market conditions, selecting the right strategy for a particular order, monitoring execution progress and making any necessary changes—still falls to human traders.

“Over the next few years, we’re going to see firms deploying technology that will help traders automatically select and implement the optimal algorithmic strategy, allowing them to increase capacity and improve overall trading performance,” said Eskandar.

However, as much as traders want to be in the right algorithm at the right time, they also don’t want to be in the wrong algorithm at the wrong time.

“Some of the market’s recent mis-steps show just how important it is to manage trading risk,” Eskandar said. “This emerging technology will allow companies to dynamically manage their algorithms and ensure the safe operation of trading desks in any market condition.”

The advent of machine-based trading algorithms is due in no small part to the capacity to analyze reams of data in real time using advanced hardware and software.

“It’s about looking for patterns in data,” said Tucker Balch, professor of computer science at Georgia Institute of Technology, and founder of Lucena Research, an artificial intelligence-based investment technology firm. “In the case of finance, you are looking for relationships between data about a company and its future price. That’s what Lucena does, and what I do with my research at Georgia Tech.”

Mathematical Models
Lucena provides quantitative analysis and statistical machine learning technology to hedge funds, wealth advisers and advanced individual investors.

Its cloud-based artificial intelligence decision support technology enables short-term investors and traders to find market opportunities and to reduce risk in their portfolio using technical and fundamental quantitative pattern matching.

The system “get as much historical data, including fundamental data and technical indicators, as possible, and seeks to find relationships between that historical data and future prices”, said Balch. “That relationship is a model, something that relates some measurable quantity of an equity to a future price,” he said.

Lucena’s machine learning-based price forecasting algorithm forecasts five, 10 and 20 trading day returns across all covered equities.

“We don’t use static models, our forecaster is revised daily to automatically adapt to changing market conditions,” said Balch. “The forecast can be used to identify short-term long or short opportunities.”

Lucena’s ultimate goal, he said, “is to bring awareness of the power of machine learning pattern analysis, and to revolutionize the underserved investment professional community, by providing the tools and technology not normally available to businesses of their size”.