Firms expect to increase investment in artificial intelligence and machine learning as a result of the Covid-19 pandemic, which also caused models to underperform.

The Rise Of The Data Scientist: Machine learning models for the future, a survey from data provider Refinitiv, found that 72% of firms’ models were negatively impacted by Covid-19 during the second quarter of this year.  As a result 12% of firms declared their models obsolete and 15% are building new ones.

The report said: “The main problem was the lack of agility to quickly adapt and include new data sets in models as circumstances changed.”

Alternative data also became more important as a provider of real-time data to derive insights for immediate action.

Amanda West, global head of Refinitiv Labs, told Markets Media: “Machine learning and artificial intelligence have taken centre stage.”

The increased investment over the past two years can already be seen in the average number of data science roles financial firms increasing by 26%, from 66 to 83. Nearly half, 40% of firms, expect to further boost investment in artificial intelligence and machine learning as a result of Covid-19.

David Craig, chief executive of Refinitiv, said in the report: “Covid-19 is the arch accelerator. In under six months we have experienced a level of technological change in financial markets that would otherwise have taken a decade to play out.”

The global survey on how capital markets are using machine learning and artificial intelligence had 423 respondents, including interviews with 260 data scientists interviewed and 118 quants.

Unstructured data

The number of firms that only use unstructured data – such as such as text, images or certain types of alternative data – was 17% in this survey, up  from 2% in 2018. In addition only 3% of the firms said they do not use alternative data sources, compared to 30% two years ago.

The survey said this growth reflects the power of AI/ML models fed with unstructured data to provide new signals to the business and revenue to the company.

A managing director in IT at an asset management firm in the US said in the survey: “We have a couple of data scientists who want to start tracking weather patterns and bringing in weather data on top of just news data to come up with alternative trading strategies that could help drive revenue for the firm.”

West added that it is crucial to have the right talent and data to deliver genuine insights for the business or clients.

“You need have a golden source of data or otherwise you just have lots of stuff,” she added. “New insights come from joining different datasets, for example, numeric and unstructured.”

For example, Refinitiv has a strategic partnership with BattleFin, who provide alternative data such as app installs, Web traffic, geospatial data and employment data.

The survey also found that 75% of firms are using deep learning. “This is a major and unexpected technology advance, considering that deep learning has previously been seen as niche, expensive and academic,” said the report.

West predicted an increase in natural language processing over the next 12 months to search unstructured data such as transcripts of conference calls, broker research and news articles.

She continued that Refinitiv Labs in carrying out more correlation work than a year ago. For example, Labs built Covid-19 News Tracker which applies artificial intelligence and machine learning to BERT, Google’s open source NLP model, and Refinitiv data sets including Machine Readable News and Company Fundamentals.

“We wanted to track the specific impact of Covid-19 by company and sector,” West added. “Alternative indicators such as geospatial data and mobile data may also show the speed of the recovery.”

Cloud

West explained that more sophisticated data analysis needs computing power which has led to a migration to the cloud, also accelerated by Covid-19.

Most financial services firms use more than one cloud provider to run their models. Amazon and Microsoft Azure are favorites in the Americas, while Google and Azure are somewhat more popular among sell-side organizations compared to the buy side according to the report.

West concluded: “It will become really apparent which firms know what they are doing and there will be a bifurcation.”