Social media is creating a torrent of data every minute.  But how much of this firehose is relevant to making better investment decisions?

As hedge funds and asset managers eye an explosion in unstructured data from Twitter, Google searches, and other social content, many are subscribing to feeds that mine this data to detect events.

Wall Street’s appetite for filtering and analyzing this social data has grown since it could be crucial to predicting the movement of stocks, currencies and derivatives.  The trend is fueled by not only the surge in social media, blogs and corporate filings, but the plummeting cost of computer servers and storage, making it possible to run analytics on the cloud.

“We are in the business of transforming social content into market data using natural language processing,” explained Gautham Sastri, president and CEO of iSentium, a Miami-based sentiment analysis firm that spoke on Markets Media’s Summer Trading Network artificial intelligence panel.

In July of 2015, iSentium had 10 clients who paid on a monthly basis. They include large quantitative hedge funds that “use machines to find statistical relationship between data points to help predict price movements,” according to The Wall Street Journal article, “Tweets Give Bird’s Eye View of Stocks.” Other clients include traditional hedge funds, family offices and high frequency trading firms.

Quantitative hedge funds have been early adopters of sentiment analysis, which lets a trader know whether consumer sentiment for an article is positive or negative.

“A lot of quants create different strategies and execute on the signal,” explained Kumesh Aroomoogan, co-founder and CEO of Accern, a startup which runs a real-time web surveillance platform that monitors 20 million web sites, pulling five million articles per day.  The company was founded in 2013 to help hedge funds find alpha from the World Wide Web.

Some of the largest hedge funds are using the firm’s data feed for algo trading, he said.

Accern calls its method “impact analysis,” which lets the trader know if an article is going to have a high or low impact on the prices.  The firm has an algorithm to track how many people have seen an article.  The typical advantage is that a trader wants to enter a position when the story has low exposure to get ahead of the market.  “And once a lot of people have seen that story, he can exit the position.”

The burgeoning interest in sentiment analysis reflects the broader interest in AI techniques, such as pattern recognition and machine learning. “There is software out there that can mine terabytes of data in seconds and predict how markets will react to certain events, “said Richard Johnson, Vice President, Market Structure & Technology at Greenwich Associates, who moderated the session.

“Pattern recognition techniques can help compliance departments to spot unusual behavior and some products can parse even large quantities of unstructured data, extract key analytics and construct a research report,” said Johnson.

FinTech Partners

As regulators seek more transparency into order routing decisions and best execution, financial service organizations are facing demands around aggregation of data to produce actionable information through analytics. As such, they are looking to partner with FinTech providers.

For example, aggregated surveillance solutions are types of FinTech solutions that are adding value to data.  Examples include Palantir Technologies, an analytics company that brings together massive data sets, scouring them for patterns invisible to the human eye.  It may have helped the US government catch Osama Bin Laden, wrote Fortune.

“Though FinTech partners can be viewed as a threat, they are also adding value to their data,” said Scott Mullins, worldwide financial services business development leader at Amazon Web Services, who spoke on Tabb Forum’s recent webinar, “Taming the Data Behemoth — How to Align Your Data Infrastructure to the 21^st Century.”

In fact, some of the new big data analytics startups are running their companies on the cloud. “Ticker Tags is a Fintech organization that streams Twitter feeds into AWS and then provides analytics of those social streams and develops tag libraries, noted Amazon’s Mullins.

Founder and CEO of Ticker Tags, Chris Camillo, is the author of “Laughing at Wall Street,” a book about how he turned $20,000 into two million dollars in a few years by identifying market movers based on social trends and public sentiment rather than relying on professionals. Instead of reading 15,000 tweets a night to identify market movers, Camillo and his chief engineer developed tags that track the social trends.

Ticker Tags applies a social taxonomy to every product, brand, competitor, topics, affiliated people, places, regulatory threats, etc., that are relevant to public stocks, according to the firm’s site.

Today, the company has over 350,000 tags.  It allows people to use the platform to stream information from Twitter and tag libraries, and look for social sentiment, according to Mullins. For example, the trend “gluten-free” can be traced to every product, company and competitor, noted the Dallas Business Journal in 2015 when the platform was launched. The mentions of the tag are indexed and tracked by Ticker Tags and consumer sentiment.

Possible Problems

Yet, Twitter has been subjected to hacks and hoaxes. Skeptics often point to Derwent Capital Markets, a $40 million hedge fund that made decisions based on Twitter analysis and shuttered after one month of operating in 2012. So it remains to be seen whether or not firms can rely on analysis of social media data as their sole input.

Even so, many of the new sentiment analysis firms are banking on their own proprietary methods, from using their secret sauce down to the arrangement of words in a tweet.

What’s the next phase?

Some of the FinTech providers have created their own index funds tied to consumer sentiment as a way to prove these techniques are generating returns that are perhaps better than humans. iSentium has an index fund based on its model. “It was up 25.3% last year and it’s doing quite well year to date,” said Sastri on the Markets Media AI panel. The S&P 500 lost 0.73 % last year.

Executives at iSentium told Business Insider last September that it was working with an investment bank to launch an ETF of their own using social data. Another rival, Market Prophit, has already developed an index of the 25-most mentioned companies on Twitter in partnership with S&P Dow Jones Indices. Its CEO spoke of launching an ETF based on sentiment expressed on Twitter.

With so much big data pouring into the ecosystem, clearly sentiment analysis is at the early stages of discovery and appears to be gaining the attention of established fund managers.  Now with institutional technology-grade platforms like Accern opening up to the individual investor, sentiment analysis appears to be going mainstream.