Data Analytics Overtakes Big Data

September 25, 2019 | By: Ivy Schmerken

Data analytics, visualization

By Ivy Schmerken

Is the term ‘big data’ headed for the dustbin of history? In capital markets, firms are experimenting with emerging technologies such as machine learning to glean insights from internal and external data sets.  Not only are brokers and large asset managers scouring unique data sets for discovering alpha, but they are also looking at data to launch new revenue streams in what’s known as ‘data monetization.’

However, the term big data is losing its relevance, according to a recent industry webinar. While the concept of big data has focused on the problem of collecting and processing vast quantities of data including social media, corporate websites, satellite images and the Internet of Things— the trend is moving toward data analytics.

Gabriel Wang

“The interest in the significance of data analytics has been growing over the last five years and taking over big data,” said Gabriel Wang, senior analyst at Aité Group, speaking on an Aug. 27 webinar “Data Analytics: Catalyst for Growth in Capital Markets.

As evidence, Wang showed a graph depicting the decline of ‘big data’ as a Google weekly search term from June 2014 through June 2019, and the rise of data analytics over the same period.

“Big data typically refers to a technology deployment strategy for data problems that are too big, too fast or too complex for conventional database technology,” said Wang.

But, “after years of building the necessary infrastructure to store and process big data, financial services realized that simply having and collecting data is no longer enough and continuing to focus on the term big data is missing the point,” said Wang.   “Instead, the industry has moved to a place where emphasizing business-oriented data strategies should be and needs to be the focus, and thus the emphasis on data analytics,” said Wang.

For example, Northern Trust recently developed a machine-learning-based pricing engine for securities lending that uses an algorithm to forecast lending rates for 34 global markets, reported Pensions & Investments.  “Our technology assesses market demand across thousands of securities and allows our traders to extract better returns for our clients,” stated Dane Fannin, head of global securities lending at Northern Trust in an Aug. 12 news release by the bank. “The potential benefits from machine learning techniques extend beyond this initial application, and we will continue exploring and developing solutions that drive value for our clients.”

Among the factors fueling the trend are sharp declines in storage costs, the rise of cloud computing and advances in data science techniques to process and analyze both structured and non-structured data.

Some of the key points from the Aité webinar are as follows:

  • The capital markets industry was one of the laggards in the adoption of a big data framework compared to other industries and other segments within financial services which are ahead of the curve. Retail banking and payments have been leveraging alternative data and data analytics for credit approval and assessment.
  • However, a few sell-side firms and top investment banks were among the frontrunners in implementing a data analytics strategy and technologies, particularly focused on risk and transaction cost analysis, said Wang.
  • On the buy side, only a few hedge funds are adopting these strategies, focusing on trading analytics or quantitative research for developing alpha generation, said Wang.
  • Most data analytics projects are at the proof-of-concept stage within business units in which analytics vendors are being evaluated. According to Aité, it is rare to see enterprise-level data analytics projects, though this is likely to change.
  • Those responsible for the data analytics function tend to be C-level executives in roles like chief data officer (CDO), chief data scientist, chief operating officer, chief information officer, chief executive officer, chief technology officer, and chief risk officers.
  • Nearly a quarter (or 24%) of respondents indicated that CEOs are responsible for data analytics projects in their organizations, while 18% pointed to chief data officers, 17% to chief data scientists, and 13% to CIOs and COOs. Three percent said that chief technology officers, chief risk officers, heads of data management or heads of advanced data analytics were functions responsible for such projects.
  • Data analytics is of strategic importance: 57% of survey respondents said they currently have a data analytics strategy in place, while 13% are currently implementing a strategy which should be in place within the next 12-24 months. Yet, not everyone has a data analytics strategy – 30% are actively considering it.

Follow the Money

Spending plans indicate that firms recognize the significance of committing capital to data analytics.   More than 40% of surveyed firms expect to increase IT spending on data analytics by at least 6% over the next 24 months, while 17% expect their spending to decrease.

About half (47%) of respondents consider data analytics to be a high priority, and most (37%) view it as a medium priority, while a minority (18%) see this function as a low priority.  Surprisingly, “firms that place a low priority on data analytics all hail from the hedge fund sector,” said Wang on the webinar. Yet this contradicted with the point that some hedge funds are developing analytics for trading or quantitative research.

Wang explained that hedge funds tend to outsource their middle and back-office functions, which means the data analytics function sits outside their trading desk and internal teams. Depending on the size of firm, some hedge funds may outsource data analytics functions related to trading, such as data gathering and data cleansing, to a third-party provider when they have limited bandwidth.

Main Drivers of Data Analytics

Risk management support was the most frequently mentioned driver cited by 60% of respondents in their decision to look at data analytics. “The ability to store and process substantial amounts of structured and unstructured data is fundamental to risk management support,” said Wang.  “Having the right data analytic tools to monitor and measure risk from those data sets have become imperative in capital market’s daily operations,” he said.

Other top drivers of data analytics projects are increasing data volume and size, programs for strategic data quality, and lower down on the list, the need for unstructured data support.

When asked what they value most about data analytics, respondents ranked better predictive ability, speed of analytics, and the ability to drill down into more granular insights as the top three aspects. Compliance appeared lower down the list, mainly for trade surveillance, but this is of increasing importance and is likely to be a key area for data analytics investment in the future, said Wang.

Almost 80% of participants ranked the ability “to scale to meet future requirements” as the most important aspect of a data analytics solution in their organization, which was almost on par with deriving signals from analytical insights (72%).

Predictive analytics is among the most relevant aspects of data analytics tools or techniques utilized by 77% of respondents. One area gaining momentum is the adoption of natural language text analytics (44%), versus other emerging technologies such as geospatial analytics (33%), voice analytics (17%) and video analytics (3%).

Data analytics strategies are being applied to a wide range of functions ranging from front office to data management. The most popular use cases are trading analytics, quantitative research, risk simulation and modeling, and transaction cost analysis, all of which are tied to the front office and revenue generating opportunities.

Because these strategies are being applied to many different areas, it is challenging to “bundle” them together under one centralized data analytics team. As a result, there tends to be disparate data analytics projects occurring across firms, initiated at the business unit level, rather than under the remit of a dedicated data analytics team.

Data Monetization

While firms invest in data analytics solutions to gain better insight into their internal customer data and digital workflows, firms are also looking to create new businesses and generate revenue streams from data offerings. “In some firms, data is shifting from serving as a secondary asset that supports positions, processes and digital strategy, to a primary asset that businesses can productize and sell,” said Wang.

More than half of the buy- and sell-side survey participants said they either currently have a data monetization strategy in place, or they have considered it and will implement a strategy within 24 months, according to Aite’s survey. Adding new services on top of existing services is the most common approach, though some are creating new business models.

However, Wang pointed out that buy side asset managers may have difficulty aligning a data monetization strategy with their traditional business goals. Though asset managers have a lot of market data and customer data to analyze, they have a regulatory mandate to protect the privacy of customer data.

Help Wanted: Data Scientists

Nevertheless, the buy-side community appears to be lagging a bit behind the curve, as compared to the sell side, noted Wang. Although quantitative asset managers and hedge funds may be utilizing alternative data and data analytic platforms for a better grasp of data in order to assist with the quantitative trading strategies, such as sentiment analysis, there are more projects on the sell side and the market infrastructure community.

One of the key factors in developing or implementing data analytics is the hiring and retention of skilled employees. In particular, the role of data scientists has become key to data analytic projects. About 73% of respondent firms either employ a data scientist or plan to employ a data scientist in the next 24 months, according to Aité’s survey.

Data scientist roles are difficult to fill as they require a blend of hard-to-find skill sets. Not only is statistical analysis required, but machine learning has become an important requirement in recruiting data scientists as well as financial knowledge and domain expertise in capital markets.

Challenges: Integration and Standards

Despite the increasing adoption of data analytic projects within capital markets, there are some challenges hindering their progress. “Integrating between different data platforms and establishing a standard process for ingesting data or moving and transforming data within the data analytics environment are the commonly cited challenges,” said Wang.

Because business units are still siloed, such that firms don’t have a single data model or taxonomy across business lines, and some legacy systems are a nightmare to extract data from, said the analyst. “Analyzing massive amounts of data sitting on a firm’s data analytics platforms and transmitting data from numerous data sources may lead to significant challenges,” he said.

Even with these challenges, firms are keen to apply data analytics to their businesses. “There is recognition that data analytics is a strategic priority in capital markets, and those firms that are behind the adoption curve will dedicate resources to catch up over the next two years,” said Wang.

brokers routing
Ivy Schmerken

Data Integrations Available with FlexTrade’s Execution Management System Technology

For further information, please contact us at

Past FlexAdvantage blog posts related to Data Issues