Quantifi Survey Measures Adoption of Data Science in Finance

Data has a huge influence on the financial services industry. The volumes of data accumulated are so large that traditional evaluation and analysis methods are no longer suitable. Firms are now recognising that big data technologies, like data science, are the way forward. Using data science can help them focus their resources efficiently, make smarter […]
2 Jun, 2020

Data has a huge influence on the financial services industry. The volumes of data accumulated are so large that traditional evaluation and analysis methods are no longer suitable. Firms are now recognising that big data technologies, like data science, are the way forward. Using data science can help them focus their resources efficiently, make smarter decisions, and improve performance.

This survey was conducted during a webinar Quantifi hosted, featuring Celent, on ‘Next Generation Risk technology Powered by Data Science’. Over 180 individuals from the financial services industry registered for the webinar and were invited to take part in the survey.


Over the course of the last few years, there has been a step change in the role that data and technology is playing in risk management and investment decision making. Powered by data science, data analytic techniques previously considered as emerging or experimental are increasingly being adopted as mainstream. Firms are deploying data science to improve risk assessment and business response strategies, and bring more rigour to their operations.

Firms that are not looking to adopt in-house data science capabilities have the option to leverage technology providers. External providers like Quantifi, who have embedded data science, can offer a range of features including the ability to compose risk analytics, product structuring and testing, hedge construction and development of trading strategies.

Regardless of the approach taken, it is clear that data science is going to play a pivotal role across a number of business and investment strategies.

1. How would you describe your firm’s strategy for incorporating data science in your investment and/or risk management process?

The ability to harness the power of data through data science is extremely valuable as it helps firms understand the nature of risks and cope better with regulations. Whether or not to incorporate data science in an organisation is an important strategic decision that requires careful consideration to avoid sub-par results. For 23% of firms surveyed, data science is already a major point of focus across the organisation and for 12% it is a major point of focus for limited adoption. Over half (53%) are currently assessing whether to implement and deploy data science. These firms must ensure they understand what they need and the scope of the tasks to be achieved by using data science tools. Done correctly, data science can offer a competitive advantage by providing valuable insights into new investment opportunities and risk mitigation strategies.

2. What is the most compelling business case for leveraging data science?

The implementation of data science is rapidly changing the face of the financial services industry. Data science can be applied to perform various important tasks. For example, it has created a new capacity for powerful analysis by traders such as using data science for event based trading strategies by forecasting defaults, earnings and corporate actions and then structuring portfolios to take advantage of possible arbitrage opportunities when an event occurs. Data science uses scientific methods, processes, algorithms, and systems to extract knowledge from data. Leveraging this data to make major decisions is a key strategic practice for any business. A good example of this would is back-testing: if a portfolio manager constructs a portfolio or a trader structures a product, these can be back-tested against historical data or stressed historical data to anticipate how the portfolio would perform. Both forecasting and decision-making as well as trading and hedging strategies ranked as the most (29%) compelling use cases for leveraging data science. Risk management and compliance (24%) was selected as another popular use case with participants.

Risk management and compliance functions are highly dependent on enterprise-grade data and advanced analytics. To support the increased volume of data required, a number of new technologies have emerged to help firms analyse this data. These cutting-edge data science platforms provide a level of sophistication that was not previously possible with traditional methods.

3. Which data science capability do you consider the most important?

In order to leverage data science and modern machine learning algorithms, it is important to have an infrastructure in place that ensures that data is of a high quality. Almost half (47%) of participants see improving data ingestion and data management (MDM) as the most important capability to develop. The garbage in, garbage out principle applies strongly to machine learning and ensuring that data is standardised, validated and made accessible throughout the organisation is a pre-requisite to obtaining meaningful results from sophisticated models. Model development and validation (Python) is also considered a key capability (41%), whereas the ability to build ad-hoc dashboards and reports (BI) received the lowest response (12%). This could indicate that market practitioners are keen to utilise the cutting-edge models and Python libraries that the machine learning community has developed in recent years. These models and libraries go beyond the simple exploratory analysis, summary statistics and dashboards that business analysts have been using for decades. Overall, the findings suggest that rather than rapidly deploying prototypes, firms are working to put in place the right foundational elements to ensure success of their data science projects. This includes focusing on data quality, model explainability and transparency. These components are key in building trust in the model when deploying it for different use cases.

4. What are the hurdles for the wider adoption of data science?

While the value of data science is becoming more recognised, it is important for firms to realise that adopting this new technology presents a number of hurdles. Respondents cited having resource with the required skills as the biggest hurdle (46%), followed by senior management commitment (25%) and identifying appropriate use cases (21%).

Even with the best data science platform, the success of implementation and results comes from having the people with the right skills who can explore and examine data to find hidden patterns, perform advanced mathematical and statistical analysis, and present actionable insights. For many firms, the real test will not be in developing capabilities, but the agility by which they can divert and redeploy their resources from everyday work streams.

21% of respondents feel the biggest hurdle is identifying appropriate use cases, although, as the diagram below highlights, there are a number of use cases across markets trading and risk, banking, investment management and non-financial risk activities.

5. What are the risks with adopting data science i.e. shadow IT, key person risk, security & privacy, other?

The successful implementation of data science capabilities will depend on a firm’s responsiveness and agility from people, process, and infrastructure standpoints. As with any emerging and disruptive technology, firms need to strike the right balance between innovation and risk management.

The statements above are a selection of responses from participants and highlight the most commonly voiced concerns about the adoption of data science from the survey. The race to adopt new technologies poses a level of risk. The most common risks associated with adopting data science, as noted by survey respondents, are key person risk and IT security and privacy.

The last decade, especially the past five years, has seen the rise of symbiotic, emerging technologies and next-generation approaches which are fanning the flames of innovation and change across the financial industry and beyond. Financial services and their risk management functions have always been a participant in, as well as a beneficiary of, technology advancements. However, with the current shift in emerging technologies, this is both a source of risk, as well as an enabler, for many institutions. The deployment of data science techniques provides a huge opportunity for firms to stand out from the competition and reinvent their businesses. Implementation of data science in an organisation requires a dedicated strategy to avoid sub-par results and information overload. When done correctly, it can offer a competitive advantage, insights and even new ways to tackle old problems.

Let's talk!

Speak with one of our solution experts