Swathes of regulatory change have had a lasting and profound effect on the financial services sector.  However, technological change and massive IT disruption is becoming increasingly important, specifically intersecting new “big data” technologies with modelling and analytics.

Machine learning is one increasingly popular modelling approach used to gain insight from the proliferation of data, for example to detect anomalies, determine trends and make forecasts.  The value of machine learning comes from its ability to discover useful patterns in complex data sets often without a predetermined model. Such models can adapt as data changes and numbers of learning samples increases.   

There are two key workflows in machine learning; ‘unsupervised learning’ and ‘supervised learning’. In the case of the latter, the model learns from – i.e. is “supervised” by - the (labelled) data available.  In the absence of labelled data, an unsupervised approach is usually followed.

While machine learning has been successfully applied in other sectors such as the robotics, medical, and defence industries, a recent survey reveals that machine learning has not yet reached mainstream adoption within financial services, with only 12% of quant-savvy respondents reporting that they already use machine learning techniques.

We see this hesitancy of machine learning adoption more a case of teams being able and confident to marry successfully the (many) methods to the data and/or problem at hand. A misalignment can have consequences: applying the wrong method to the wrong data-set can generate poor results, perhaps leading to bad trades or poor risk factor identification. Such concerns noted in the survey include over-fitting (23%) and that machine learning is too much of a black box approach (22%). Despite such concerns, many respondents are keen to explore opportunities with 40% stating they want to learn more. 

The survey represents a cautious financial services reflection of this week’s The Future of Life Institute Open Letter: Research Priorities for Robust and Beneficial Artificial Intelligence, which espouses interdisciplinary collaboration to agree a formal “robust” and ethically-astute application of machine learning and other methods. 

So despite concerns and initial hesitancy, machine learning is here to stay and I expect to see its use grow. A sound understanding of both data and methodology is important to enable effective adoption and avoid potential pitfalls, but when applied successfully can, and in some cases already does, provide significant competitive advantage. Traditional statistics and equation- based modelling will not disappear, but machine learning is one major and increasingly powerful cog within Financial Services’ emergent fast-paced “big data” systems. Like any powerful modelling technology, users must handle with care to generate opportunity and reduce risk.