AI for hedge funds: How can machine learning and code optimisation generate greater alpha?

The applications of AI in the financial sector are multi-faceted. In this article, we explore some key use cases of AI for hedge funds. We also look at potential challenges in implementing AI-based solutions, and how hedge funds can circumvent these challenges.

In an analysis done in 2020, consulting and research firm Cerulli claims that there is increasingly strong evidence for hedge funds to use AI technologies. Cerulli’s findings note, “The cumulative return of [Europe-domiciled] AI-led hedge funds was almost three times higher than that of the overall hedge fund universe during this [2013 – 2019] period: 33.9% compared to 12.1%.”

AI-driven hedge funds currently use machine learning for tasks such as analysing data, making stock trades and calculating payouts. It is worthwhile discussing some of these applications in detail.

Applications of AI in hedge funds

Here are four ways in which we think AI can help hedge funds maximise their trading outcomes:

Algorithmic trading: Trading involves considering a range of independent variables which impact the value of assets and making investment decisions that lead to higher returns. Human traders and traditional computational modelling may not be able to sift through large bodies of information efficiently enough to make timely trading decisions.

With AI-based algorithmic trading, numerous machine learning models can be easily utilised to conduct automatic trading, harnessing new insights which were not attainable before.1 AI-based algorithmic trading models also facilitate independent trading with minimal intervention from human traders. For example, based on historical and predicted asset pricing, a model can be developed and trained to make a trade at a given time point. Compared to traditional trading techniques, this model will process a larger body of data at an expedited rate, making the trading process more accurate and efficient.

Volatility forecasting: Market uncertainties make accurate volatility predictions crucial in fund management. Needless to say, a better understanding and prediction of volatility lead to improved trading decisions and higher returns. Conventional approaches and econometric modelling can predict volatility, but often they are unable to map complex and nonlinear relationships between factors that contribute to volatility.

However, machine learning-based approaches are able to make much more precise predictions of volatility. Particularly by taking more flexible approaches to understanding variance (an underlying measure of volatility), machine learning models are able to increase the accuracy of volatility predictions. 

Signal monitoring: Studies have shown that alpha on new trades decays in about 12 months on average. Since trading decisions are made based on predictive relationships and signals, it is essential that funds monitor and retrieve high-quality signals. Signal overcrowding can be particularly concerning, leading to overlapping trading positions and alpha decay.

However, with machine learning-enabled technologies, hedge funds are able to identify diverse and hitherto uncommon signals, allowing them to avoid trading based on overcrowded signals.

Natural Language Processing (NLP), a branch of AI that looks at analysing and deriving insights from large bodies of text data is particularly useful for hedge funds to derive foresight and signals from unstructured textual data from a wide variety of sources such as news, social media, blogs, and transactions.

Generating alpha factors: Alpha measures the performance of a fund in comparison to an appropriate benchmark, and is an indicator of the value a fund manager adds or subtracts from a portfolio.6 Signals that lead to greater alpha than the returns of the benchmark index are considered alpha factors. Alpha factors are used to explain the behaviour of factors effecting the market and they also capture market risk. Feature engineering, a key component of machine learning, can be use in trading to supplement research into alpha factors.

In AI-based trading, by leveraging feature engineering, factors that better capture the risks embodied by the return drivers are generated from original data and they are manipulated to derive more impactful features. These features become the alpha factors which can then be used to generate greater alpha.

Causal inference: There are many factors (features) that can help explain financial phenomena which are of interest. Unfortunately, as is the case most of the time, we do not know which factors are directly affecting (causing) each other. AI-based solutions are able to shed more light on understanding the inter-relationships among features and selecting the most appropriate subset of features relevant for a specific machine learning task. Understanding the causal direction will help hedge fund managers ask more informative questions and construct better trading decisions

Challenges in adopting AI-based solutions

While the use of AI in hedge funds seems rather appealing, it is more so conceptually than in practice.

• Time-sensitive investment decisions need to be made fast before market conditions change, and this is particularly crucial for hedge funds. However, building a custom AI solution or a machine learning model for a given investment task can be extremely time-consuming. Based on data in the Algorithmia 2020 State of Enterprise Machine Learning report, companies can take from 8 to 90 or more days to deploy a single machine learning model. Hedge funds often do not have the liberty to spend so much time building, evaluating, and deploying a machine learning model.
• Developing even the most basic version of a machine learning model can be costly, which includes but is not limited to model infrastructure, data support, and engineering costs. You can expect to spend around $60K over the first five years for the model.
• The model building process would require subject matter experts, but it is often difficult and costly to find and hire AI experts.

Circumventing the challenges: Code optimisation for ML

In order to successfully integrate AI into the investment decision-making process, hedge funds need to explore technology solutions that can function with minimal financial, time, and computational demands. These AI systems should also be able to efficiently generate models that are both explainable and not overfitting or underfitting.

In trading, efficiency is money. It is estimated that a 1-millisecond advantage in trading can be worth $100 million a year. Code optimisation reduces inefficiencies at the source code level, reducing latency and improving model performance by over 50%, ultimately leading to faster and more profitable trading outcomes.