The Market Data Fire Swamp

Market Data in the trading industry reminds me of the Fire Swamp in the movie “The Princess Bride” (1987). Do you remember the Fire Swamp? It was characterized by three primary dangers - The Creatures of Unusual Size, The Lightning Sand, and the Fire Spurts.

The SEC is currently facing off with the industry over market data.

My experience going through the move from free market data in the futures and options world to required registration and payment for market data has given me this insight. So let me lay out our market data “fire swamp” for you.

The market data belongs to the exchanges.

No matter how much you want to argue this point, and how much you wish it wasn’t so, this is a fact. You can’t spin it. You can’t manipulate your way around it. The exchanges put huge amounts of effort into ensuring that their systems are available and online at all times. One rarely hears of a system outage on the matching engines. That high-quality level of stability costs.

The exchanges are for-profit corporations.

Guess what? They want to make a profit and market data represents millions of dollars of revenue.

The Three Dangers of the Market Data “Fire Swamp”

The Creatures of Unusual Size

The regulatory environment has allowed the exchanges to remain Self Regulatory Organizations (SROs). The exchanges, with their league of auditors and internal regulatory staff, have the ability to look into all activity of those trading on the exchanges and connecting through to the exchanges. Furthermore, they are also the only game in town for that activity.

This means that if they have cause they can become a very large pain with their audit capability and their ability to issue fines.

The Lightning Sand

The lightning Sand was faster than quicksand. Wesley and Buttercup were swallowed without warning. In the market data fire swamp you have lightning sand - but you likely created it yourself.

The agreements with the exchanges prevent you from re-distributing the data. Engineering teams sometimes see the data available and believe that it has been paid for therefore it can be distributed. They call it fancy names like “derived data”, “information bus”, “internal APIs”, “data lakes”, and others. The end result is that you may have some lightning sand in your shop that may need to be either reported, paid for, or at least properly tracked and provisioned.

The Fire Spurts

The exchanges have schedules of fees for how you use the market data. It is easy to be using free data and all of sudden begin using it for live trading. It is also easy to move from being classified as a retail trader to classified as a professional trader. It is easy to be using one market and quickly add another market and not realize that you have increased your market data usage because each market, or sub-exchange, has different market data rates.

Fees can spurt up and burn you.

Safely Navigating The Fire Swamp

The biggest need that I see in the industry is for solid market data governance by the consuming firms. This means covering the following:

1. Tracking your user static data (names, contact info, email)
2. Tracking systems (including internal systems) that consume market data
3. Mapping your users to the systems
4. Tracking how the data is used on each system
5. Tracking fee schedules from the exchanges.
6. Keeping historical - temporal data - on all changes to 1-5

I have a guideline I follow in many operations models, and it will help you in navigating your market data fire swamp:

“That which is measured gets better. That which is measured and reported upon gets better faster. That which is measured, reported upon at regular intervals with accountability gets better fastest.”

Links & References (aka - what caused me to write this blog):

https://www.bloomberg.com/news/articles/2018-10-03/nyse-proposes-that-sec-revise-its-test-of-how-exchanges-get-paid 

http://www.mondovisione.com/media-and-resources/news/statement-at-the-roundtable-on-market-data-and-market-access-sec-commissioner-e/?