Marketplace analytics - Stop running your business blindly

Marketplaces are emerging in every sector and the financial sector is no exception to this trend. For every product or service, for which multiple parties interact, marketplaces unfold to better match the supply and demand and improve the cooperation between the involved parties.
Marketplaces have however unique dynamics, due to their multi-facial nature, which means that the balance between the interests of the different involved parties needs to be carefully managed (i.e. achieve and conserve a critical mass for both supply and demand).

In the spirit of Peter Drucker’s teaching "If you can’t measure it, you can’t improve it", it is important to measure every aspect of the marketplace, in order to properly manage it.
Marketplace owners should be able to answer instantly several business questions, e.g.

• What is the number of new visitors in the last X days?

• What is the percentage of visitors turned into new (paying) customers?

• What is the level of satisfaction of the users over time?

• What are the different customer touch points?

• What are the areas users get stuck?

• What is the average lifetime value of the customers?

• …​

In order to answer these questions, an immediate visualization of several key metrics or KPIs (= key performance indicators) is required, including the possibility to dig into the details of these metrics. These metrics should steer all decisions, allowing to make those changes to the platform that generate the most value over time. This means that each metric should be actionable (e.g. a metric like a number of transactions per day is not actionable), measured continuously and shared across business and IT, allowing everyone to work towards the same goals (different goals between IT and Business only leads to silo-thinking and conflicts of interest).

Of course, the metrics only give insights in what is currently (and in the past) going well and what is going wrong, but don’t give much info on the right plan to improve the platform. This is where techniques like canary testing and A/B testing come in the picture, allowing to see the effects on a change rapidly and in a controlled fashion.

A platform owner should also have different views on the metrics, as a marketplace can be confronted with different issues, e.g.

• Business view: providing info if the system is supporting properly the business

• Performance: monitoring of the technical performance of the platform, i.e. monitoring average response times, tail latencies…​ at different levels (i.e. how much time does a business process take, how much time takes an API call, how much time does a query in a database take…​)

• Errors/Bugs: identify erroneous situations at different levels, e.g. business processes unexpectedly aborting, API calls failing, disks running full…​

• Availability: is the platform available for end-users and what is the availability of different components (e.g. screen, API, micro-service, Docker container, server, disk…​)

• …​

Technically metrics are collected by different tools:

• Web metrics: a tool like Google Analytics or Adobe Analytics captures the key information of the usage of the website, collecting info like number of visitors, where are they coming from, what pages of the website are they visiting.

• Business metrics: different solutions exist, depending on the underlying technology. Some business metrics are exposed by "Business Activity Monitors" (BAM) often built on top of a BPMS (Business Process Management System), others are custom-built in the application, while others show business dashboards created from log entries stored in e.g. ElasticSearch. Often these tools are also integrated with some kind of BI-tool (Business Intelligence) allowing to further drill-down on the metrics (e.g. Tableau, SAP BusinessObjects, QlikView, IBM Cognos analytics…​).

• Application metrics: collect and visualize several metrics on the functioning of different components within an application. E.g. the availability of the application, the latency in different components, number of successful/failed API requests split by API…​ Also these metrics can be collected and visualized by different tools. Often such monitoring is included in middleware products like API Gateways, ESBs or BPMS systems, but often also measured via robot-based monitoring, by log analysis (e.g. ELK stack) or by exposing a few applicative metrics (captured by e.g. Prometheus).

• Technical metrics: collect and visualize information about the health of individual pieces of software and hardware. Often this includes monitoring the availability of containers, disk space, memory and CPU usage…​ Different tools also exist to support those use cases (e.g. IBM Tivoli Monitoring).

Independent of the tooling and type of metrics, several steps are always identified to come to an actionable metric:

• Gather the relevant data

• Process and analyze the data (e.g. filter, aggregate…​). This can be rule-based, but machine learning techniques can also be exploited.

• Visualize in pre-defined reports and dashboards, with possibility to drill-down

• Act upon metrics/insights. This can be manually, but also automatically via throwing alerts (mail, SMS, Slack messages…​) or launching specific actions/processes (e.g. recycling mechanisms, restarts, disabling parts of the platform, activate back-up plans…​)

When focussing on the web and business metrics, allowing to monitor the "consumer" (= buyer/investor) and "producer" (=issuer/supplier/seller) side of a marketplace, we can identify 3 groups of metrics:

• Usage metrics: these metrics help to understand how many people visit the platform and how they spend their time there. It will help to answer questions like where the customers found out about the marketplace, which campaigns are driving the most traffic, what are the visitor statistics…​ The 3 most important metrics in this category are:

  • Monthly Active Users (MAU): this metric counts the number of unique users who have visited the platform at least once during a certain time period. This number should be growing.

  • Bounce Rate: the bounce rate measures the percentage of users, who visit the website without any engagement (i.e. bounce right away). This number should be as low as possible (even though popular marketplaces like eBay, Amazon and Etsy also have bounce rates between 20 and 25%).

  • Time spent on site: the average time users spend on the marketplace. Normally the more time users spend on the marketplace, the more profitable. If the time is high, it is however important to drill-down to see if the long times are not caused by bottlenecks or difficult usability of the website.

• Transaction metrics: these metrics allow to see if the delicate marketplace balance between producers and consumers is healthy. Several metrics allow to get rapid insights into this balance:

  • Liquidity: this metric expresses the expectation of a provider to sell something (= provider/seller liquidity) or of the consumer to find something he is looking for (= consumer/buyer liquidity). Typically, the liquidity is expressed via following metrics:

    • Provider/Seller liquidity: percentage of listing that result in transactions within a specific time period, i.e. "# Listings with transactions / # Total Listings". This figure should be as high as possible.

    • Consumer/Buyer liquidity: probability that a visit will turn into a transaction, i.e. "# Transactions / # Visits". This figure should be as high as possible.

    • Provider-to-consumer ratio / Buyer-to-Seller ratio / Investor-to-Issuer ratio: for this figure there is no optimal (e.g. ratio of Airbnb is 1:70, Uber is 1:50 and eBay is 1:5), but the figure is important to know where to focus on. The lower the figure, the more the platform should focus on increasing the number of producers/suppliers (while still ensure that Provider/Seller liquidity remains strong).

    • Buyer/Seller overlap: percentage of Buyers also acting as Sellers.

    • Transactions Per Buyer (TPB), i.e. "# Transactions / # Buyers".

    • Transactions Per Sellers (TPS), i.e. "# Transactions / # Sellers"

  • Repeat purchase ratio: percentage of transactions, which are repeat purchases, i.e. purchases from visitors who have already made purchases on the marketplace. This metric should be as high as possible, if the Quick Ratio is not dropping (avoid that only a few very loyal and active customers remain).

  • Quick ratio: percentage of new and resurrected users (i.e. users which did not have any activity for a long time but became active again) versus churned users. This metric should also be as high as possible (showing an increase of active users)

• Business Metrics: the business metrics express how the marketplace is doing financially and answer questions like how much money did the marketplace make today, what is the average transaction amount…​ The most important metrics are:

  • Gross Merchandise Volume (GMV): total sales value of products or services through the marketplace during a specific time period.

  • Total marketplace revenue: the total commission generated by the marketplace

  • Take Rate (Rake): percentage of the Gross Merchandise Value captured by the marketplace, i.e. (Average) Take Rate % = GMV / Total marketplace revenue

  • Average Order Value (AOV): the average value of 1 transaction, i.e. AOV = GMV / # Transactions. This figure gives an indication of the type of transaction which are done.

  • Average items sold per order: the number of items executed in 1 transaction (e.g. via a shopping cart), i.e. # Items sold / # Transactions

  • Average amount of repurchases: number of repurchases a customer will do on average.

  • Customer Acquisition Cost (CAC): price paid to acquire a new customer (i.e. marketing, support/community management, increased infrastructure cost…​). This number should be as low as possible, featuring an organic growth of the marketplace.

  • Customer Lifetime Value (CLV): average total amount of revenue expected from a customer. CLV = AOV * Average amount of repeat purchases

Apart from the above metrics, which can be automatically measured, it is also important to understand why certain metrics behave as they behave. In-depth analysis and trial-and-error mechanisms (like A/B testing and Canary testing) can help with that, but it’s also important to talk to the users, via user groups and surveys. This can be unstructured, but in order to monitor it’s also important to get certain actionable metrics, i.e. so-called "User Satisfaction Metrics". These metrics can be collected via specialized online tools, like UserReport, UserVoice, Qualaroo or SurveyMonkey. 2 metrics are often used to express user satisfaction:

• Net Promoter Score (NPS): for this score, users are asked how likely they would recommend the marketplace to a friend or colleague, providing a score from 0 to 10. The replies are then categorized in 3 categories: 9-10 are promoters, 7-8 are passives and 0-6 are detractors. Afterwards the percentage of customers who are Detractors is subtracted from percentage of Promoters. This gives a score ranging from -100 (everybody is a Detractor) to +100 (everybody is a Promoter). A positive NPS is good, while an NPS of more than 50 is excellent.

• Sean Ellis Test: for this score, users are asked how they would feel if they could no longer use the marketplace. The user can give 4 possible answers: very disappointed, somewhat disappointed, not disappointed or user already doesn’t use the marketplace. The score calculates the percentage of "very disappointed" answered compared to the total number of answers. A score of more than 40% gives an indication of product/market fit. A score below is an indication change is required to the marketplace.

Actively measuring all of the above metrics should give valuable insights on how to take the right decisions for growing out a successful marketplace.