Paul the Octopus

He’s done it again!

You got to love Paul the Octopus. To those of you living across the pond and not following football (soccer) news, that’s the cute octopus-turned-oracle that managed to predict each and every game Germany played in the World Cup, and foreseen that Spain will beat the Netherlands in the finals.   

Paul’s feat is outstanding. The German national team lost against Serbia, won the two games against Australia and Ghana, sent England home in the early knock-out phase, passed Argentina in the quarter finals, was defeated by Spain in the semis, and won the third place by winning over Uruguay. Then Spain and Netherlands met for a tough decider in the finals, and won the World Cup championship. All of these results were forecasted by Paul.

The naïve odds (assuming both sides are of equal strength) are 1:256. I didn’t calculate the product of the bookie odds (which reflect the team’s relative strengths), but they must be north of 1:100 as some of the games weren’t that trivial to call.

The now legendary prophet, a 2-year-old octopus, was given prior to each match a choice between two food canisters decorated with flags of the opposing teams. He went after what later proved to be the right canister every time.

100% detection rate.

Uncanny.

This reminds me of a conversation I recently had with a security manager in one of the major European banks. The bank implemented a transaction monitoring system and had unbelievable results; but when I met him in a security event organized by the bank a few months later, his face was ashen.

“We’re down to 97% detection”, he mourned.

When I pointed out this is actually a pretty amazing result, he just wouldn’t have it. “It was 100%. I want it to stay 100%. Anything less is unacceptable”.

Since it was a bit difficult to reason with this statement, I pulled a senior Gartner analyst who was about to leave after giving an excellent presentation on recent trends in online fraud and mitigation. “99%”, she joked. “99% detection is good”.

What is good detection, anyway? Well, to start with, never talk about detection without talking about false positives. In the first-ever deployment of a risk-based authentication system, back in August 2004, the bank had 150 basis points of eCommerce fraud (that’s 1.5% of transaction value) and 30:1 good-to-fraud ratio, i.e. for every fraud stopped there were also 30 good transactions declined. This is a horrible mix. Following the deployment, fraud levels dropped to a manageable 20 basis points and false positives to 2:1 good-to-fraud.

But this didn’t equate to 100% detection. In fact, in later years when the system started to cover online banking fraud, the typical balance was around 80% detection while flagging less than 0.3% of genuine transactions for review or challenge. Some banks decided to focus on detection and increased the operational resources to review additional cases or challenge additional users (e.g. with out-of-band authentication); some banks were OK with a slightly less detection level as they didn’t have a lot of fraud to begin with, and couldn’t justify working out many suspicious cases.

There are many things you can do to increase the detection rate, but they all have a cost. You either invest in more technology, more people to review transactions, more analysts to devise fraud strategy, more genuine customers you reject. At the end of the day it’s a cost-benefit analysis that tells you were to stop.

In any event, 100% detection over time isn’t a sustainable strategy.

Unless you hire Paul the Octopus, of course.