Financial institutions are required by law to know their customers (KYC), and refuse services to any party involved in financing terrorism or money laundering. Unfortunately regulations are open to interpretation, leaving banks struggling to define who is a high risk customer, as they attempt to please both the regulators and the public.

Allowing parties on government sanctions lists to open accounts results in huge fines, but a better-safe-than-sorry approach can damage your reputation on the high street. A number of institutions have received significant fines from industry bodies for failure to detect money laundering activities. On the other side, institutions have suffered reputational damage for the closure of accounts due to a perceived risk that is not necessarily apparent to the account holders. This creates a plethora of challenges for banks.

Investigatory work by journalists this year has shed further light on how banks manage both customer and regulator needs, and introduced a new player onto the field for consideration – the data provider.

Typically banks use an automated tool to match new customers and their existing customers against sanctions lists in regular review cycles, with the lists provided by a database subscription service. These data providers combine lists issued by governments together with their own research to produce a comprehensive file of who not to accept.

Banks can choose which lists they want to use for their screening, but they must use the right lists for the regions in which they do business, so a multi-national institution will be bound to comply with a number of different authorities.

But a bank's actions are only as accurate as the data they receive. For example, organisations may be included on a sanctions list due to historic links with a named individual. In such cases the reason for the original listing may have been addressed many years ago but if that label is unchanged the bank may well decide to close the account rather than risk a large fine.

Providers do their level best to ensure that data is as detailed as possible, but often sources are contradictory and constantly changing in response to global events and occasionally are not sufficiently or frequently updated. Suppliers to the industry maintain that their data comes from reliable sources and the decision to open or close accounts lies with the bank.

So how could this process be improved? Everyone agrees that terrorist groups and other criminals should be excluded from financial services, whilst well-meaning customers should not be disturbed. The balance is hard to find, and often the banks end up paying twice for their judgement, through both fines and customer opinion.

The key elements of sanctions detection are an understanding of the legislation, good tools to detect any matches, and accurate data – but this is hard to put into practice. Within the City of London it is estimated that close to £1 billion is being spent annually by organisations undertaking KYC remediation projects, to avoid fines and sanctions by the regulators. Perhaps regulators could do more to work with banks on understanding the rules? Perhaps banks should invest more in their software? Perhaps data providers should increase the accuracy of their data? But who should bear the ultimate responsibility; in the end it is society that benefits, or is penalised for mistakes. Isn't it time for all parties involved in this critical piece of the financial services industry to work together for the common good?