Royal Bank of Scotland is refusing to confirm or deny reports that an inexperienced IT operative in India was partially responsible for the computer meltdown that afflicted payments processing at the bank last week.
According to reports in UK tech rag The Register, the junior technician in India accidentally wiped out a queue of batched payments while backing out of a routine software upgrade.
Quoting a 'source familiar with the matter', the Register says that the situation spiralled out of control during an attempt to reverse a planned upgrade to the bank's mainframe-hosted CA-7 workload automation engine.
"When they did the back-out, a major error was made. An inexperienced person cleared the whole queue ... they erased all the scheduling," says the source. "That created a large backlog as all the wiped information had to be re-inputted to the system and reprocessed. A complicated legacy mainframe system at RBS and a team inexperienced in its quirks made the problem harder to fix."
Earlier this year the bank moved to recruit CA-7 batch support staff in Hyderabad on a salary of £9000-£11,000 per annum while slashing higher paid equivalents in the UK.
If substantiated, the revelations could prove damaging for the bank's embattled boss Stephen Hester, who on Tuesday denied claims that the breakdown was in any way connected to underinvestment in IT and the offshoring of jobs to India.
In an interview with Sky News that touched on the outsourcing rumours, Hester said: "Well I have no evidence of that. The IT centre - our main centre, we're standing outside here in Edinburgh, [is] nothing to do with overseas. Our UK backbone has seen substantial investment."
Analysts at Ovum point to the problems faced by large banks in running mission-critical processes on ageing legacy systems with a diminishing pool of skilled and experienced staff.
"With most banks under heavy cost pressures, this means relatively junior staff are often given responsibility for systems where they have little experience beyond the routine, particularly in a stress situation (as with RBS) where things go outside normal operations," states Ovum. "The challenge for banks is that an operation (e.g. the software update to CA-7) that would score relatively low on the risk register escalated rapidly into a critical error. It illustrates that conventional cost versus risk decisions can be problematic, particularly for mission-critical (and their supporting) systems."
Separately, the FT is reporting that RBS is considering legal action against Computer Associates, the supplier of the CA-7 software. The board of the bank spent much of Tuesday in crisis talks, says the FT, after which it decided to launch a full-scale review.
The FT quotes an insider at the meeting: "It was certainly an issue with this software. We will still have to establish if this was their fault or if it was our handling of the software."
These are choppy waters for the bailed-out bank group, with politicians also demanding answers. The chairman of the Treasury Select Committee, Andrew Tyrie MP, has written to RBS chief Hester, and FSA chairman, Lord Turne, calling for action. "We need to know exactly what went wrong and what will be done to give us confidence that similar mistakes are not repeated in the future," says Tyrie. "Basic bank functions are crucial in a modern economy. It is deeply concerning that they appear to have been put at risk again."
Inexperienced RBS operative's blunder led to bank meltdown - The Register