Australian banks suffer ATM problems after cloud service outage

So Amazon cloud should relocate to "stormless" sites?

You see concentration in one place is always the problem. 

Look at successful examples- Nature: Dispersed fauna & flora. Cellular: Spread of connectivity across areas. 

Outsourcing to clouds does mostly make sense for businesses, but who said clouds shouldn't be thinly & smartly spread ?

Sounds like a lack of Comms resilience - always build network seperacy in bu design. I have seen too many Banks and Mega Merchants hit major outages due to poor comms network design.

*by design. I blame the iOS keyboard and not my fat fingers - honest

Say 20m bank accounts, say a few hundred Kb for each to record last N transactions, a 16Gb memory card could store the whole ledger. Put debit card system on the blockchain so there's nothing in the middle to go down anymore.

This won't stop the digerati from blaming legacy the next time RBS' payment networks go down! On another note, I should add this as a sure-fire way to kill cash.

How To Really Kill Cash 

BlockChain 2.0, the sequel. The Consultants are back and this time they're invoicing... Not convinced on the USB idea - I think they used to do something similar on reel to reel tape. I'd much rather use a distributed data aggregation service to ensure the BlockChain ledger survived the apocolypse unscathed.

So... "a storm caused a major outage at a cloud computing service"?? Surely these banks did not have everything in a single data processing center. Or did they? I can't believe they haven't heard of, or implemented, geo-replication for a critical service like this. If they didn't have replication implemented, blaming the cloud service vendor seems short-sighted.

Having alternate facilities to shift the workload to in case of a site outage is quite helpful, but it is also very helpful to have the required data already sitting there. There have been cases (eg. the recent Salesforce.com outage in the US) where apparently the data had still to be moved over to the alternate site - which can take a few hours. However, taking such precautions will result in additional cost ...

@Gerhard... I fully agree with you that data needs to be already available at the alternate site. I should have probably been more clear and explained I referred to real-time / on-line geo-replication. In my ignorance I don't know if that is supported by AWS, however we use it in our Azure services... and we're not a bank processing critical client transactions and services...! The cost overhead is actually very low as most of the infrastructure in the secondary site does not need to be spun-up until actually needed.

Banks have known about fault-tolerant, clustering, ACTIVE:ACTIVE and other onprem redundancy / failover technologies and, where justified, implemented them ages ago. I remember a midsized Indian bank buying a Tandem mainframe with 100% fault tolerance to run its ATM switch 25 years ago. I doubt if the banking industry needs guidance on how to go about securing its infrastructure.

Point is, after migrating to the cloud, why the heck should banks still have to worry about infra? Every cloud service provider promises to shield end users from infra worries. High time PaaS / IaaS vendors fulfilled their promise or stopped making tall claims like this. Otherwise, whatever little legacy transformation is happening in BFSI will stop.

@Ketharaman... I agree, but I believe a big mistake a lot of people make is that when you port Financial Services to "the cloud", the route being followed is highly secure, locked-down, environments that are IAAS... where the customer - not the cloud vendor - still has to define and manage their redundancy needs. There are significant economic benefits to this still. But I'm not sure it's fair to lay the reponsibility for Business Continuity wholely on the doorstep of the cloud vendor.

@DanielSmith: If these banks were to sue AWS, Amazon would win the case hands down - I'm sure there's adequate fineprint in an AWS contract to put the blame back on the banks. But then, what's the difference between these purportedly CX-driven new-age companies and the purportedly-price-gouging-hidden-cost-laden old-age banks?

@Ketharaman... :-) It's really just a question of perceived flexibility and economics. The fact that you / we can spin-up additional resources very quickly as and when needed, and take the same down again when the workload drops off... all without the traditional Capex.

But - and I think we agree - there's a gap in people's perceptions about who really needs to take responsibility for ensuring business continuity in this virtual world.

Today, in my mind, it is still the customer, and has to be. Tomorrow, maybe a cloud vendor comes up with a solution that allows them to handle the same without creating any data / access security concerns. If they do, and Azure is maybe closer than AWS to this with their PAAS offerings... then they will bring huge value to the table.

@DanielSmith: I agree with you about the crux of the offering. To take an analogy, if my factory suffers a power failure, the only way I can ensure business continuity is by having a DG set that will provide standby power. My argument is about messaging and communications: My utility company upfront tells me to invest in a DG set if I'm expecting to run my product facility 24/7/365. Whereas what I hear from any PaaS / IaaS provider is "leave your infrastructure worries to us". As a technology marketer, I shouldn't complain but, as a technology consumer, I can't help doing so!Therein lies the disconnect:)

Before we jump to the conclusion that it is the Cloud/PaaS environments fault - have we considered that perhaps the Applications were not implemented correctly by the customers? Migrating into a Cloud/PaaS environment has serious implications and systems often need significant redesign to operate as expected in such environments - and enabling leverage of features such as demand elasticity. I still think this is a sloppy comms design issue.

A little bit late to the party here, but Matt makes a valid point, we are all too quick to point the finger.  we do know have any real knowledge of what services/applcation were ported to the public Cloud, we have no knowledge of where the failure occured.  All we know that it affected ATM and ETFPOS devices in some cases and mobile banking on others.  from my experiance these three services do not share any commonality external to the banks systems, but they do interact with and travel over many third party services.  Some services are more easily removed from study, than others.

Root Cause analysis of a fault should not be left to the masses.  it may come to fruition that AWS is to fault, but that does not mean that AWS is to blame, at the end of the day a company is still responsible for their Data and it's access, this cannot be offloaded ot a provider.  Resilance and redundancy are different sides of the same coin and putting a front end customer facing system, coupled with a main method of moving money into a inherently none redundant system is foolish,  AWS's idea of a failure domain is significantly different to what the average layman not working on global Cloud infrastructure would consider a failure domain.  it is very likely that AWS are not even in breach of any SLA's regarding this outage.

To me the old addage of Buyer beware is so apt with Public Cloud services.  if am person comes with a deal that looks too good to be true. it very likely is.

To me, it all boils down to the type of infrastructure the bank or other customer organization chooses to run their application on. For instance, they can decide for running on a very reliable and secure infrastructure as described by Ketharaman further up in this discussion string. However, if they want to run in the cloud and still want the same kind of reliability and security, they need to make sure that their cloud provider uses that same kind of infrastructure. Failing to do so and then trying to blame the cloud provider just makes no sense.

AWS et al do not provide such reliable and secure infrastructure at this time. However, there are payments service providers using such infrastructure and offering their service to banks since decades, long before the term "SaaS" had been invented. A good share of the world's payments are run by such service providers on behalf of banks who do not want to run such systems themselves, and this approach works nicely since many years.

However, assuming that general purpose cloud providers running on plain vanilla gear can deliver the same kind of infrastructure reliability at much lower cost just isn't very realistic. And of course, they can't deliver payments as SaaS either, thus a lot of effort and cost remains with the customer organization. 

I agree completely, my personal view is that Banks are expecting that the likes of AWS and Azure provide same level of service and resilience provided by "legacy" 3rd party payment service provided