The Hidden Risk of a Meltdown in the Cloud

[DeltaV]

The Hidden Risk of a Meltdown in the Cloud

A growing number of complexity theorists are beginning to recognise this problem. The growing consensus is that bizarre and unpredictable behaviour often emerges in systems made up of "networks of networks".

An obvious example is the flash crashes that now plague many financial markets in which prices plummet dramatically for no apparent reason. Understanding how and why this happens is the focus of much research.

Given that cloud is clearly clearly becoming a network of networks that is rapidly growing in complexity, it's not hard to imagine that the computing equivalent of flash crashes are not just likely but inevitable.

Icebergs in the Clouds: the Other Risks of Cloud Computing

[Diogenes]

The Hidden Risk of a Meltdown in the Cloud

A growing number of complexity theorists are beginning to recognise this problem. The growing consensus is that bizarre and unpredictable behaviour often emerges in systems made up of "networks of networks".

An obvious example is the flash crashes that now plague many financial markets in which prices plummet dramatically for no apparent reason. Understanding how and why this happens is the focus of much research.

Given that cloud is clearly clearly becoming a network of networks that is rapidly growing in complexity, it's not hard to imagine that the computing equivalent of flash crashes are not just likely but inevitable.

Icebergs in the Clouds: the Other Risks of Cloud Computing

And yet people can't seem to consider the same complex dynamics in social networks i.e. "societies."

[ScottL]

While I'm pretty anti-cloud migration due to my own perceived security concerns, I don't see all the arguments in the article as being issues. The idea of networks in networks is very misleading or misunderstood, depending on your view. The idea of the cloud is 1 conglomerate decentralized network providing services (data, web, etc.). The issues with such networks is that they have to retain an overhead for routing. As for a cascading or "flash" meltdown, the network can be splintered to stop this activity with appropriate preventive measures. I still don't trust them though.

[DeltaV]

There are clearly psychological factors involved in financial markets (such as perceptions of "danger" or "threat"), high-speed automated trading considerations aside.

For a cloud "meltdown" to occur, I'm guessing there would also have to be a significant psychological contribution (maybe a perceived supervirus threat), causing network admins to "Dive! Dive! Dive!".

Or, conversely, lulling them into a false sense of security, while "Son of Stuxnet" does its thing.

[choff]

I'll never forget the time VLAN 1 was deleted on this one network removing the spanning tree. The admin said one moment he looked at the rack and all was as usual, the next moment he looked and all the lights were solid green, total lockup.

They put all this redundancy into networks and the complexity goes way up, but then management lays off half the staff and things don't get done.

[palladin9479]

Business's need to stop treating IT people as a liability. They are there for a reason, and you won't notice the lack of their presence until the absolute worst moment.

Honestly I really hate the word "cloud", it means absolutely nothing. It's a marketing term used instead of IT outsourcing, which is what it really is. There is no actual definition of what "Cloud computing" is, no technical requirement or standard, absolutely nothing. You can take two machines, install a web-server onto each of them and have them replicate to each other and you just created a "cloud".

Web applications have been around for years, email longer. Web based file storage has been around since the 90's. There is absolutely nothing new about "cloud computing", it's been done since the late 70's with mainframes. The only difference is a new buzzword and the buyer of the service not knowing where their data is. If they knew where the data was then it becomes remote data storage, otherwise known as data warehousing.

[choff]

Algorhyme

I think that I shall never see
a graph more lovely than a tree.
A tree whose crucial property
is loop-free connectivity.
A tree that must be sure to span
so packet can reach every LAN.
First, the root must be selected.
By ID, it is elected.
Least-cost paths from root are traced.
In the tree, these paths are placed.
A mesh is made by folks like me,
then bridges find a spanning tree.

Radia Perlman

[krenshala]

Business's need to stop treating IT people as a liability. They are there for a reason, and you won't notice the lack of their presence until the absolute worst moment.

Thats the fun of IT. If we are doing our jobs properly, you won't even notice we are there unless you look for us specifically. Unfortunately, this leads lots of folks that manage money for various companies to think IT is unnecessary ("Why are we keeping all of them? I never see them do anything!") ... so they cut half the staff and replace half the rest with minimum wage types that barely understand what they are trying to maintain.* Then, of course, they complain nothing works any more or the IT folks are incompetent because things are constantly breaking down. Its so nice working in a company where everyone is "IT", and understands at least in general how it all works; don't have to deal with most of that now.

As for the "Cloud", everyone in IT knows the Cloud is that part of the diagram that represents "the rest of the internet". So, Cloud Computing is moving your stuff "out there ... on the interwebs somewhere". And yeah, I have some of the same reservations about using it for some things. For others it makes sense, however.

*Yeah, the last 20 years and all the "Learn to be an IT professional, today!" types have me a bit jaded ... in my experience MSCE == Minesweeper Champion and Solitaire Expert, not a skilled Windows Administrator. It still annoys me that one guy was an MCSE that didn't know how to map a network drive *grrrrr*

[ScottL]

Honestly I really hate the word "cloud", it means absolutely nothing. It's a marketing term used instead of IT outsourcing, which is what it really is. There is no actual definition of what "Cloud computing" is, no technical requirement or standard, absolutely nothing. You can take two machines, install a web-server onto each of them and have them replicate to each other and you just created a "cloud".

That's called "clustering" and is standard practice for any high-traffic site. While you are correct on there being no standard, there does exist a definition. The "cloud" is the idea of utilizing existing computer resources in a meaningful and efficient way. It's all about utilizing idle cycles and memory.

Think of it this way, you're paying the bills to keep a web server on 24/7, but the site is only receiving maybe 1000 hits per day. That means, at any given second, its statistically likely that no hit is occuring, and so your server is just sitting there. Now instead of just sitting there, what if your server could recognize that no requests are being processed and do another meaningful task your company may need. Instead of purchasing new physical servers, this new task could be put into a local "cloud" to be spread across your server room or data center. The service could be executed by any server within, because there is always several idling.

[krenshala]

Honestly I really hate the word "cloud", it means absolutely nothing. It's a marketing term used instead of IT outsourcing, which is what it really is. There is no actual definition of what "Cloud computing" is, no technical requirement or standard, absolutely nothing. You can take two machines, install a web-server onto each of them and have them replicate to each other and you just created a "cloud".

That's called "clustering" and is standard practice for any high-traffic site. While you are correct on there being no standard, there does exist a definition. The "cloud" is the idea of utilizing existing computer resources in a meaningful and efficient way. It's all about utilizing idle cycles and memory.

Think of it this way, you're paying the bills to keep a web server on 24/7, but the site is only receiving maybe 1000 hits per day. That means, at any given second, its statistically likely that no hit is occuring, and so your server is just sitting there. Now instead of just sitting there, what if your server could recognize that no requests are being processed and do another meaningful task your company may need. Instead of purchasing new physical servers, this new task could be put into a local "cloud" to be spread across your server room or data center. The service could be executed by any server within, because there is always several idling.

This is exactly why so many are switching from individual servers to one server, or a clusters, running virtual machines (VMs). If you pack half a dozen (or more) VM images onto a single server (or cluster), they would keep the server much more efficient, usage wise, as you would only have idle time when none of the VM were doing anything. Of course, if they all try to do stuff at once, you may be better off with individual servers (but not always).

[ScottL]

Clustered VMs have their place and then bandwidth becomes an issue. My problem with cloud computing is the security aspect. I don't trust a mechanism that distributes my services or data across servers around the world. If I provide a major service, what stops anyone in any country ouside of the publishing origin from intercepting and/or injecting their own data stream within my service? Furthermore, if I'm providing data, encrypted or not, several countries require the PGP keys be willfully handed over to authorities upon request. So now either my data is subject to their laws or my service is susceptible to tampering.

[krenshala]

Clustered VMs have their place and then bandwidth becomes an issue. My problem with cloud computing is the security aspect. I don't trust a mechanism that distributes my services or data across servers around the world. If I provide a major service, what stops anyone in any country ouside of the publishing origin from intercepting and/or injecting their own data stream within my service? Furthermore, if I'm providing data, encrypted or not, several countries require the PGP keys be willfully handed over to authorities upon request. So now either my data is subject to their laws or my service is susceptible to tampering.

Yeah, I agree. I just shake my head when I see the commercials for "Store your data in the Cloud, so its Secure!." According to "them", giving my information to someone else makes it more secure than keeping it in-house.

Now, Cloud computing is useful in that if the environment is set up correctly your data/VM can be transparently moved to another system/cluster if resources or hardware become a problem. This can even be done while the VM is in use (depending on the exact issues going on). Like everything else, it just isn't the be-all, end-all of how to do things in the networking world.

[Netmaker]

Multiple times in the past the entire Internet (or extremely large swathes of it) have been brought down either by fat fingers or ignorance when an ISP has "blackholed" the Internet.

Effectively what occurred is that the operator broadcast that they had the shortest path to every route in the Internet. Consequently all Internet traffic was sent to that operator. Much hilarity and hair pulling then ensued.

The future issue is that nobody will truly understand what the Internet is and consequently not even be in a position to devise "appropriate preventive measures" because there will be no way to know what to prevent against.

The transition to IPv6 may provide for some excellent opportunities to prove the point as we work through exactly what the failure modes are in practice as opposed to theory.

A Cisco or Juniper core router code update also provides similar opportunities.

There's an old adage that it's impossible to make a system that's foolproof because fools are so ingenious.

While I'm pretty anti-cloud migration due to my own perceived security concerns, I don't see all the arguments in the article as being issues. The idea of networks in networks is very misleading or misunderstood, depending on your view. The idea of the cloud is 1 conglomerate decentralized network providing services (data, web, etc.). The issues with such networks is that they have to retain an overhead for routing. As for a cascading or "flash" meltdown, the network can be splintered to stop this activity with appropriate preventive measures. I still don't trust them though.

[kunkmiester]

Most of the hype about "security" is about access, IIRC. The "cloud" allows access anywhere, and redundancy means you're less likely to loose data to hardware failure.

[Netmaker]

The "cloud" is the idea of utilizing existing computer resources in a meaningful and efficient way. It's all about utilizing idle cycles and memory.

Somehow I don't believe that the marketeers pushing the concept of "The Cloud" would include Folding@Home or Seti@Home in their definition. Nor would they particularly want you to think of the possibility of using your own under-utilized equipment as a basis for creating a local "cloud".

A more appropriate analog to the "cloud" is the old fashioned time-sharing services. I say this because the marketeers are the ones who have created and are pushing the meme of the "cloud" and while everybody is slapping the "cloud" label on their product and attempting to sell it as such (much as MS did with .Net) ultimately the major marketeers (Amazon, Google, Microsoft et al) will drown the rest out as far as defining what the term means.