Compare cloud

It is important to evaluate cloud services before purchasing

Intended Audience: CxOs, Other IT decision makers

Coffee with Steve

It was a Friday afternoon in late 2013, when we reached the CEO’s office of ABC corporation (name changed), to share what we have been championing — Cloud Evaluation as a Service, and to learn from their experience as they had recently started using the cloud to increase the capacity of their data center.

Steve (name changed) is the Group CEO of ABC corporation having many verticals. They recently signed a big contract for delivering a year-long corporate training program which required a significantly expanded data center to deliver the program efficiently. Instead of adding new physical capacity to their existing in-premise data center, Steve decided to use public cloud services from a major vendor. 

Alex, an IT manager in Steve’s team, had prepared a detailed proposal using which Steve could make his decision about going to a public cloud vendor. The proposal was based on several important requirements that need to be considered while selecting the public Cloud Service Provider (CSP) for their requirements. Let’s look at some of those requirements.

As many training modules were developed in .NET, they initially thought that one of the major cloud vendors supporting .NET would be the natural home for them. After some research they discovered that cloud services from this vendor were still maturing, remember it was sometime in December, 2013; moreover, this vendor’s cloud support could not respond convincingly to many of their queries. This prompted Alex to consider another major CSP with a huge market share to understand the services they offered. This second vendor, being one of the leaders in cloud service offerings, had solutions for all technical requirements of ABC corporation. The overall cost of computing resources also looked reasonable to Alex. Finally, they considered two other vendors, reviewed their capabilities and attempted to match them to their requirements and business priorities before making a final purchasing decision. 

Based on the internal review and the proposal from Alex, Steve decided to start their journey into the cloud with one of the biggest CSPs in the business. What Steve didn’t realize is that while Alex did a reasonable job comparing CSPs, he also relied on an old adage that said ‘no one gets fired for recommending IBM’s servers’. Alex adopted that adage to cloud and transformed it to say that ‘no one will get fired for recommending one of the leaders of Gartner’s Magic Quadrant for public cloud services’.  ABC corporation expanded its capability through cloud; delivery of training modules started for the major new contract and Alex’s recommendation and efforts were very much appreciated.

Then came a bolt from the blue! The monthly bill for compute resources, which was estimated to be around $1500, was almost touching $2000 — 33% more than expected! This was a major crisis considering upcoming expansions, and Steve started wondering if he did the right thing relying only on Alex’s internal recommendations while selecting the CSP. 

Steve and Alex got into a detailed analysis of the billed items to see what caused the surge and how they can manage these costs better. 

Lessons Learnt

  1. Their analysis showed that the unplanned excess in billing was caused largely by their applications exceeding the allotted IOPS (I/O per sec) thresholds. Their chosen CSP billed its customers for IOPS, even though this is something which is generally not in a customer’s control. Even plain vanilla instances with just the OS installed on it can have significantly large IOPS (possibly triggered by OS updates etc.), thereby causing possible billing surges. 
  2. Since the in-house IT team did not understand the cloud billing model well, they did not shutdown the ‘not-in-use’ instances, incorrectly assuming that ‘not-in-use’ also implies ‘not-to-be-billed’. 
  3. Again, due to lack of experience with cloud storage, they were using the most expensive form of online storage available when most of their needs could have been met by less expensive archival storage.
  4. Steve concluded that he should have used a professional, unbiased evaluation agency to help him select the best CSP for his requirements – one that felt no need to stick to any adage !


An SOS from Adrian

It was a Sunday morning in the summer of 2014. I got an intriguing call from Adrian, the CEO of a successful enterprise. “How do you say that these cloud services are elastic? And here, we have lost data of two days!” — Adrian wanted an explanation for something I did not have any context for.

After calming him down and talking to him for a few minutes, I got the complete picture. Adrian was very upset that business critical data captured by the thousands of sensors across different geographical regions of his business were not being recorded consistently by his application servers. This was absolutely critical for his business analytics service to be useful to his customers. 

The main issue was that their instances were consuming the allotted quota on the Solid State Drive (SSD) storage and the data received on Friday and Saturday evening could not be saved. In fact, one particular week, the SSD limit was reached on Friday evening itself and the issue was discovered only on Saturday night when the automated weekly summary of activities showed several error messages!

On top of it, when they contacted their cloud vendor for additional SSD storage space, they were told to move to a new instance with more SSD space to get a contiguous space; the process took extra time making their downtime longer.

They had made their decision to go with this vendor based on the suggestion from their IT Manager, Samuel who had recommended this particular vendor ONLY for its very simple and easy to understand pricing model. However, not considering many other factors that are critical while making cloud purchasing decisions was making them run into unexpected issues. 

Lessons Learnt

  1. Only one factor was considered before selecting the cloud vendor when other important factors like supportability, elasticity and ease of use should have been considered. 
  2. Notifications and messaging were not setup correctly for exception situations. 
  3. They had incorrectly assumed that sufficient resources had been provisioned for development and production environments and both were deployed within the same cloud configuration.

Summary

While most CxOs understand and follow extensive processes – including evaluation – when it comes to, say, hardware procurement, when it comes to cloud resource procurement, required due diligence is often not done – perhaps because of the thought that it would easy to switch to another cloud if one does not work.

In the case of Steve, they should have understood every aspect of the vendor’s billing process, instead of getting surprised after already committing their investments to that vendor. They should have realized that correcting cloud purchasing decisions can be expensive and its better to get a professional evaluation done to minimize risk.

In case of Adrian, some of their basic assumptions costed them dearly. The amount of storage required is a function of the rate at which the data is captured and the number of days those data should be kept in the online storage can’t be left on some assumption. In this case a cloud consultant or a cloud expert or even a solution architect from the cloud service provider could have helped in better scoping the requirements.

Corollary

  • Steve decided to migrate to another public cloud service provider to rein in operating cost and moved out of their first “cloud home” within one year.
  • Adrian separated out their deployment environment and enabled important monitoring and alert features from day one in the new deployment platform.


Notes: These are real case studies. The names of the persons and the companies have been altered at their request. 

 

Choosing Right IaaS Service Provider for your Enterprise

The choice of cloud infrastructure providers is huge. There are the big providers like Amazon AWS, Microsoft Azure, Softlayer (now part of IBM), Rackspace, Google Compute Engine, CenturyLink, GoGrid (now part of Datapipe), Dimension Data, CSC and many other vendors vying aggressively for the $20 billion market.

Choosing the right cloud service provider for your needs is not an easy task. This article focuses on the aspect of choosing the right Infrastructure as a Service (IaaS) for your organization’s needs.

Having choices is much better than not having choices, however multiple alternatives pose a difficult problem of choosing the right service provider with right features and services at right price. Almost all vendors reduce prices on a regular basis, add new functionality all the time, increase their global spread etc., in a bid to out-manoeuvre each other. Some vendors offer very fine grained compute configurations, reserved instances, spot instances, high IOPS, bare metal infrastructure and some of them even throw in basic platform services along with many other attractive options.

So how can one go about selecting the right cloud service provider for their use case? Many users feel that it is very difficult to evaluate various players and choose the “follow the herd” approach and think it is the safest. Well it may work for some customers by sheer coincidence. However every organization’s requirements-  both near and far term are different, so one cloud fits all may not work in their favor. That brings us to the topic of how one can evaluate all the major public cloud vendors effectively and objectively.

This article discusses how researchers at RightCloudz Technologies solved this complex problem with their unique (patent pending) evaluation engine. The first step in evaluating any product or service is to list out requirements that product or service needs to be met. This is true with respect to evaluating a cloud service provider. Let us take a look at the high level requirements to be met by a cloud service provider that provides infrastructure as a service.

Requirements

To evaluate cloud service providers objectively and in their entirety, the requirements needs to be broken down into its constituent measurable parameters. Let us take a brief look at some of the major requirements and their constituent high level parameters.

Compute: The heart of IaaS is compute. Some of the major aspects needed to be considered while evaluating compute requirement are capabilities that the vendor provides in offering flexible and extensible compute resources like memory, disk, virtual CPUs, operating systems, on demand /reserved instances, range and granularity of compute configurations.

Data protection: Data protection is a very important requirement as all of customer data is outside their premises and on public cloud infrastructure. Hence making sure that their data is safe and secure is important to the cloud vendor too. From an evaluation perspective, it is important to check how good the vendor is in giving fine grained control access, having encryption at various states, version controls mechanisms, providing audits, logs, and in providing backup/restore of customer data.

Infrastructure and integration: Infrastructure is responsible for making life simple for you and your customers. Some of the key aspects that a customer need to measure are: Can the vendor provide choice for deploying applications close to your users or customers, ensuring the lowest possible latency and best user experience? How well is the connectivity within and across regions? It is also important to measure the service features and capabilities that help customers to easily integrate their existing infrastructure with Cloud. These include data export/import, backup, hybrid clouds, VPN, clustering, gateways etc.

Operating cost: For most enterprises, cost saving is the prime business driver for adopting cloud. However it is very important to consider all cloud infrastructure related cost parameters including compute, storage, network, support, data transfer / connectivity, monitoring etc. Most top vendors have similar pricing for instances for similar configurations. However operational costs include transaction and management costs and that can vary a lot from vendor to vendor. And are varying amounts of free transactions that are provided by each vendor.

Performance: Performance across all services is important for a pure IaaS offering. However based on technical use case that the service is user, an enterprise can lay more emphasis on certain performance parameters than others. Some of the common performance related parameters to consider are high speed connectivity, cache management, network scaling, load balancing, self-healing mechanisms, auto scaling, managing fail-overs, disk i/o etc.

Storage: Storage capabilities are a key part of IaaS offerings, and most vendors try to provide flexible and extensible storage resources like very wide range of disk space, elasticity of disk space, built-in redundancy, variety of storage options including SSD, archival storage etc.

Security & Compliance: While security and compliance have always been important criteria for cloud users, multi tenanted infrastructure poses higher risks. Vendors have features and capabilities that provide varying levels of security in the Cloud infrastructure to data, users, connections, applications including physical & electronic security to data centers. Hence it is important to look at all aspects of security in closer detail.

Compliance to regulatory requirements and certifications are equally important factors to choose the right IaaS vendor. Cloud compliance issues arise as soon as cloud user decides to make use of cloud storage or backup services. By moving data from their internal storage to someone else’s they are forced to examine closely how that data will be kept so that you remain compliant with laws and industry regulations.

Application services: Many IaaS vendors have begun including basic platform services to add capabilities that support application development including pre-configured applications. In addition vendors try to get edge over other vendors by providing versatile set of APIs, toolkits, mobile app support, various media services and other platform related services. It is important to measure and evaluate these aspects based on end user needs.

Understanding RankCloudz

RankCloudz is RightCloudz’s patent pending evaluation engine to help enterprises to evaluate and rank top public service providers. This example illustrates how RankCloudz methodology has been used to evaluate cloud vendor services that provide moving in-premise development and test environments to the cloud.

RankCloudz has a large knowledge database that includes very granular raw data on hundreds of parameters for all the top infrastructure cloud service providers. RankCloudz uses this knowledge base and user inputs on priorities for the above requirements and applies normalization techniques on raw data to compare and contrast various service provider offering to come up with a ranked list of service providers dynamically.

The evaluation process looks at each business and technical requirement required for such a business case in great detail, determines vendor services that support those requirements and then evaluates each feature or attribute of those services. RankCloudz has an on-going process to collect fine-grained data about vendor services and features. This up-to-date data helps RankCloudz to rank all vendors fairly and in an unbiased manner at any point in time.

RankCloudz in use – choosing right vendor for development and test

The versatile RankCloudz evaluation engine can be used to evaluate and rank cloud services providers for various cloud usage scenarios.

As an illustration, the paper shows the results from RankCloudz while choosing the right vendor for software development and test environment. The results are graphically represented in the form of a stacked bar chart as shown below.

Compare IaaS Vendors

Each bar represents a Cloud service provider and their scores against each requirement. In the above evaluation, 13 requirements have been used and scores for each requirement is normalized and stacked as shown in the above graph. The top most bar in each bar graph represents scores of how well the “Application Services” requirement is met by each of the cloud service provider. The next bar represents “Compute” requirement and similarly the rest of the eleven requirements are stacked one below the other to form the score for each cloud service provider. In this example, Amazon AWS ranks first followed by Microsoft Azure.

Here is an example of how the evaluation looks like when only Operating Cost of various vendors is considered. You may notice that priority for “Operating Cost” has been set to 10 and rest are set to zero.

Operating cost comparison

Depending on user’s business and technical needs, the various requirements can be set to priority values ranging from 0 to 10 giving a very fine grained control on specifying relative importance of requirements.

Best way to understand how RankCloudz works is to try it yourself by registering free at

https://www.rightcloudz.com/RankCloudz

or write to info@RightCloudz.com for more details.