The cloud, in general, is absolutely brilliant. It’s amazing how quickly you can put together some complex, scalable, and reliable solutions within a short period of time with very little upfront investment. It’s inevitable all of us are going to start using one or other services from our preferred cloud providers like Microsoft Azure, AWS, Google App Engine, etc. moving forward.
[image courtesy: National Nuclear Security Administration / Nevada Site Office]
We have been heavily using Microsoft Azure for the past 12 months or so and we are in the process of bringing in some new products that utilize some cool technologies like SQL Elastic Pool, EventHubs, App Services, Web Jobs, plus a few more.
Cloud Migration Challenges
Here are some of the problems we faced especially on pricing and not having control.
1. The wrong type of service provisioned
For the INTEGRATE 2016 event, I asked my team to provision a demo environment to showcase what’s new coming in BizTalk360 v8.1. Everything went super cool and after a month while glancing through the credit card bills I noticed our Azure charges for that month on a particular subscription was over £2000. After digging in, we figured out the developer who created the BizTalk virtual machine, accidentally provisioned a BizTalk Enterprise image running at a cost of some £2k/month. This can easily happen and can result in an absolute disaster costing £1000’s if not monitored correctly.
2. Unattended subscriptions
As part of some of the projects we are working on we have multiple subscriptions for QA, Staging, Production, etc. We recently came across an account which was never used for months and quietly incurring a charge of around £600/month.
3. Scared to kill a running resource
We have seen this instance as well, we don’t know whether a specific resource (VM, Database, Storage, Queue, etc.) is really used or not? So we keep incurring the charges or otherwise ,we have to have the guts to kill it. This just remind me of a story when I was working for a big financial organization, there used to be a desktop PC underneath my desk and someone gave me the instruction “don’t ever turn it off”, we guess some important jobs are running on that machine!! I never turned that machine off during my 5 years with that organization.
4. Cloud providers trap you in with more dependent services
Cross-selling in businesses is not new, organizations especially the e-commerce ones like Amazon will cross-sell you a bunch of other things which you may or may not want. You buy a flip chart, they will cross-sell you pens and additional flip chart books. In the cloud world, it’s becoming more and more common, you start with something small and soon realize you use 5 other services to do the job.
We started using SQL Elastic pool and one of the features that makes it really interesting is the ability to run SQL scripts against all the databases in the pool (Elastic jobs). It looks like a pretty simple requirement, but in order to use this feature, you will require Azure Cloud Services, Worker Role VMs, Service Bus Standard Subscription, Standard SQL database, and storage. So even before you realize it, your costs start to bubble up very quickly. We took a management decision not to use “Elastic Jobs” even though we use SQL Elastic pool.
5. Cloud providers trap you with granular pricing metric
With the world moving more towards PasS (platform as a service) based platforms, the cloud providers and are strategizing pricing more granular. It’s good in one way, you pay for what you use, on the other hand, it introduces the complexity of predicting your cost. Yesterday Azure LogicApps went GA and if you look at the pricing model for LogicApps it’s based on the number of actions you use within the workflow (LogicApp). In the BizTalk world, when a developer is writing a workflow (Orchestration) he or she will only focus on the core business problem, free to use any number of shapes (actions)/logic to solve the business problem. But in this new world, a bad developer or poorly designed workflow will cost you a fortune.
Summary
Moving to the cloud brings a whole new thinking in terms of cost. The wastages will be enormous if you are not cautious.
Now it’s so difficult for you to predict the cost of building and running a solution. Things like developers control the cost of the solution, not knowing you might need a bunch of other dependent services for your solution all makes you a bit worried. The only way to improve your position is by bringing in more concrete processes to your organisation, taking advantage of things like Dev-Test labs, put some processes in place to turn off unused resources in the evenings, weekends, etc, putting some hard limits like using dedicated credit cards with capped limits for QA, Dev subscriptions, allocate a key person for each subscription to watch out for the cost and review them on a monthly basis.
