VMware Photon Platform, Notes from the Field - Entry Two

Time to Build The Cluster Services

Is it a bird,  plane or the Photon Platform? In the early days of Photon Platforms announcements, in my own mind, I had it pegged as a purpose built platform alternative to 'vSphere Integrated Containers (VIC)'. Keeping in mind that VICs sole purpose in life is to enable Containers in vSphere with a cool concept of a 1:1 alignment of VM to Container. This meant a container can have all the flexibility inherent in a container with all the control and security of a VM. Alright come on down Photon Platform just bigger, better more focused right? Well not really....

So one thing that strikes you with the Photon Platform is the flexibility of choices it provides. From a scheduling cluster manager's perspective you can utilise Swarm, Kubernetes or Mesos out of the box. On the other side, this is ESXi after all (marketing slides aside that keep alluding to the slimmed down hypervisor) so it will happily run Virtual Machines  and Containers side by side (well containers within VMs anyway).

Being able to support both is important as not all workloads are created equal in this new age. Let's be honest, not everyone working in this new 'Cloud Native' world has grown a beard, rides a bicycle and wears skinny jeans, there is a variety of personalities that need to be catered for. What is common is we don't all need the assurance and crutches that come from the more traditional platforms. If I fall over it is ok, I will just get myself back up. In the traditional world I would have crutches, cushions, people watching me, ready to catch me if a stumble, etc. This is because in this traditional world I am seen as needed to be treated like a VIP (i.e. Very Important Process) and require all the care of advanced services to keep me going (and the costs associated with those advanced services). Sometimes a more traditional approach may be preferable to containerising (is that a word?) the workload so Virtual Machines live on even if they are seen as cattle (I'll let Duncan Epping define that for you).

This mixed capability is emphasised by the fact that all the management VMs are hosted out of the hypervisors that are flagged as Management hosts (you designate if a host is 'Management' so participates in the control plane or 'Cloud' which only provide a hosting platform capacity, or both). Anyway myself for one, I'm a big fan as by including the ability to host containers and VMs as this provides that flexible, cost and scale focused platform I want without having to look at alternatives such as Openstack. Add to this the fact that it also includes support for the 3 main container and workload cluster scheduling services out of the box and your covering a lot of bases for possible consumers of Photon Platform.

My personal choice is to go forward with Mesos at this stage as it gives me the open choices I want when paired with Marathon. I have played with all three though on Photon as all it takes is that you import the 3 images and enabled the 3 cluster types. I am not here to tell you how to do this or how to create a cluster as there are plenty of blogs that do that, I just want to give my view and some lessons I learnt on the way that may help someone out there. 

It is important to note that when creating clusters, they align to there traditional model and not some Hybrid like in VIC. What I mean by that is that if you create a Docker Swarm cluster, the slave nodes will be running the containers in a shared multi-node format. The nodes are utilising PhotonOS as there Operating System but we don't have the 1:1 Container to VM alignment of VIC. This is important to consider as your sizing of those nodes should reflect your needs. There are default 'Flavors' (remember Flavors define the resource configuration profiles) aligned to the different clusters but they can be overwritten at the API, CLI level when creating a cluster.

This system is built as a multi-tenant service from the ground up so when you create workloads they need to be placed within a Project assigned to a Tenant. As you would expect, a Tenant gets an allotment of Resources (vCPUS, memory, Disk Capacity, VMs, etc) which are then subdivided into Resource Tickets (think Gold , Silver Bronze classes or whatever). A Project is then assigned a Resource Ticket which it then further carves out a resource reservation /limit out of. All the way down this logical structure you can add Access Controls via the integration with VMware Lightwave. 

So importantly as a tenant, I can create my own cluster service to host workloads or just create VMs directly based on virtual appliances or virtual disks imported into the system.  I can then control my VM workloads via the CLI and API into Photon Platforms Controller VM as you would of normally via vCenter, or use the cluster manager of choice to control the workloads within them.

As a final statement on the installation of the clusters what I can say is this. I have built all 3 flavours using simple sandbox methods as well as going through the full manual processes (more then once as I like punishment) and there is a highly compelling value proposition here. To be able to create these services as you require them  on a production class platform so simply is huge. There will be challenges with the way it is done now such as how do they stay close to release parity (hence the compelling part of it being Open Source), but this is a new era platform that brings together the traditional on-premise controls to the new cloud native era's service requirements in the one stack. This is great stuff for a version 0.8 release!

Some Notes On Cluster Enablement:

Be Wary Of Resource Stinginess

My nature is to give less then more when allocating resources. This sent my on a dance to the requirements of the various clusters as I did not have enough resources aligned to my Project.  Just be aware of the sizing needs (they vary of course depending on your own configuration requirements) and if you are resource constrained, use custom 'Flavors'. I prefer to bang my head on the wall so got very use to the 'Photon Cluster Create' operation being closely followed by the 'Photon Cluster Delete' operation to delete the failure and started again (side note, it didn't like me trying to recreate deleted clusters with the same name, didn't ping this down but got into the habit of using unique names each attempt) :)

Cluster Creation IP Address Requirements

Within the instructions it is emphasised that DHCP needs to be disabled / not present in the management network which is where the clusters are installed. Ironically, if DHCP is disabled the cluster deployment fails with a 'Unable to Obtain an IP Address' error. To progress forward I re-enabled DHCP in the management network (the one that the Photon Controller is installed into). The more accurate requirement description is to ensure the static IP addresses you provide (i.e. for the 'zookeeper' servers in Mesos and 'etcd' servers in Kubernetes and Swarm) are not also served within the active DHCP scopes.

Those Damn Certificates

Maybe you lucky that your company provides an open network and all traffic is created equal. That is not the case in mine, we trust no-one and inject ourselves into every certificate coming into the network. Any problems with this is mitigated by adding our own certificates into the trusted root but this is difficult for those environments that are created dynamically as these clusters are.

When you create a cluster PhotonOS based VMs are created corresponding to your configurations requirements (i.e. how many slaves, how many provide the interconnect service) and are then configured by templates contained on the Controller. Where the certificates being trusted becomes critical is the fact that the services run as containers within the VMs and as such the images need to be downloaded as part of the initial execution. The end result is the cluster creation operations fail with 'Time Exceeded' errors.

Anyway its an easy fix! I edited the template files contained in the controller directory '/usr/lib/esxcloud/deployer/scripts/clusters' with descriptive names such as 'kubernetes-master-user-data-template' to inject our certificates into the VMs. This at least made that problem go away and you of course could do this to apply any other customisations that may be required. It would be nice to see some option in the future to inject certificates into the process. I also provided this same feedback for 'vSphere Integrated Containers' as had the same problem there!

VMware Photon Platform, Notes from the Field - Entry One

First, some overview ramblings:

Ok, first of all let me say this, I am excited about the changing landscape in the industry. This shift to containerisation is exciting to me in two ways. Firstly, as an automation focused engineer, this just seems like the natural progression / evolution of the current platform (incoming alert, Unikernels). Secondly, the technology landscape is very exciting, lots of new tools (or is that toys) to play with!

That said I have been fortunate to be working with VMware (shout out to Roman Tarnavski @romant) for a while now on their rapidly evolving 'Cloud Native' program. With this a couple of recent announcements have there two primary offerings 'vSphere Integrated Containers' and 'Photon Platform' are both now Opensource on github so now everyone can have a go. A key point of differentiation between the two is that VIC leverages your existing VMware vSphere platform to host container centric workloads on a 1:1 basis on a Virtual Machine, the other, Photon Controller still leverages ESXi (today anyway) but has it's own control plane.

Photon Controller is a replacement to vCenter focused on the operational requirements of the container workloads, not virtual machines. Why you may ask? An easy way to think about it is the rate of change and scale that us usually associated with a container aligned environment versus the long term service life and different scale models associated with traditional Virtual Machines drives a different model. 

If you are all in for containers, or alternatively, have a substantial container footprint today (or will soon) you have some considerations when assessing VIC versus Photon Platform:

• Will you be limited by the sizing maximums of vSphere. Think vCenter, is 10,000 VMs enough when 1 VM equals 1 container how about the change rate of those VMs.
• You also may be questioning the commercials around vSphere in a container world, do you really need those advanced availability services (e.g. HA/DRS) in a container world. I would say no, the idea of containers is pretty simple, let them scale out and have availability controls north of the service such as with NLB (I know it is an over simplification) or south with stateful data services (should be scale out themselves but that is for another day).

Neither of these statements cover the why VMware anyway? Isn't this new world order a shift away from the vendors we typically associated with infrastructure, doesn't the consumer in this new world not care about infrastructure? All I would say there is that there is a world of difference between developing in a sandbox environment and having your workload run in a production class platform with all the controls, monitoring and reporting thats in place within organisations today for traditional workloads. I also do a lot of testing in the Cloud and directly on my laptop, tat is the beauty of containers, you can just shift around. But for it to go production, there are certain requirements that a lot of organisations need to comply to. This may be due to regulatory demands, data sovereignty requirements, cost controls, to name a few.  

The idea of either offering is to be able to provide the application developers, operator, <insert whoever else here> the same user experience, with the same toolsets but aimed at a different platform. This platform then is able to have the same operators (VMware Admins) look after it in the same / similar way that they do there vSphere environment. Everyone wins :)

Anyway, next I will look at the install and configuration stage. There is some great content at github and by other bloggers such as William Lam. But I am finding some differences with my own experiences to what others have documented. 

Some quick notes:

Controller Install VIB Upload fail

I kept getting file upload errors when the installer tried to upload the Photon VIB to the Management ESX host (see these issues in the log /var/log/esxcloud/deployer/deployer.log) . I still have not isolated where the issue is and not been able to isolate it as yet in the source code. What I did do though is simplified my naming convections and the issue went away. I noticed all the blog samples where pushing into default bare ESX hosts where I aligned mine to my own pseudo naming standards. In the end I simplified my environment by taking hyphens out of my port group and datastore naming and was now able to install. 

YAML Install File with CLI

The examples provided and the exported file provide the 'image_datastores' field within the 'deployment' section as an array. I ended up getting errors around the conversion of the field to an array when attempting to deploy through the cli. This occurred regardless if I had or or more values assigned. As such changed the field to a straight string and it worked. Change shown below:

Uploading Images getting HTTP 413 Error
To enable management clusters (schedulers) you need to upload the corresponding dish images into the Controller (Mesos, Kubernetes or Swarm). I found that regardless of the image I was getting a NGINX error of 413 'Request Entity To Large'. You don't have to be a google ninja to identify the error quickly with the reported resolution to set the 'client_max_body_size' value in the 'nginx.conf' file.On the controller VM the NGINX service runs as docker container called 'ManagementUi' so it was easy to then pull the configuration file down with a 'docker cp' command to have a look, low and behold the setting is not there!

OK, now I have just a rat down he drainpipe I can say that I was on the wrong track. NGINX looks after the User Interface but not the API. Looking at the docker containers there is one running HAPROXY which is the loadbalancer service, called you guessed it 'LoadBalancer'. Jumping into this VM and looking att he file 'haproxy.cfg' you can see that the API is served by ports 28080 and 9000.  By switching the photon cli target to either these ports let the images upload successfully (e.g. photon target set http://10.63.251.150:9000).

I took the long way but got there in the end, time to start playing!!!

Docker-machine and VMware vSphere Clusters, Lessons Learned

I have been doing a bit of work with various flavours of containers lately which comes with the highs and lows of being a technologist. One thing that was interesting is that with the recent updates to docker-machine, my setup for binding and deploying to our vSphere environment no longer worked.

This issue presented itself when attempting to provision a new instance with the following error:

Running pre-create checks...

Error with pre-create check: "default host resolves to multiple instances, please specify"

Digging deeper and I found out that where I was using the cli argument / variable 'VSPHERE_COMPUTE_IP' to specify a ESXi host to bind to, you know need to use 'VSPHERE_HOSTSYSTEM' (or as a cli argument '--vmwarevsphere_hostsystem'). As this is new documentation is fairly light as are real examples. What the documentation does provide though is two syntax examples as follows:

for cluster -                      VSPHERE_HOSTSYSTEM=<Cluster Name>/
for stand-alone host         VSPHERE_HOSTSYSTEM=<Cluster Name>/*

Using those two syntax examples in the lab I first attempted 'VSPHERE_HOSTSYSTEM="UCS8#General Use/' but this resulted in the same error message. It was then also notable that examples were pushing to single node clusters, the test cluster I was targeting has 4 nodes.

Anyway to cut straight to it, the documentation is a little incomplete. You do need to still target a specific host in the cluster and docker-machine will not do that for you. As such in a cluster with multiple hosts the actual syntax is:

VSPHERE_HOSTSYSTEM=<Cluster Name>/<Host Name>

 so for my setup it looks like this

VSPHERE_HOSTSYSTEM="UCS8#General Use/esxrack01.vce.asc" 

I guess while I am here another hint does not hurt. The other piece of lovely syntax requirements is the optional variable for targeting a specific ResourcePool, to do this you use the variable VSPHERE_POOL. The syntax for this is

VSPHERE_POOL="/<DataCentre>/host/<Cluster>/Resources/<Resource Pool>/..."

So again in our lab setup the variable looks like this:

VSPHERE_POOL="/ASC/host/UCS8#General Use/Resources/Containers" 

Docker-Machine and vSphere together are a great team and well worth a play so hope this helps. 

I am also fortunate with doing a bit of work with VMware's vSphere Integrated Containers (VIC) which takes these issues away and provides a very strong alternative to other scheduling/clustering services such as DOCKER SWARM within your vSphere environment but ore on that in another post later.

What is VMware VSAN 6.2 Erasure Coding?

I have been asked what is the 'Erasure Coding' support in VSAN all about as it gets referenced in the same sentences as the new support for RAID5 and RAID6 configurations (only available in all flash configurations).

VMware is stating Erasure Coding as a term to reference for any data partitioning scheme that allows data to be fragmented but recoverable in case of failure even if some parts  are missing (use of parity). In general reference they are referring to the support of Erasure Coding to represent the fact that they have moved from a mirroring scheme to support both RAID 5 (support for 3+1) and RAID 6 (Support for 4+2) to increase the level of usable data from RAW Capacity across nodes.  

Prior releases only supported a 1+1+witness or worst (1+n where n=failure tolerance but each n was a mirror copy) scheme hence you always needed a minimum 3 nodes to start with but only really got 50% usable capacity at best. This is why the Erasure Code support is getting such big focus as it makes VSAN more efficient with physical resources.

Worth noting that Erasure Coding is not referring to a disk configuration as with RAID but how the fragments are distributed between hosts. It is for this reason that depending on the choice mode also drives the minimum number of hosts required (3 for standard setup, 4 for R5 and 6 for R6).

So Who Does Use Linked vCenters?

OK, I am sure that most onboard and I put up my hand, I personally am a fan. When I talk to a lot of clients though, generally I find that most aren't.

Two linked vCenters in Web Client

In the old days we linked the instances together with the linked mode utility (or unlinked it with the same tool). Nice and easy and all of a sudden security and licensing information is shared between vCenter instances. The big gain though was that through the C# client (oh I do miss your usefulness sometimes in this depleting role world) I had a one stop shop to access up to 12 vCenters. Now it is even easier, just have the vCenters in a common Platform Service Controller (PSC) SSO implementation (common site/ multi site doesn't matter).

This in turn enables some very cool features beyond the original benefits. The one that drive me to link mine up was the ability to enable cross vcenter migrations. Being someone that is fortunate to have access to a well equiped lab I have enabled this with VPLEX for the stretched storage and use both SRM 6.1 and vMotion in vCenter to control the migrations (more on that in another blog post I think). 

This blog is not to tell you how to link, that is covered in a lot of other blogs written by very smart people. More I want to draw attention to issues that surface a lot with the newer Enhanced Link Mode with those add-on services that use the web client.

As I have just experienced an issue again I am writing this while it is fresh in the mind. What I see is that with a lot of product teams that do include Web Client plug-ins, there seems to be quite an absence of testing vCenters in Enhanced Linked Mode. Have you ever had this style of conversation:

me)              I am <inset problem description here> in the web client 

developer)   what version are you running

me)              The one listed in requirements

developer)   Are you using the Windows Version of vCenter or the Appliance?

me)              I am using the (whatever) in linked mode

developer)   ohh, we haven't tested that

Anyway some gotchas I seem to experience include:

• Only one vCenter is supported in the linked set for the plug-in
• Only the first vCenter is visible to the plug-in
• Plug-in is unable to differentiate (filter) objects out that are only present within the aligned vCenter (e.g. all datastores across all vCenters are displayed)

So the point of this entry beyond the rant is just to bring attention to the fact that there are some considerations when writing a plug-in for linked vCenters. If you are running into issues and you have got linked vCenters I would suggest that the first step should be test against a stand-alone vCenter. This helps isolate the issue and can help when you are talking to the support/product/vendor teams.

Welcome VxRail, VCE's Entry into the HCIA Market

You want the ease of a Hyperconverged appliance but still have all those advanced capabilities that service enterprise data centres, then the new VCE VxRail is for! This is not so much an evolution of the VMware EVO:Rail appliances but more a revolution.

Why do I say that? The Achilles heal of EVO:Rail was the lack of flexibility, all you could do was chose between two RAM configurations (no variant in disk or CPU), 128GB or 196GB which for some reason had the 196GB model referred to as the desktop service model... never quite got my head around that one, why wouldn't other services want a larger memory footprint? The other side was the fact that you could only grow your appliance service out by adding additional appliances, start with a 4 node 2U appliance and then grow in increments of 4 from there till you hit the 64 node ceiling (aligned to the maximum size for a HA / VSAN cluster).

Now think the VCE VxRail appliance here we have flexibility that can meet a huge range of use cases, your choices now include:

• Starting point is now 3 nodes (today) and growth can then be in single node increments up to the 64 node maximum
• CPU model options provide a range of 6 cores (single socket) through to 28 cores (dual socket) per node
• Memory configurations from 64GB through to 512GB
• Disk configurations of Hybrid (Flash + Spinning HDD) or all flash ranging from 3.6TB through to 19TB per node 
• Even networking options will support 1G connections on the low end nodes with the standard being dual 10G 

You can also now mix different model configurations and EVO:Rail appliances under the same management. Phew... all that flexibility but with the ease of use that comes with the VxRail appliance.

Some other cool features of note are things such as:

• All the coolness of VSAN 6.2 with Dedupe, compression, flash optimisation, erasure coding, snapshots, cross site HA (yes you can link two remote appliances into a vSphere Metro Cluster)
• VM Level replication with EMC RecoverPoint/VM (yep included) 
• Cloud based storage tiering EMC Cloud Array (yep also included)
• VCE Vision software for service health and interoperabilty compliance monitoring and reporting
• vRealize LogInsight for lag aggregation and analytics
• vSphere Data Protection (EMC Avamar)
• Appliance Market Place, your VxRail App Store for easy request / enablement of additional services

And that is not the end of it, too much to list :)

If you want to have a look for yourself have a look at this demo of the VxRail Manager I recorded!

More information is available at http://www.vce.com/products/hyper-converged/vxrail

Phew, vForum Day One Done!

Well just a quick one as thought it was time for a new entry and having just done day one at vForum at Luna Park it was a good chance for an update. I had the great oppurtunity to present in the EMC session today with Matt Zwolenski, our ANZ SE leader on some of our cool software based solutions.

This 40 minute section covered 4 demos (maybe to many for 40 minutes) that we produced out of the local solution center. I think it could be a week before my sleep catches up for that one. It is interesting just how much time filling a 40 minute segment can take up in preparation. This is amplified by my belief that if I am going to show it, I need to build it and prove it first. All demos I show I also buil and produce myself. Maybe that explains the dodgy Camtasia call-outs.

That said we showed:

• The ViPR Data services
• This is very relevant for VMware as it is now providing the Object Storage Services and the snpashot repository for the Database-as-a-Service offerings in vCloud Air
• VVols with the EMC VNXe
• A demo showing the 4 parts of implementing Virtial Volumes 1) Protocol Endpoint 2) Storage Provider, 3) Storage Containers 4) Storage Policies
• Recoverpoint for Virtual Machines
• Very cool replication technology now available at a VM granular level
• The VMware + EMC Hybrid Cloud
• Showed the vRealize suite servicing Virtual Machines,catalogues (vRealize Automation), VM mobility (vCloud Connector), Cost transparency (vRealize Business),  Storage-as-a-Service (vRealize Automation + ViPR)

As a techie I really do enjoy building this stuff and the technology from both VMware and EMC is very cool. It is hard to put a prep time on the work to produce the demonstrations but it was stretched over weeks. With 3 of the 4 solutions shown, done with release levels still in beta or earlier it did bring on some interesting challenges.

It does go a long way to show how far these technologies have come. The old days of any service management solution being an endless drag on professional services are definitely starting to move behind us.

Anyway day one done, one speaking slot and a live vBronwbag podcast completed. Once the vForum roadshow completes in ANZ I will publish the 4 demos into Youtube.

 

Thin Provisioning with VAAI and VASA in vSphere Working Together

That Damn Pesky Thin Provisioned Threshold Alarm in vCenter

Have you ever had those alarms go off in vCenter telling you that your thin LUN has exceeded its consumed space threshold? This is an interesting repercussion of VASA (storage awareness reporting)   feeding its view of the thin provisioned LUN back to vCenter and then it reacting to it.

So first let's have a look at the issue. Thin provisioning in a block storage world tends to start small and then continue to grow until the configured upper limit is reached. Makes sense and it is what you would expect, as data matures the efficiencies are reduced with Thin Provisioning. In the wonderful world of virtualisation where VMs tend to be fairly fluid due to provisioning, deletion, moving activities a LUN can be full one moment and half empty the next. Add vSphere Storage Clusters and SDRS and it does not even need manual intervention.

Anyway back to the problem at hand, the thin datastore all of a sudden looks full so you shuffle stuff around and clean up space. All good at the datastore level but you are still getting alarms off the datastore, specifically the 'Thin-provisioned volume capacity threshold exceeded' alarm. The reason behind this is that once a block has been written to the array does not know how it is being utilised so reports it as consumed.

This is what VAAI Unmap is all about, giving the host a way of clearing a previously written to block in a way that the array can act on. The drawback on this is that there VAAI unmap is still not an automated process so needs to be executed through the CLI (esxcli storage vmfs unman <datastore>) or via PowerShell as a manual process.

Be aware though as this only returns space back to the array that has been released at the datastore level such as what would happen if you deleted a VM. There is also the in-guest space issue, a thin provisioned VMDK will also grow in much the same way that a datastore does. You can also return space back that has been deleted to within the guest through tools that need to be run within the guest such as Microsoft's SDELETE (http://technet.microsoft.com/en-us/sysinternals/bb897443.aspx) or VMware's own Guest-Reclaim tool (https://labs.vmware.com/flings/guest-reclaim).

By getting both your in-guest and ESXi host level space reclaim strategy in place you can ensure that VAAI + VASA = Happy House and you are getting the most efficiency out of your storage (at the space level anyway).

EMC and Microsoft Applications Working Together

EMC Storage Integration Suite

I recently recorded a number of demo videos showing off some of the cool capabilities of the EMCs ESI. These videos show simple storage provisioning activities but aligned to applications such as Microsoft Exchange and SharePoint.

ESI is a very cool tool that is available from EMC for FREE (yes you read that correctly) and provides a rich set of capabilities including:

• Simple user console for storage, replication and application management
• Support for Windows, HyperV, vSphere and XenServer
• Powershell library with almost 200 cmdlets
• System Center Orchestrator Integration Pack
• System Center Operations Manager Management Pack
• Hyper-V VSS Provider and associated PowerShell library
• Support for Exchange (up to 2013) including Native and Third party replication (enabled by RecoverPoint)
• Support for Sharepoint and SQL Server

Demos:

Storage provisioning with EMCs Storage Integration Suite

EMC Storage Integration with Exchange using ESI

Exchange Database Provisioning with EMCs ESI

Exchange Database Replication Enabled with EMCs ESI

SharePoint Content Database Provisioning with EMCs ESI

Product Page:
http://www.emc.com/data-center-management/storage-integrator-for-windows-suite.htm

SCVMM 2012R2 Powershell Cmdlet Changes

Damn, SCVMM Not Supported

Funny, but sometimes changes occur with very little warning and SCVMM 2012R2 has had some that I keep tripping over. Within the fantastic EMC Solution Center in Sydney, Australia which I am fortunate to have a great collection of toys (oops, I meant IT equipment) to play with (again... I meant to say develop and test against) I have come across this issue a few times but find it very difficult to dig up information related back to it.

It relates to the extensive and must say, fantastic set of PowerShell cmdlets that come with System Center Virtual Machine Manager. Firstly the initial changes were very obvious, 2008->2012 the cmdlets started to prefix the command with 'SC', An example of this is that Get-VMMServer became Get-SCVMMServer.

This in itself makes absolute sense, you need a unique identifier when your environment could have multiple libraries that may provide the same cmdlet such as happens with a cmdlet like Get-VM which is supported by vSphere, SCVMM and Hyper-V Powershell libraries so no problem there. What did happen though is Microsoft, I assume to soften the impact, did support the old cmdlets via aliases. These aliases were retired with the release for 2012R2 which I also fully understand as you need to move on eventually, so whats the problem?

The issue is that not everyone has moved onto the cmdlets even though they have had a good chunk of time. My guess this may be because they didn't need to change so why bother, until now. Anyway I have been caught out with this issue with products from both Microsoft (e.g. Team Found Server's Lab Center) and VMware (vCloud Automation Center).

So if you find something is not supporting SCVMM 2012R2 but did 2012SP1, this is not a bad place to start looking at when trying to determine why.

Cloud + SDDC, Now it Makes Sense!

So It Is Not Just Fluff!

Funny, for years now I have been I have been talking, presenting and building Cloud based services, in the early days I just did not realise it. It was at VMworld in 2009 that the term Cloud first got passed around in a way that every vendor seemed to be jumping on it and attaching it their product lines. The reality is though was that there was no clear definition of what was a Cloud.

So what is a Cloud Service, firstly it needs the following

• Automation: delivery and lifecycle management needs to be automated which in turn, done properly, reduces risk and increases agility. Delivering a Virtual Machine to a client in 7 days (or even a day) does not represent what a Cloud is meant to provide the customers. This has a preceding activity, processes need to be well defined, no use automating something to be delivered as-a-Service without knowing all the speeds and feeds that relate to it. Lifecycle is also critical, need to ensure that you don't end up with VM sprawl.
• Assurance: There needs to be a way that the administrators and customers are aware of the integrity and performance of the service. This directly feeds into operational health and performance monitoring, chargeback/showback facility and service assignment states. A customer and administrator need to know how well a service is run, how much is consumed and how compliant it is to the SLAs assigned to it.
• Self Service: There needs to be an electronic shop front to the services on offer. A cloud is consumed on demand as is the norm now for the general tablet and smart phone community as provided by Google Play, Apple and Microsoft Stores. This in turn leverages automation to provide the requested service. Can you imaging grabbing an app from your iphones stores and then waiting a month for it to be available :)

So interestingly there have been a number of products available that were either the good friend of the consultants who then were charged with the implementation of them, or just not good enough to be taken seriously when looking at your Data Center, really just pimped up provisioning engines. 

This is where the Software Defined Data Center comes in and all the related services. The SDDC to me is providing two critical functions

1. The enablement of automation
2. The extensibility of the capabilities of existing physical assets, think what VMware has done to your servers with ESX but across the rest of your DC assets including networking and storage 

 Looking at Infrastructure-as-a-Service, the two primary go to solutions are vCloud Automation Center (VCAC) from VMware and the new kid on the block, the Windows Azure Pack (WAP) from Microsoft. Both of which are positioned as platform neutral and providing the shop front to your DC services. The each have good and bad points, I will look to cover them in future posts.  

You also the SDDC enabling solutions coming in thick and fast, think VMware with NSX for networking, EMC with ViPR for storage, Microsofts all Infrastructure embracing System Center supporting automation of bare metal hosts, virtual machines, networks through the virtual networking stack and storage management.

The point is, in the past 1000s of hours in scripting and coding was required to enable a cloud (it kept me busy so I didn't mind myself), now it is becoming reachable with the push towards the SDDC and the maturing and simplification of the service management solutions. 

What makes me think this, well it use to take months to get a service up and running for POC following the fundamentals of the Cloud as listed above. I would also spend a fair bit of time deep in some form of IDE cutting code. 

Now I find myself complaining when it takes me a few days to build one up from scratch or POC in a few days. 

Is it finally the year of the Virtual Desktop?

I still remember it clearly, July 2006 and suddenly from VMware a new acronym was released into the wild 'VDI'. It wasn't that the virtual desktop had not been thought of as yet as there were already a number of options available and the 'offshore developer' and 'test' use cases were already being utilised by some pioneers, it was just that all of a sudden the concept had its own identity. Riding on the wave of success of the new VMware ESX platform (remember GSX Server?) virtualisation was suddenly looking grown-up and the possibilities were causing a notable stir.

At this stage being an Aussie working for an investment bank in London we had already deployed with reasonable success a Microsoft Virtualisation Server (MSVS) based solution to meet the needs of the packaging   authoring and test environments internally (and with 6000+ packages there was a lot of need!). Shifting focus to VMware's ESX 2 for the server development environment seemed like a logical 'grown up' next step to the virtualisation journey. The MSVS environment already enjoyed a limited amount of automation utilising the COM  automation libraries so developing around the VMware SDK was a natural progression. I was lucky enough to work with some pretty cleaver guys (although I would never tell them I thought as much) and at the end between us an automated service was developed that included the provisioning of the ESX Hosts, Windows Server guests and a pretty slick P2V process. All this helped the environment supporting the development environment to grow at a significant pace in various locations around the organisations international sites.

It was around the third quarter of 2006 that an issue with the limited availability of electricity to our London based Head Office was becoming a focus point and ideas were being thrown around on how to resolve it, step forward 'VDI'!  The challenge was raised, could the lessons learnt through virtualising server workloads be applied to good effect to provide a platform to serve desktops and in doing so reduce the power requirements to a standard end users desk?

I wont go on about the pain of the next 6 months but will say that having worked closely with the engineering and development teams of VMware, Wyse and Leostream we rolled a desktop replacement solution into our London, New York and Singapore offices that provided a fully distributed, redundant, 'work anywhere, return home' desktop service built on the newly released VI3 platform. There were a lot of lessons learnt on the way in areas such as the guest build (we deployed the virtual desktop solution on the back of the Windows XP migration), developing an effective management system, managing end-user expectations, training support personal and suddenly realising significant advantages available.

How brilliant a virtual desktop solution is for an organisation, it can provide:

• Workplace flexibility - No longer being constrained to a specific desk, office or even country when accessing your desktop. How good is it to be able to work from home from your own personal machine as if you are in the office.
•  Business Continuity - Say good-bye to those pools of cold machines sitting at a recovery site just in case a site suffers some kind of significant outage. Think how easy a BCM process can be if you are not tied to a specific location and you can leverage high availability of your data centres. 
• High Security - No longer having any risk of loss of data if machines go 'walking' from an office. All content is secured within virtual containers in a secure data centre facility. It is by no small coincidence that a major fan of the technology was our internal security team.
• Improved support model - This is an easy one, you put desktops on the floor you need people to walk the floor to support them, you put your desktops in the data centre and stateless thin clients in the office then you can refocus some of those people on supporting the desktops from a central location reducing the time and effort required. Add to this the ability to fully automate the management and provisioning of the environment and you can realise a pretty slick quality of service.
• The Green Office - Who would have thought it, a technology that brings significant benefits to the corporate office and to the environment at the same time. When the virtual desktop is paired with a thin client device drawing only 7-14 watts of power an office can easily realise a reduction of 100 watts of power per desktop device. In corporate offices this can result in a significant carbon emission reduction not to mention a reduction in the power bill (have you ever asked how much your company pays for electricity?).  The other benefit is the quality of power provided to the floor, with a desktop there is a risk of corruption and loss of data when a sudden loss of power is experienced, this is not the case when using a stateless thin client - are office wide power supply assurance services such as UPS required to cover everything or even anything. What level of savings can this provide to facility fit-outs?

I will not gloss over the journey to a mature virtual desktop solution, as any early adopter can testify, there have been many challenges on the way. Pain was experienced in many areas including:

• Bridging the usability gap between a standard desktop and virtual desktop
• Making the cost model fit 
• Transforming a support model to fit the service
• Experiencing pain points at scale
• Beating the Monday morning 'access storm' blues
• Managing the capacity in way to ensure the best cost versus performance balance

The advantage any adopter in 2011 has though is that there is now a lot of intellectual collateral out in the wild from real world 'lessons learnt' at enterprise level scale. There is also the benefit of the maturing of the desktop brokers and connection protocols on offer from vendors such as Citrix with XenDesktop and VMware with View. In the final stages of the VDI evolution all the focus had swung towards the protocol and the value added features it provided. This will continue to evolve rapidly as will the continued efficiencies in the back-end infrastructure helping the cost models to make solid economical sense (this is a big topic and I will dedicated an upcoming blog entry to this topic alone) along with the ease of acceptance of the offering with the end-users.

I spent a lot of time since 2006 around industry technology round tables and councils hearing everyone talk about there 'proof of concept' and 'pilot' virtual desktop environments with very little moving beyond. The atmosphere of these changed quickly in 2010 with everyone seeming to ramp up and start to take it seriously. Bring on 2011 and I now spend more time talking to clients about virtual desktops then servers and you definitely do finally get the feeling'2011 could well be 'the year of the virtual desktop'.

+++++++++++++++++++++++++++++++++
As a sub note to this in 2014, there are two things that have changed in the last three years

1. As it should have been, VDI is no longer approached as the answer but as a part of an organisations overall End User Computing strategy. This is a good thing as the drive for change is from the business perspective and not from the technology which is what VDI is
2.  Technology to support VDI has advanced hugely which in turn has resulted in dramatic cost reductions while at the same time, improving the overall user experience. This has happened at the transport protocol, server hosting and at the storage layer. I have been lucky enough to spend quite a number of hours with EMCs new all flash array XtremIO within our lab environment and am still blown away by what I have pushed this array to do in performance and storage (thanks to inline dedup) you can get this device to do. Capacity  for 2000 desktops in the space within a rack that normally would be associated with a network switch.... sign me up :)