Recently I was working on a project that involved VMware App Volumes, otherwise known as AppStacks. The basic premise behind AppStacks is that they facilitate application delivery to virtual desktops. For this particular project, it involved over 100 applications being delivered to nearly 500 desktops across two datacenters.
Now AppStacks are just virtual machine disk files (.VMDKs) that contain one or more shareable applications, the applications in question could be relatively simple applications like Adobe Reader or more complex applications that have several dependencies and licensing requirements. These VMDKs are attached to virtual machines to enable the application to be run on many VMs at once, in essence, AppStacks are shareable, one-to-many, read-only volumes. Using AppStacks enables quick, secure and inexpensive cloud computing and virtualization management, of thousands of applications and virtual machines as if they were one.
The applications themselves become portable objects that can be moved across data centers or the cloud and then shared with thousands of virtual machines quickly and easily.
Now, one of the current challenges around AppStacks is how do you protect them? The VMDK files themselves cannot be easily backed up as Veeam only backs up the virtual machine and any VMDK files attached to that virtual machine. AppStacks are not virtual machines, they are purely VMDK files are temporarily attached to many virtual machines. The VMDK files are attached to any number of VMs at any given time and they only remain attached as long as the user is logged into the machine. To complicate it further, users will usually have a different variety of VMDKs attached based on their needs and role within the business.
So how can I protect my AppStack volumes? Well, if your storage vendor can back up your VMDK files at the datastore level, you can try that, but not all storage vendors allow that kind of backup.
Alternatively, the AppVolume Manager does allow the creation of ‘Storage Groups’ to replicate AppVolumes between multiple datastores. The issue remains that this type of replication is not a good backup, for example, storage failure or if someone were to delete an AppStack then it’s going to delete the AppStack volume on both datastores. The interval for replication is apparently 1h by default.
Another option is to utilise the VMware fling that was recently made available by Chris Halstead,
Dale Carter & Stephane Asselin. The fling connects to both the App Volumes Manager and Virtual Center using API calls to create a backup virtual machine, the underlying VMDK files of selected AppStacks and Writable Volumes are attached to the backup virtual machine. Once the VMDKs are attached to this ‘backup virtual machine’, Veeam can then back up the VMDK files.
Unfortunately, this method requires manually assigning each AppStacks to a backup VM, anytime someone creates a new AppStack you will need to open up the fling and attach it to a backup VM to be protected. We were after a more automated solution and due the very nature of flings, we were hesitant to utilise this in a production environment. Our solution was to copy them using SCP, since AppStacks are read only, corruption should not be an issue so backing them up should be as simple as just copying them to another location.
Veeam did have a standalone product called FastSCP but this has since been integrated into the free version Veeam Backup. By utilising ‘File Copy’ Jobs within Veeam Backup we can copy these AppStack VMDKs using SCP. The ‘File Copy’ Job can be easily configured with a schedule and provides the bonus of traffic compression and empty block removal to reduce copy times and improve performance. Veeam also generates a one-time username and password for each file transfer session. For example, if you have to copy 3 VMDK files, the program will change credentials 3 times.
The last challenge we encountered was how to perform SCP operations from a vSAN datastore. In the end, we used the ‘Storage Group’ capability of the AppVolume Manager to replicate AppStacks to another datastore that wasn’t a vSAN then Veeam performed a ‘File Copy’ which copied the VMDKs to a safe location protected by an existing backup solution. Since our only datastore was a vSAN datastore we configured an ‘OpenFiler’ VM to provide an iSCSI LUN to run up another datastore. Technically it’s a nested datastore but we only care about enabling access to the AppStacks VMDKs so we can copy them.
Do you know of a better way to protect AppStack volumes? Let me know in the comments section.
Disclaimer: AppVolumes v2.1 was the version used, v3.0 has since been release and may solve some of these problems.