Tag Archives: VMware

Converting Hyper-V VMs to VMware using Veeam

I recently had the opportunity to migrate a small regional office from Hyper-V to VMware by leveraging Instant VM Recovery of Workloads to VMware vSphere VMs available in Veeam Backup & Replication v10. With this latest Instant VM Recovery, Veeam can instantly recover any Veeam backup created by any Veeam product to a VMware vSphere VM. In other words, physical servers, workstations, virtual machines or cloud instance Veeam backups can now be restored to VMware. Veeam even handles the P2V/V2V conversion automatically with it’s own built-in logic. How good is that!

To get started, a backup was taken of the source Hyper-V VMs prior to the scheduled outage window. During the outage window, the source Hyper-V VMs being migrated are powered down and the Veeam backup job is run again to ensure all changes to the VM disks have been backed up. Once the backup has completed, the source VM should not be powered on again.

To start the migration process, browse to the backups, right-click the VM to be migrated and click ‘Instant VM Recovery’ then ‘VMware…’

At the next screen, ensure the selected restore point matches the time for when the VM was shut down and the last backup ran.

Continue reading

Upgrading the home lab ESXi 6.5 to 6.7

As with most IT enthusiasts, I use a homelab for tinkering, troubleshooting and hopefully a bit of learning. My homelab compute consists of a single server with an i7-3770 CPU / 32GB DDR3 RAM / 1TB SSD connected to an Intel BOXDQ77MK motherboard, basically a glorified PC. While my homelab can’t run dozens of virtual machines concurrently, it’s power-efficient, cheap and importantly quiet.

I built the server back in 2013 and to date, the server has been rock solid. At one point the server even hosted this very blog for over 3 years. The version of ESXi is a bit old and overdue for an upgrade so I thought this would be a good opportunity to document the process for anyone else interested in the ESXi upgrade process. Specifically for my case, I’ll be upgrading from VMware ESXi 6.5.0 (build 5969303) to VMware ESXi 6.7 (build 15160138).

Continue reading

Veeam Replication Job Stuck at 99%

Recently I had the opportunity to deploy Veeam B&R utilising Cloud Connect Replication for a customer to replace their existing DR solution. We were running into an issue with a couple replication jobs that were sitting at 99% for longer than I would expect, in some cases for several hours.

I wasn’t sure what it was doing as there was no network traffic, CPU or even disk usage on the on the source that could be detected. The Veeam job showed no tasks currently underway and  I didn’t want to speak to the Service Provider to check their end until I had verified everything was working as expected at the source so I kept digging.

Continue reading

vExpert 2017 Award

A few days ago VMware announced the second half of 2017 list for vExperts recipients. A vExpert is someone that has demonstrated significant contributions to the VMware community, as well as the desire and willingness to share expertise with others.  Contributing is not always blogging or Twitter as there are many public speakers, book authors, script writers, VMUG leaders, VMTN community moderators and internal champions among this group.

I am stoked to share that I was awarded the vExpert title under the Evangelist path for 2017.

Congratulations to all the other VMware evangelists, partners, and employees that have made the list for this year.

The announcement page can be found here.

Poor Performance & Power Management on VMware

Poor performance experienced by your VMs may be related to processor power management implemented either by ESXi/ESX or by the server hardware.

One real world case I recently encountered with a customer involved VMware Horizon View and large delays experienced by their end users. Applications took unusually long times to open and general performance was quite bad. This was quite apparent when comparing the same applications on a thick client to running it on a virtual desktop.

After running through the usual checks consisting of VMware Health Analyzer, checking for over-subscription and over-utilisation there were no red herrings immediately apparent. What we did discover though (which is detailed in ‘Best Practices for Performance Tuning of Latency-Sensitive Workloads in vSphere VMs’) involved changing a BIOS setting on all of the ESXi hosts. Specifically the setting for power management on the ESXi HP Hosts to “Static High”, that is, no OS-controlled power management.

While we are working through the other recommendations provided in the VMware Health Analyzer report and have already made some changes to the configuration, nothing has resulted in a noticeable improvement with the exception of the power management setting. The customer has reported that after changing this particular power setting it has provided the most significant improvement in performance of anything previously attempted (hardware and software).

 

Progress Controller: [VCSA ERROR] – Progress callback error

Deploying vCenter Server Appliance 6.0 and run into this beauty,

‘Progress Controller: [VCSA ERROR] – Progress callback error’

Turns out the vCenter Server Appliance installer will fail if more than one DNS server is provided. Fantastic…

To workaround, provide one DNS server IP during the installation wizard. Once the VCSA is installed and running you can then provide the secondary DNS IP.

Backing Up VMware AppStacks/App Volumes

Recently I was working on a project that involved VMware App Volumes, otherwise known as AppStacks. The basic premise behind AppStacks is that they facilitate application delivery to virtual desktops. For this particular project, it involved over 100 applications being delivered to nearly 500 desktops across two datacenters.

Now AppStacks are just virtual machine disk files (.VMDKs) that contain one or more shareable applications, the applications in question could be relatively simple applications like Adobe Reader or more complex applications that have several dependencies and licensing requirements. These VMDKs are attached to virtual machines to enable the application to be run on many VMs at once, in essence, AppStacks are shareable, one-to-many, read-only volumes. Using AppStacks enables quick, secure and inexpensive cloud computing and virtualization management, of thousands of applications and virtual machines as if they were one.

The applications themselves become portable objects that can be moved across data centers or the cloud and then shared with thousands of virtual machines quickly and easily.

Now, one of the current challenges around AppStacks is how do you protect them? The VMDK files themselves cannot be easily backed up as Veeam only backs up the virtual machine and any VMDK files attached to that virtual machine. AppStacks are not virtual machines, they are purely VMDK files are temporarily attached to many virtual machines. The VMDK files are attached to any number of VMs at any given time and they only remain attached as long as the user is logged into the machine. To complicate it further, users will usually have a different variety of VMDKs attached based on their needs and role within the business.

So how can I protect my AppStack volumes? Well, if your storage vendor can back up your VMDK files at the datastore level, you can try that, but not all storage vendors allow that kind of backup.

Alternatively, the AppVolume Manager does allow the creation of ‘Storage Groups’ to replicate AppVolumes between multiple datastores. The issue remains that this type of replication is not a good backup, for example, storage failure or if someone were to delete an AppStack then it’s going to delete the AppStack volume on both datastores. The interval for replication is apparently 1h by default.

Another option is to utilise the VMware fling that was recently made available by Chris Halstead,
Dale Carter & Stephane Asselin. The fling connects to both the App Volumes Manager and Virtual Center using API calls to create a backup virtual machine, the underlying VMDK files of selected AppStacks and Writable Volumes are attached to the backup virtual machine. Once the VMDKs are attached to this ‘backup virtual machine’, Veeam can then back up the VMDK files.

Unfortunately, this method requires manually assigning each AppStacks to a backup VM, anytime someone creates a new AppStack you will need to open up the fling and attach it to a backup VM to be protected. We were after a more automated solution and due the very nature of flings, we were hesitant to utilise this in a production environment. Our solution was to copy them using SCP, since AppStacks are read only, corruption should not be an issue so backing them up should be as simple as just copying them to another location.

Veeam did have a standalone product called FastSCP but this has since been integrated into the free version Veeam Backup. By utilising ‘File Copy’ Jobs within Veeam Backup we can copy these AppStack VMDKs using SCP. The ‘File Copy’ Job can be easily configured with a schedule and provides the bonus of traffic compression and empty block removal to reduce copy times and improve performance. Veeam also generates a one-time username and password for each file transfer session. For example, if you have to copy 3 VMDK files, the program will change credentials 3 times.

The last challenge we encountered was how to perform SCP operations from a vSAN datastore. In the end, we used the ‘Storage Group’ capability of the AppVolume Manager to replicate AppStacks to another datastore that wasn’t a vSAN then Veeam performed a ‘File Copy’ which copied the VMDKs to a safe location protected by an existing backup solution. Since our only datastore was a vSAN datastore we configured an ‘OpenFiler’ VM to provide an iSCSI LUN to run up another datastore. Technically it’s a nested datastore but we only care about enabling access to the AppStacks VMDKs so we can copy them.

Do you know of a better way to protect AppStack volumes? Let me know in the comments section.

Disclaimer: AppVolumes v2.1 was the version used, v3.0 has since been release and may solve some of these problems.

vCenter Support Assistant 6.0

I heard about an interesting tool today for vSphere that can collect diagnostic data about your vSphere environment then alert you about any potential problems to your vSphere environment before they can cause an outage.

The tool is called VMware vCenter Support Assistant and its a free download from VMware.
It works by regularly collecting support bundles (configuration and usage data) and transmitting them to VMware who will automatically match them to a list of known problems.

It is packaged as virtual appliance, the file is 648MB and it works with vCenter Server 5.1 / ESXi 5.0 or above

Available here:

https://my.vmware.com/web/vmware/details?downloadGroup=VCSA600&productId=491

 

 

Powershell Issues

I was recently working on a script to send an SMTP email reporting on VM’s that only have a certain amount of free guest VM disk space left.

It turns out to use ‘Connect-VIServer’ cmdlet in your PowerShell script. You have to follow two steps:

1) Install VMware PowerCLI in your machine. It will register itself in PowerShell during the installation.

2) Use ‘Add-PSSnapIn VMware.VimAutomation.Core’ to add the registered PowerShell VMware Snapin to the current session.

So essential, the first lines of your script should have the ‘Add-PSSnapIn VMware.VimAutomation.Core’ entry otherwise the native Microsoft PowerShell

will fail due to unknown commands.

For those interested, the script looked like the below.

Add-PSSnapIn VMware.VimAutomation.Core

Connect-VIServer <servername>

$treshold = 10GB

$report = Get-VM | where { $_.PowerState -eq “PoweredOn” -and $_.Guest } | Get-VMGuest | %{
$vm = $_
$_.Disks | where {$_.FreeSpace -le $treshold} | `
select @{N=”VMName”;E={$vm.VMName}},Path,@{N=”DiskFreeGB”;E={[math]::Round((($_.FreeSpace)/1GB),2)}}
}

$emailFrom = <from-email-addr>
$emailTo = <to-email-addr>
$subject = “Free disk space less than ” + ($treshold/1GB) + “GB”
$body = $report | ft -AutoSize | Out-String
$smtpServer = <smtp-server>
$smtp = new-object Net.Mail.SmtpClient($smtpServer)
$smtp.Send($emailFrom, $emailTo, $subject, $body)
A big thank you to LucD on the VMware Communities Forum.

Customization of the guest operating system ‘windows7Server64Guest’ is not supported in this configuration

Encountered this problem recently while I was migrating a vSphere environment across to newer hardware. I needed to migrate a template across which required converting the template back to a VM, then using Veeam migrating across to the new vCenter, once there I converted back into a template. Also along for the ride were a couple of guest customization scripts which I exported from the source and imported on the new hardware. Easy right?

Unfortunately, whenever I tried to deploy a VM from this template it threw the following error “Customization of Windows guests fails with the error: Customization of the guest operating system ‘windows7Server64Guest’ is not supported in this configuration”. Now VMware does have a knowledge base article on this problem but it simply asked me to install VMware tools. Well, VMware tools were already installed. I found another suggestion here Pepper Crew Blog but that did not solve the problem either.

What ended up solving the problem for me was converting the template back to a VM, powering the VM on then updating the existing VMware Tools to a newer version. Once I had completed the VMware Tools update it had resolved the problem.

Link to VMware Knowledge Base Article: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1020716