I was recently involved with a Veeam deployment that was running into problems during testing, their only performance tier had run out of space. Though this wasn’t unexpected as the disk provisioned was undersized and just temporary until testing was finished, it was preventing new backups from finishing successfully.
The full performance tier belonged to a Scale-Out Backup Repository that was also configured with a capacity tier (copy + move mode) backed by an immutable AWS S3 bucket. Worth mentioning those backup files in the capacity tier were still within the immutability retention period.
According to the user guide“If you use the scale-out backup repository, keep in mind that the Delete from disk operation will remove the backups not only from the performance tier but also from the capacity and archive tier. If you want to remove backups from the performance tier only, you should move those backups to the capacity tier instead. For details, see Moving to Capacity Tier.”
Attempting to perform a “Delete from disk” operation was failing with the error “Error: Unable to delete backup in the Capacity Tier because it is immutable”.
While attempting to instantly recover a Hyper-V VM to an ESXi host, Veeam encountered the following error, ‘failed to publish VM %name% Error: One or more errors occurred’.
Some Veeam users were discussing the same symptoms when ‘Server for NFS’ feature is enabled on the Veeam forums. In this particular case, the VBR/repository role is configured on a StoreEasy 1460 which runs Windows Storage Server 2016 standard and ‘Server for NFS’ feature was indeed enabled.
As described by Andreas Neufert from Veeam, Virtual Disk Development Kit (VDDK) is provided by VMware which is leveraged by Veeam to perform backups and other functions. In this instance, there appears to be a bug introduced in VDDK after VMware introduced “a faster way to process data over Network NBD mode (async processing)” which is causing certain VMs to fail during backup jobs.
The recommendation from Andreas is to ensure the underlying ESXi hosts and vCenter are updated to the latest builds from VMware or Veeam support can add a registry key on the VBR server which disables the new VMware processing.
Veeam will always recommend calling support to obtain the reg key, this is to ensure you are applying the registry tweak for the right reason. However, if you are confident that you are affected by this issue, the details for the reg tweak have been provided below.
Anton Gostev recently wrote about a bug that will impact a lot of Veeam environments so I thought it would be best if I mentioned it here to help get the word out. Veeam have also created a KB article you can find here detailing this issue.
If your Veeam Backup & Replication console is showing a “Failed to check certificate expiration date” message upon opening the backup console, it means that your default self-signed certificate is about to expire.
A self-signed certificate is an identity certificate that is signed by the same entity whose identity it certifies. Veeam uses certificates to implement secure communications between your backup infrastructure components, as well as with any managed backup agents in your environment.
Now Self-signed certificates are automatically renewed every 12 months by your Veeam Server but due to a bug introduced in v9.5 U3a, the Veeam Backup Service will still have old information about the absolute certificate even after a new self-signed certificate is automatically generated. If you ignore this message, once the self-signed certificates are automatically renewed after 12 months, agent management functionality, as well as all granular restores will start failing.
Typically this will occur 1 year from the certificates creation date so the best course of action is to remedy the situation as soon as you see the error message and before the self-signed certificates expire. The fix is to manually generate a new certificate as described in this Veeam User Guide. Please note that this process will automatically restart the Veeam Backup Service so it’s is recommended to ensure no active jobs are running.
Worth mentioning, Veeam administrators can select or import their own certificate but most organisations are still using self-signed SSL Certificates which are generated when Veeam Backup & Replication is installed.