SureBackup Job Failure: "The resource 'VeeamBackup_' is in use."

A SureBackup job in a vSphere environment fails with the error:

The resource 'VeeamBackup_<hostname/FQDN/ip>' is in use.

The error as found in the SureBackup Job log%programdata%\Veeam\Backup\SureBackup_<job_name>\Job.SureBackup_<job_name>.log:

Error        Failed to connect backup datastore to the ESXi host "esx01.domain.tld". (Veeam.Backup.Common.CRegeneratedTraceException)
Error        Fault "ResourceInUseFault", detail "<ResourceInUseFault xmlns="urn:vim25" xsi:type="ResourceInUse" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><type>Datastore</type><name>VeeamBackup_vbr.domain.tld</name></ResourceInUseFault>" (Veeam.Backup.ViSoap.ViServiceFaultException)
Error        VimApi.ResourceInUse
Error        The resource 'VeeamBackup_vbr.domain.tld' is in use. (System.Web.Services.Protocols.SoapException)

The hostd log of the ESXi host associated with the datastore, the following error may be found:

RemoveDatastore] Datastore VeeamBackup_<hostname/FQDN/ip> still has VMs registered.

This error occurs when the VeeamBackup_ datastore, created by vPower NFS, cannot be accessed or interacted with, and the vSphere environment reports "Resource In Use."

The vPower NFS datastore stays mounted to the ESXi host. Each time you start an operation that would use the datastore, Veeam Backup & Replication checks whether the vPower NFS datastore is mounted to the host. If you manually unmounted the datastore or the datastore was unmounted by other causes, Veeam Backup & Replication remounts the datastore.

Known Complication

Should a VM other than those Veeam creates become assigned to the VeeamBackup_ datastore, it will cause the datastore to be marked as in use. When this occurs, the error "The resource is in use" will occur when Veeam Backup & Replication sends the request to unmount the datastore, and the vSphere environment prevents the unmount action. The most commonly reported scenario is for the vSphere Cluster Service VM (vCLS VM) to become placed on the vPower NFS datastore by accident.

To resolve this issue, review the following procedure:

1. Before you begin, ensure that no tasks of the following type are running:
   • Instant VM Recovery
   • SureBackup
   • Linux Guest File Restore (Multi-OS)
2. Within a vSphere Client, locate the VeeamBackup_ datastore mentioned in the error.
3. Check if any VMs are associated with that datastore.
   • Any active VMs associated with the VeeamBackup_ datastore must be migrated to a different datastore using storage vMotion.
   • If a VM named 'vCLS' is found on the datastore, use storage vMotion to migrate it to a different datastore. Then, consider configuring the vSphere Cluster Service to only use specific datastores, avoiding the VeeamBackup_ datastore.
   • If the VeeamBackup_ datastore is listed as (inaccessible) and there is a VM associated with the datastore, consider the following:
     • If the VM contains critical data, contact Veeam Support. Veeam Support can help determine whether the VM's underlying data is still present in the vPower NFS cache; if it is, it may be possible to get that rogue VM back into a working state so it can be Storage vMotioned to a different datastore.
     • If the VM is non-critical, remove it from inventory.
4. Unmount the VeeamBackup_ datastore.
   • In some rare instances, you may have to open a vSphere client directly to the ESXi host(s) associated with the datastore and perform the unmount operation.
5. Once the VeeamBackup_ datastore has been unmounted, rerun the SureBackup job.