Data consistency is the sum of validity, accuracy, usability and integrity of data between applications and across an operating system. This is a critical characteristic of any backup solution. In this blog, I will discuss data consistency in more detail, define different levels of consistent backups and explain how to make sure that your Veeam backup is consistent.
Today, it’s hard to imagine a situation when data consistency doesn’t matter for a company. It always matters, especially when the data is being changed actively like in bank systems, simultaneously making thousands of financial transactions within an account database.
Levels of backup consistency
There are a few levels of backup consistency which can be identified by whether backup can be taken from the live system, if interdependent files are assured to be identical and whether there is protection against transactional data loss (application-aware).
Let’s consider these levels of backup consistency one by one.
Inconsistent backup
The first and oldest is inconsistent backup. It can be easily imagined as a simple file copy operation which brings all computer files to another place such as a disk drive, USB disk or a NAS. If a change in the files happens while backup is running, then the result is an inconsistent backup. In this case, backup will not be equal to original system files. Further, many files that make up the guest operating system of the VM may have different timestamps; making an unclear recovery point.
This method doesn’t save any data from system memory and might have an issue with access to files locked for reading.
Crash-consistent backup
The next level is crash-consistent backup. The main difference from inconsistent backup is that crash-consistent backup’s data saved within the same moment of time. It can be imagined as an immediate snapshot of all files on a disk.
However, there is a reason it’s called “crash-consistent.” If you restore the system from this backup, it’s the same as restoring a system after a hard-reset since neither opened files nor pending I/O operations are saved during this backup. So some applications would demand special handling and post-crash maintenance. That might be especially harmful for various database applications.
For example, Microsoft Exchange would require you to setup a special recovery group and integration of log files. Microsoft SQL or Oracle would need to roll transactions which were ongoing for backup time and not committed into DB. This is loosely equivalent to just pulling the power on the server if it were physical or powering off the VM, the disk is paused right at that point in time with no preparation.
File-level consistent backup
In this backup, all of the files in the system have a consistent state unlike crash-consistent where files weren’t saved with latest updates. Nevertheless, ongoing application transactions still aren’t taken care of.
For VMs running on an ESX(i) host, VMware offers VMware Tools quiescence. When taking a snapshot of a running VM, VMware Tools quiesce the VM file system.
Transaction-consistent backup
This backup doesn’t contain the aforementioned downsides and has all the data from the same time point, including completed transactions. The challenge here is to achieve it on the “live” system when downtime is forbidden and applications should be running 100% of the time.
Veeam can make transaction consistent backups with the help of Microsoft Volume Shadow Copy Service (VSS). Microsoft VSS is able to coordinate all specific actions which should be done on a system in order to “quiesce” applications in the OS. It can pause the disk I/O when triggered and assist the backup software. Microsoft VSS is included in all Windows OS since Windows Server 2003.
Application-Aware Image Processing (AAIP) is Veeam’s proprietary technology that ensures successful VM recovery, as well as proper recovery of all applications installed on the VM without any data loss. Veeam Backup & Replication does not deploy persistent agents inside VMs, but uses a runtime coordination process on every VM that is started once the backup operation is requested and removed when finished. This process was designed to prevent possible agent-related activities such as pre-installing, troubleshooting and updating the agents.
To trigger VSS freeze, Veeam Backup & Replication goes as a VSS requestor and requests Microsoft VSS to create a consistent and reliable state of application data before taking a VM snapshot or a Hyper-V shadow copy of the VM. Then the VSS provider coordinates with VSS-aware applications to freeze all I/O operations at a specific point in time.
This guarantees that there are no unfinished database transactions or incomplete application files during data copying operations.
How to make sure that backup is consistent?
When a backup job is being configured, find the “guest processing” step and turn on “enable application-aware processing” for VMs being backed up with this job. Then run the job and make sure it will run successfully.
Note that leveraging Veeam’s AAIP, critical application health such as log truncation for SQL and Exchange are performed. This is an important maintenance step as part of the transition to an agentless backup. Otherwise, the logs would grow out of control.
To enable VMware quiescence for VMs in the job, follow the steps described in the Veeam Help Center.
When you turn on both Application-aware image processing and VMware Tools quiescence, Veeam will use AAIP as the default method, and if VSS is not possible, fails over to VMware Tools quiescence.
Please keep in mind you can always check your backups using SureBackup functionality by testing their restore in an isolated sandbox.
Helpful resources:
- Free White Paper: The VSS Role in vSphere, Hyper-V and Agent-Assisted Backups
- Veeam Blog: Backing up MySQL on a Linux VM
- Veeam Help Center: Application-Aware Image Processing
- Veeam Community Forum: Transactionally Consistent Backups
Questions and comments are welcomed as always.
