Life is easy if you are running Windows because if you want to create image-based VMware Backups with Veeam Backup & Replication, Quest vRanger, PHD Virtual or any other VADP using competitor you can use VSS. Backing up Linux is much more complex as there is no equivalent. What you get are crash consistent copys from your virtual disks. After some research i couldn't find any established solution. This is the backup vendors answer (I am not talking about application aware backups as this is another problem):
veeam Backup & Replication v6
Veeam refers to the "Enable VMware tools quiescence" option. But is this true? Yes, there is this option and you can enable it. But the vmsync driver inside you virtual machine is disabled by default. So if you activate "Quiesce", nothing actually happens. The backup succeeds but all you get is an inconsistent state.
Source: User Guide
Quest vRanger 5.3.1
The solution Quest provides is only a small hint: Install VMware Tools, create freeze Scripts and enable Guest Quiescing. But who will support my custom script?
Source: Quest Solution SOL84967
PHD Virtual Backup
PHD Virtual does not provide any information about consistent linux backup. The only thing i could find was a note "Quiesce? Windows only!"
Possible Solution?
So, how to create a filesystem consistent linux backup with veeam, vRanger or PHD virtual? As every vendor is doing the same -triggering the VMware API- the answer is identical. But first let's have a look at the basics. What do i have to do to get a consistent state? And how can i determine that my backup is consistent?
An inconsistent filesystem has to be recovered prior to mount. Using dmesg you can determin whether is was consistent or not:
Consistent filesystem mount:
root@ubuntu:~# dmesg |grep EXT [3.711991] EXT4-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [7.685314] EXT4-fs (dm-0): re-mounted. Opts: errors=remount-ro
Inconsistent filesystem mount:
root@ubuntu:~# dmesg |grep EXT [3.780568] EXT4-fs (dm-0): INFO: recovery required on readonly filesystem [3.780855] EXT4-fs (dm-0): write access will be enabled during recovery [4.153234] EXT4-fs (dm-0): recovery complete [4.178622] EXT4-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null) [8.058018] EXT4-fs (dm-0): re-mounted. Opts: errors=remount-roThis test was made with the current Ubuntu 12.04 LTS.
Solution 1: Custom fsfreeze Script
The fsfreeze command suspends and resumes the access to an filesystem. After suspendig the access, the volume is in an consistent state and can be copied. Please note that fsfreeze is a recently published tool and not available on older systems.
1. Install VMware Tools
2. Create custom scripts:
root@ubuntu:~# touch /usr/sbin/pre-freeze-script root@ubuntu:~# touch /usr/sbin/post-thaw-script
3. Edit both scripts and add your mountpoints. Your file should look like (Only one mount):
root@ubuntu:~# cat /usr/sbin/pre-freeze-script #!/bin/sh fsfreeze -f / root@ubuntu:~# cat /usr/sbin/post-thaw-script #!/bin/sh fsfreeze -u /
4. Make both files executable:
root@ubuntu:~# chmod 755 /usr/sbin/pre-freeze-script root@ubuntu:~# chmod 755 /usr/sbin/post-thaw-script
5. Activate "Quiesce" Option in your backup client
During backup the backup client triggers the vCenter server to make a snapshot with the "quiesce" option. This involves both scripts to freeze and unfreeze the write IOs on the filesystem during the snapshot creation.
Solution 2: vmsync
As mentioned above VMware has created a sync driver that allows to create consistent backups. Unfortunately i couldn't find any information about this driver and it is disabled by default, giving a small explanation:
[EXPERIMENTAL] The VMware FileSystem Sync Driver (vmsync) is a new feature that
creates backups of virtual machines. Please refer to the VMware Knowledge Base
for more details on this capability. Do you wish to enable this feature?
This comment emphasizes that this feature is not supported at the moment. I also couldn't find this Knowledge Base details.
To enable vmsync driver you have to enable it during the installation, or run vmware-config-tools.pl later:
root@ubuntu12:~# vmware-config-tools.pl Initializing... Making sure services for VMware Tools are stopped. vmware-tools stop/waiting [EXPERIMENTAL] The VMware FileSystem Sync Driver (vmsync) is a new feature that creates backups of virtual machines. Please refer to the VMware Knowledge Base for more details on this capability. Do you wish to enable this feature? [no] yes
After activating vmsync and "quiesce" option in your backup client you can create consistent backups. I have tested both solutions in testing environments with Ubuntu 12 and RHEL6 systems and was able to create consistent backups. But please note that this is not supported by VMware or any Backup Vendor. So please test it out before you roll it out into production.
Excellent post. Quiescing linux FSs does not appear to be a concern to many vendors or admins and they just rely on crash consistent snapshots. Sure 99% of the time the journalled FS will take care of it for you but for absolute mission critical applications can you take that chance? Symantec actually include a linux VM quiesce utility (SYMCquiesce) which is cool I guess.
It's hard to check these to ensure they are working.
The old prefreeze and post thaw scripts don't seem to be working, and there is no way to tell if the vmsync driver is successful or not that I can tell.
You could perform a rehersal restore and check the mount in the messages logfile. If the filesystem was in an consistent state you should see something like "mounted filesystem with ordered data mode".
Another indicator could be that if you activate "Guest Quiesce" and it fails, you should get an error message from your backup client as it can not create a backup without successfully creating a snapshot.
Pingback: FTF Veeam v6 Replication – Applications In The VMs | VMETC.com
The first solution will not work. Once the / is freezed, vmware lost access to /sbin/fsfreeze ... so it is not possible to thaw.
You need to consider that noatime option needs to be added to the mount point temporarily for the scripts working.
So you need to check and preserve the mount options of "/usr/sbin/pre-freeze-script" and "/usr/sbin/post-thaw-sript", and add noatime to the mount point with remount, then restore the mount point options in post script.
Submitting the request to Redhat to add the same comments to manual of fsfreeze.
so the "/" mount point must be remount as "noatime" to solve the problem
To find out whether vmsync driver is installed:-
lsmod | egrep -q '^vmsync'
case $? in
0) echo "vmsync installed";;
*) echo "unable to confirm that vmsync is installed";;
esac
Pingback: PRE und POST Backup Skripte mit VMware | Backupinferno
Pingback: VMware Sync Driver - fsfreeze - consistent snapshot - TecHub
Hi,
Thank you for your post.
Is it usable for LVM disks?
man fsfreeze have said me: