1178 - How to use logical volume management (LVM)

Redstor's LVM Snapshot feature for the Linux ESE Agent is used when a crash-consistent backup is required, for example when backing up database files that can be written to while a backup is in progress.

Configuration

The LVM Snapshot option will only be available if LVM is installed on the machine. The feature can be enabled in Options and Settings > Advanced in the Agent.

Note: If LVM is not installed on the machine, it is possible to install LVM and create additional volumes as LVM volumes. However, if you want your root filesystem to be an LVM volume, you will need to build a new machine with LVM and migrate your data to it.

LVM options can be profiled in the Management Console under Configure > Profiling > LVM - Advanced.

When LVM Snapshot is enabled, at the start of each backup a snapshot is taken of each LVM volume that contains files selected for backup. These snapshots are used to read the contents of these filesystems during backup. Non-LVM volumes are processed as usual. The snapshots are deleted upon completion of the backup.

Snapshots are created with a size expressed as a percentage of the volume size. Before enabling LVM snapshots, you need to ensure that there is sufficient free space in each volume group for snapshots of all the logical volumes in that volume group.

For example, in a volume group with 100GB of space, of which 75G is allocated to the data volume, the remaining 25GB of free space will be sufficient for our default 10% LVM Snapshot size (10% of 75GB = 7.5G).

The backup of each LVM volume will be in a crash-consistent state. However, the snapshots will not be simultaneous, which means that the backup will not be crash-consistent across volumes. As usual, the backup of non-LVM volumes will not be consistent.

Troubleshooting

• Insufficient free space

This error occurs while creating a snapshot for a volume. It happens because there is no free space in the volume group, which means ESE cannot create an LVM snapshot unless space is freed up. (This is an LVM constraint and is not specific to ESE.)

LVM snapshots differ from VSS snapshots. When you take a VSS snapshot, it uses free space within the filesystem to store the snapshot (or you can give it another drive on which to store the snapshot, but it is still stored within the free space of some filesystem.) By contrast, LVM snapshots require space outside of the filesystem - in the VG (volume group). In LVM, you have PV (physical volumes), which are conventional partitions on a disk (or an entire disk). You then add PVs to a VG, so the VG is like a combined pool of storage space across one or more PVs. You can then divide the VG into one or more LV (logical volumes), and then you format the LV with a filesystem in which to store your files. You can change all this dynamically, adding new PVs to a VG to create more space, or remove failing PVs (assuming there is sufficient free space in the VG to move the data off that PV.) If there is free space in the VG, you can create new LVs, or extend an existing LV.

An LVM snapshot is a snapshot of an LV, but LVM has to store the contents of the snapshot somewhere, i.e. when some block of the LV is overwritten, LVM needs to remember what that block contained at the time the snapshot was taken. Because LVM sits below the filesystem, it does not have knowledge of the contents of an LV, so it cannot store snapshot data within the filesystem like VSS does. It does CoW (copy-on-write), so it is only storing what was changed since the snapshot was taken, but it needs somewhere to store those changes. The way it does that is to preallocate a chunk of free space from the VG where the LV resides. So there has to be free space on the VG to be able to take a snapshot. This error shows up when the VGs have no free space.

To get more free space in a volume group, you have two options:

• Add more PVs to the VG so as to create more free space. If you have free space on an existing disk, you can extend the disk. If not, you will need to add more disks. Steps for both methods are shown below. 
• Shrink an existing LV (first shrink the filesystem, then the volume) to free up existing space in the VG. 

For the first option, there are two ways to increase the free space in the relevant VG.

1. Add another disk

• Create a partition table on the new disk using gparted.
  
  
  
  
• Under Create as, choose Primary Partition.
• Under File system, choose lvm2 pv.
• Reboot the machine so that the partition is correctly detected.
• Run ‘pvs’ to show that the partition is present.
• Create a PV on the partition.
• Extend the VG using the new PV by entering vgextend <volume-group-name> </dev/partition>, e.g. vgextend ubuntu-vg /dev/sda2

2. Extend the volume group’s existing disk

• Create a new partition using the extra space.
• Reboot the machine so that the partition is correctly detected.
• Run ‘pvs’ to show that the partition is present.
• Create a PV on the partition.
• Extend the VG using the new PV by entering vgextend <volume-group-name> </dev/partition>, e.g. vgextend ubuntu-vg /dev/sda2

For methods 1 and 2 above, use the following commands:

• fdisk /dev/sda
• ‘n’ to create new partition
• ‘p’ to create as primary partition (default start and end sectors are accepted)
• ‘2’ for partition number (the default in our case)
• ‘t’ to set the type
• ‘8e’ to set the partition to Linux LVM
• ‘w’ to write the changes and ‘q’ to quit fdisk

• Snapshot is invalid

Two errors will be logged: one for the files on that volume that have not yet been backed up, and one at the end of the backup indicating the snapshot ran out of space.

This can happen if the default 10% volume size allocation in ESE is insufficient. For example, if you have the default allocation set and you have a 20GB volume used for a file share, ESE will take an LVM snapshot of 2GB (10%) and start to do a backup. If, before ESE finishes backing up the volume, more than 2GB worth of changes are written to the volume, then LVM runs out of space to track those changes. Because LVM can no longer track what has changed since the snapshot was taken, it marks the snapshot as invalid. ESE can no longer read the files from the snapshot to finish backing up that volume, and you will see this error.

The solution is to increase the percentage of space used for the snapshot (which then obviously requires more free space in the VG.) Alternatively, you may be able to reduce the number of writes to the volume during backup, perhaps by changing the backup schedule.