942 - ESE: How to keep a local backup copy – Redstor

Copies of your most recent backups can be kept on a local machine in addition to those saved on the Storage Platform. This facilitates faster restores than would typically be experienced when the Storage Platform is remotely located. During the backup process, your data is streamed to the Storage Platform as usual, but an additional stream is sent to the local copy location, essentially resulting in a duplicate. Details of this process can be seen in the backup log.

Enable this in your ESE Agent under Tools > Options > Local Backup Copy.

How many copies can I keep?
Where can I keep my local copies?
What about disk space?
Are my normal backups safe when errors occur?
Can this be profiled from the Storage Platform?
How do I restore from a local backup copy?
Can I do a Full System Recovery from a local backup?

How many copies can I keep?

The maximum number of local copies is limited based on two factors:

The retention preferences set in your ESE Agent under Tools > Options > Local Backup Copy.
The relevant account's group roll-up retention set on the Storage Platform under Configure Group.

Example scenario: The Agent is set to keep 6 copies. The group on the Storage Platform is set to keep 3 backups, to roll up when 5 backups are reached, and to keep 1 roll-up:

For backups 1 to 5, a local backup is kept each time (i.e. 5 local copies, and 5 backups on the Storage Platform).
After the 5th backup, a roll-up on the Storage Platform will reduce the number of online backups to 4 (i.e. 1 roll-up and 3 backups).
After the 6th backup, you'll have 5 online backups (1 roll-up, 3 previous backups, plus 1 new backup).
You would expect the Agent to have kept a 6th local backup. However, the Agent will consolidate (roll up) the 1st and 2nd local copies into one to match the Storage Platform's count of 5 "online" backups.
With the 7th backup, you'll have 6 local copies for a short time, but this will again be consolidated to 5 (the previously consolidated copies 1 and 2, with the addition of backup 3) at the next backup, since the Storage Platform still maintains the maximum of 5 online backups.

Note: After deactivating this setting, local backups will remain on the machine. If enabled again, they will gradually be removed as the retention window shifts with each new online backup.

Where can I keep my local copies?

On the local machine
On a network share (UNC path)*
On a removable drive

*While the ESE user interface for Linux does not currently allow a network share to be selected as the location for a local copy, it is still possible to store the local copy on an NFS or CIFS share. See Article 1296.

What about disk space?

Take note that, depending on the size of the backup, several local copies can quickly consume available disk space.

The following factors affect the size of a local copy:

Files larger than or equal to the small-file threshold of 1024 KB are kept in a common location known as global storage. If a file that already exists in global storage is backed up, it is not stored again and only references the copy already in global storage. This is called deduplication.
Note: Global storage is located in the local copy location's .global folder.
Files that recur in multiple folders or are common between backups, and are smaller than the 1024 KB threshold, will have multiple copies stored in the local copy location (i.e. no deduplication).
Note: These are located in a date-labelled folder in the Account ID folder within the local copy location, e.g. \Local Copy\70b18fba-8e03-450b-a48d-24ac43d97d32\20170913121417.
Files are stored in compressed form and are also encrypted. Compression effectiveness depends on the file type and contents.
Changed files are not patched between backups. Instead, they are backed up in full each time a change occurs.
After each new backup, the oldest local copies that fall outside the configured retention setting are removed, and a new baseline backup is created. This is called local backup consolidation.

Example scenario for illustrative purposes:

5 folders (20 files) are included in the Backup Selection, containing 10 large files ≥ 1MB (totalling 100 MB) and 10 small files < 1MB (totalling 7 MB).
2 large files are identical (10 MB each); 2 small files are identical (500 KB each).
The local copy retention setting is 5.

Result:

From backups 1 to 3 no data has changed, so the total space is: 100 MB - 10 MB + 7 MB = 97 MB - 5% compression = 92.15 MB (20 files). (One of the identical large files ≥ 1MB has been deduplicated.)
On the 4th backup three changes occur. One of the large files of 10 MB has changed, and two identical small files of 600 KB each have been added: (10 MB + 1.2 MB) = 11.2 MB - 8% compression = 10.304 MB + 92.15 MB = 102.454 MB (now 22 files). (To reiterate: Changed files are stored in full, small files are not deduplicated, and compression effectiveness varies.)

Please note that this also applies to selected databases and larger files that change frequently - every backup run will back up a complete copy of these files to the local copy location. This can result in very large local copy sizes for Exchange, SQL servers or virtual hard disk files whenever multiple local copies are configured. If this behaviour is undesirable, reduce the number of local copies.

Suggested size calculation

Because of the above factors, the size calculation function (switched on by clicking the Size button on the Agent toolbar) will not provide an accurate indication of the size of the local copy.

Instead, the size of the local copy’s date-labelled folder on disk can be used to determine each local copy’s size, e.g. \Local Copy\70b18fba-8e03-450b-a48d-24ac43d97d32\20170913121417.

Otherwise, use the example above as a guide to calculate the size of a local copy.

Limitations

When the local copy setting is disabled, existing local copies will not be deleted.
When a full Snapshot backup has been done but not yet imported into the Storage Platform, the next backup with a local copy will ignore the Snapshot and create an incremental copy based on the previous backup.

Are my normal backups safe when errors occur?

Issues such as lack of free disk space or folder permissions that have changed will prevent local copies from being saved. Unless problems are encountered when streaming your data to the Storage Platform, your normal backups are safe regardless of the success of the local backup.

For other errors see these logs:

Saving local copies: Errors are logged in the Backup Log when local copies are saved.
Consolidating older local copies: Local copies are consolidated to maintain the retention settings (see above). Problems with consolidation are logged in a seperate log indicated in grey.

Can this be profiled from the Storage Platform?

Administrators can profile local copy related settings from the Storage Platform Console.

Note: While it is possible for advanced users to make changes to the local copy configuration, such as to the settings localcopy.enabled, localcopy.retention and localcopy.path, by editing the a5backup.properties file, this is not possible if credentials are needed for the storage location. In such cases, all changes must be made through the ESE user interface or via remote access so that the user can be authenticated.

How do I restore from a local backup copy?

Find the backups in the Restore tab as you normally would, but look for the dates listed under Local in the Backup list.

Can I do a Full System Recovery from a local backup?

Yes, a Full System Recovery can be done from a local copy, but only on Windows.