Backup transfer deduplication can be enabled in ESE by going to Tools > Options > Performance.
When enabled, this feature aims to transfer identical files only once to the Storage Platform by leveraging previous backups by the same account or other accounts. This is particularly useful for files that are moved regularly, as our system will check if a file has been backed up elsewhere before transferring it in full. Files are backed up at their original size so they can be restored individually.
In addition, if VSS is enabled on the ESE Agent, files included in the backup selection that are identical on the same volume are sent only once.
The transfer deduplication phase (as indicated in the progress window during a backup) works as follows:
- Identify all files that must be processed for backup (by hashing all selected files and generating a preliminary index).
- Communicate with the account's StorageServer to determine which files it hasn't received yet (by using their hashed values from the preliminary index).
- Start processing all files to ensure no changes have occurred in the meantime.
- Send only the files identified in point 2 to the Storage Platform.
This method reduces the file transfer size, and prevents files from being processed twice, increasing disk activity during a backup. However, the comparison between the preliminary index (generated in point 1) and the index from the previous backup (retrieved from the StorageServer in point 2), which is required for deduplication to take place, may extend the backup duration, resulting in a slower rather than a faster backup.
- When backup transfer deduplication is enabled, partial backups (as with incomplete or interrupted backups) are no longer automatically purged on the Storage Platform unless the backup was successful. This is to be expected, since transfer deduplication leverages files from previous backups, including partial ones, to deduplicate transfers of subsequent backups. Purge partial backups manually from the Storage Platform Console if disk usage is being adversely affected.
- Windows Server Deduplication will not reduce your total data with Redstor, it is only to reduce disk usage on your machines.
Interaction with local backup copies
If local backups are also enabled, no deduplication occurs to the local backup destination.
Interaction with VSS
If VSS is disabled, identical files within the same volume will be sent separately and will not be deduplicated.
To enable VSS, Open the ESE Agent and go to Tools > Options > Advanced.
- Note: VSS does not apply to network shares.
Please sign in to leave a comment.