Redstor Pro can be configured to back up large file counts in a relatively short time period by changing the way that small files are handled. The ability to achieve these backup speeds is dependent on the environment, and is easier to achieve within a LAN speed environment where compression is not essential.
Tests were conducted on two servers with the following specifications:
- SERVER1 - Server 2003 x86 - Physical Server - 4GB Ram - 4 Core Processor - Static Data
- SERVER2 - Server 2008 x64 - VMware - 8GB Ram - 4 Core Processor - User Data
SERVER1 contained 6 267 660 files and SERVER2 contained 1 748 432 files. Before the settings were modified for large file counts, backups would take approximately 24 hours (sometimes much more) to complete. After the settings were optimised for large file counts, the backup times dropped significantly, which can be seen in the table below.
| Account | Backup Time | Version | Type | Prep. Time | Data Protected | Files Protected | Transferred | OS |
| SERVER1 | 30 Sep 2011 05:50 | 6.1.0.4 | SE | 10 hours 50 min | 938.21 GB | 6 267 660 | 2 407.13 MB | Windows 2003 |
| SERVER2 | 29 Sep 2011 21:58 | 6.1.0.4 | SE | 2 hours 57 min | 991.01 GB | 1 748 432 | 2 544.69 MB | Windows Server 2008 |
Settings used
The settings used to achieve this massive improvement can either be configured by profiling, or added into the a5backup.properties file for individual Data Management Agents.
|
Setting |
Profiling |
a5backup.properties |
Note |
| Disk Usage | Minimize Disk Usage=True | backup.minimize.diskusage=True | True or false |
| Background Transmission Queue Capacity | Background Transmission Queue Capacity=50 | backup.txqueue.capacity=50 | Number of files |
| Backup Transmission Queue In-memory Limit | Backup Transmission Queue In-memory Limit=3072 | backup.txqueue.inmemory.maxfilesize=3145728 | File size in bytes in the properties file and KB in profiling |
| Small File Patching | Disable small file patching=102400 | backup.patch.minfilesize=102400 | File size in bytes in the properties file and profiling |
| Small File Compression | Disable small file compression=102400 | backup.compression.minfilesize=102400 | File size in bytes in the properties file and profiling |
| Retry Attempts | Retry Attempts=50 | backup.max_retry=50 | Number of connection retries |
| Retry Sleep | Retry Sleep=360 | connection.retries.delay=360000 | Sleep period in milliseconds in the properties file and seconds in profiling |
| Connection Timeout | Connection Timeout=999 | connection.timeout=999000 | Timeout in milliseconds in the properties file and seconds in profiling |
Note: Backup Transmission Queue In-memory Limit, Background Transmission Queue Capacity and Disk Usage have been deprecated in the V7 SP Console. This functionality can still be used on V6 Agents backing up to V7 Storage Platforms by configuring the options on the Agent side. V7 Agents feature our Streaming Backup functionality, which effectively replaces the background transmission queue. Please enable Streaming Backup to use this functionality.
Further Suggestions:
You may also want to investigate the NTFS change journal, as using this decreases the amount of time taken to build the backup selection list (after the first backup, and if the selection has not changed). Details of this setting can be found in Article 484.
Comments
0 comments
Please sign in to leave a comment.