Capacity management approaches
There are two approaches to managing storage capacity on a Storage Platform:
- Manual space management prevents the Storage Platform from doing any load balancing automatically, which means that all load balancing within Storage Pools will require manual intervention.
- Threshold space management requires a storage threshold percentage to be specified. The Storage Platform will then automatically attempt to keep all StorageServers within a Storage Pool at or below the specified threshold.
Caveats when using threshold space management
The scenarios below illustrate cases where the threshold-based automatic load balancing may not work as expected.
1. Storage Pool contains offline servers
If the Storage Pool contains offline servers, it is unable to queue automatic account moves for load balancing purposes.
2. Storage Pool contains StorageServers without MirrorServers
If the Storage Pool contains mostly StorageServers that are not connected to MirrorServers, but at least one of the StorageServers does have a mirror, the Storage Pool will be unable to queue automatic account moves for load balancing purposes. This is to prevent accounts from being moved from a server with a mirror to a server without a mirror.
3. StorageServers have inconsistent HSM settings
If the StorageServers in a Storage Pool have inconsistent Hierarchical Storage Management (HSM) settings, the Storage Pool will be unable to queue automatic account moves for load balancing purposes. HSM must either be disabled everywhere or enabled everywhere. This is to prevent accounts from being moved from servers where data would be retained via HSM to servers where data would not be retained via HSM.
4. Some StorageServers are not suitable for new accounts
Accounts are created on a random StorageServer within the Storage Pool that has been specified under Location for new accounts in the group configuration window of the Storage Platform Console (see Article 1117). The location may include StorageServers that are above the specified threshold. This can be circumvented by locking any StorageServers that should not receive new accounts that are being created. StorageServer locking (and unlocking, when the server is ready to receive new accounts again) is done manually from the Console.
5. All StorageServers are above threshold
Threshold-based load balancing requires the target servers' capacity to be below the specified threshold. This means that when all StorageServers are above the threshold, no automatic account moves will be actioned. In this scenario, the Storage Pool in question would either need to be managed manually, or its threshold would need to be increased in a way that splits the StorageServers' usage to balance the capacity.
Best practices for using threshold space management
Monitor storage usage
It is imperative to keep a close eye on how a Storage Pool's usage is growing in relation to the threshold you have set. These values can both be tracked from the Storage Platform Configuration view in the Storage Platform Console. The pool average usage appears at the bottom of the table when a Storage Pool is selected.
If the average usage grows above the threshold, it is important to adjust the threshold. This is to ensure that:
- load balancing is working efficiently, selecting accounts based on the storage usage, and that
- all StorageServers within a Storage Pool do not exceed the threshold simultaneously, which would stop all load balancing.
Lock StorageServers as needed
If one StorageServer in a pool is much closer to 100% usage than other servers in the same pool, it may be worthwhile to lock that StorageServer to prevent new accounts being created on it. This will avoid the scenario where the addition of a large new customer creates a capacity risk on the StorageServer.
Regularly review your list of locked StorageServers to ensure that, once a server's usage returns to the pool average, the server is unlocked for new account creations again. The locked status of a StorageServer can be seen in the Console's Storage Platform Configuration view.
Keep settings consistent
When faced with StorageServers that are going to be inconsistent with the rest of the Storage Pool for an extended period of time (e.g. known offline, known missing mirror, known HSM differences), keep those servers in a different Storage Pool to prevent these inconsistencies from getting in the way of automatic load balancing. If additional Storage Pools are required, log a ticket with our support team.
Move accounts to improve load balancing
Identify accounts that are suitable for being moved off when needing to load balance manually between StorageServers, whether as part of standard workflows or when the server is nearing 100% usage (when automatic management has not been able to keep the server at an acceptable usage). The report AccountsOnSS can be generated from the Console and used to identify sensible candidates. This report takes the StorageServer IP/port combination as its parameter in the format Address:Port.
The automatic load balancing is sequential in nature, so if a server is close to reaching 100% capacity, it can be beneficial to add manually queued account moves alongside the automatic load balancing to increase the rate at which data can be moved off the StorageServer. The automatic load balancing tends to pick a small number of larger accounts, which can take longer to finish and free up space. It is therefore useful to manually add smaller accounts to the account move logic, as these will finish before the larger accounts, freeing up space sooner and preventing the StorageServer from reaching 100% usage.
While there is no maximum number of concurrent moves that a server will attempt to run, it is advised not run so many that the average speed of each move drops too low to be useful. This number will depend on the performance of the underlying StorageServers, network and disks.