Link Search Menu Expand Document

File Backup Proxy

Proxy Placement

As a general rule the proxy should be as close to the source data as possible with a high bandwidth connection. The traffic from the source to the proxy is not yet optimized, meaning that 100% of the backup data will be transferred over this link.

Also consider a good connection between proxy and repository. Optimized data (normally ~50% of the source data size) will be transferred here.

Proxy OS requirements

We recommend the latest supported version of Windows Server OS for all proxies.

Proxy Sizing

Getting the right amount of processing power is essential to achieving the RTO/RPO defined by the business. In this section, we will outline the recommendations to follow for appropriate sizing.

Processing Resources

Proxies do have multiple task slots to process source data. It is best practice to plan for 1 physical core or 1 vCPU and 2 GB of RAM for each of these tasks. When sizing a proxy, do not forget to dedicate necessary resource to the operating system (usually 2 core and 4 GB RAM).

A proxy task (core) processes 1 file share at a time and CPU/RAM resources are used for inline compression and/or encryption. It is currently not recommended to oversubscribe the number of tasks per core.

Backup IO control and proxy resource usage

When configuring the backup job, the backup IO control parameter will affect the way proxy resources will be used to backup the share by controlling the number of proxies allowed to backup a given share and the number of concurrent threads per task following rules below :

IO control Max Proxies per share Threads per task
Lower impact 1 1
Below normal 1 4
Normal 2 8
Above normal 4 16
Faster backup unlimited 16

The backup IO control is a static setting, it is not based on latency monitoring.

Expected performance

Each proxy task (core) is expected to process 100 MB/s or 5 million files per hour. Depending on the Backup IO control seting, these numbers may slightly vary if the number of threads per core is modified. Note: these numbers are given assuming the infrastructure itself won’t present any limitation.

Guidelines

According to previous statements:

  • Generally speaking, the “Normal” setting will be optimal. An exception might be the processing of a single, exceptionally large share.
  • If you have less shares than number of proxy cores, then consider spreading these cores across multiple proxies. If you have more shares than proxy cores, then consider adding cores to your existing proxies before adding more proxies.
  • When using virtual proxies, keep in mind that the number of cores per proxy must not exceed 8 to avoid co-stop.

Examples

Example 1

3PB source data, 10M files, 40 shares, 2% change rate, 8 hours backup window.

  • Incremental throughput requirements: (3PB x 2%) / 8 hours = 7.68 TB/h (at 100 MB/s)
    • 7.68 / 0.34 = 22.58 File Proxy cores.
  • Files to process per hour: 10M / 8 hours = 1.25M/h (processing 5 millions files per hour)
    • 1 File Proxy core.
  • The highest core count is 22.58

Required file proxies resources are 24 cores and 48GB of RAM. These resources can be deployed on a single physical proxy (of at least 26 cores) or four virtual proxies (8 cores each). “Normal” and above backup IO control setting can be used.

Example 2

50TB source data, 300M files, 2 SMB shares, 3% change rate, 8 hours backup window.

  • Incremental throughput requirements: (50TB x 3%) / 8 hours = 0.1875TB/h
    • 1 File Proxy core.
  • Files to process per hour: 300M / 8 hours = 37.5M/h
    • 37.5 / 5 = 7.5 cores
  • The highest core count is 7.5

Required file proxies resources are 8 cores and 16GB of RAM. These resources must be deployed on a minimum of four proxies (two shares, one proxy task per share) of 4 cores each. “Normal” and above backup IO control setting can be used.


Back to top

Copyright © 2019-2021 Solutions Architects, Veeam Software.