Veeam Agent for Linux

Veeam snapshot kernel module

Veeam Agent for Linux leverages its own Linux kernel module, sometimes referred to as the Veeam Agent for Linux Driver, to enable snapshots and changed block tracking of volumes.

The driver’s maps of changed blocks is not stored on the corresponding volumes, it’s only available in RAM. Thus, after rebooting the Linux computer, these maps are lost and the next incremental backup will have to re-create them by reading all data of each volume again.

This will most likely create higher cpu/disk load and longer backup duration than usual. However, the size of the incremental backup will not increase as the newly created block maps will be compared against the previous backup’s metadata to make sure only changed blocks get transferred to the backup repository.

Starting from Veeam Agent v6, there are two types of kernel modules.

veeamsnap: This is the original kernel module for legacy kernels and was also shipped with previous versions
blksnap: This is a newer kernel module. Veeamsnap is highly dependent on certain Kernel API calls that where partially deprecated in kernels shipping after 5.10. In effort to avoid compatibility issue in the future, the blksnap package is provided. In effect, it ships 2 kernel modules:
1. bdevfilter: A filter framework to intercept IO passing to block devices. There is an ongoing effort to donate this code to the community and include it in the Linux kernel directly avoiding any instability on legacy API calls. Integrating with bdevfilter could also be done by other software not written for backup specifically.
2. blksnap: A filter that plugs into bdevfilter, providing snapshot and changed block tracking support.

In general, if you use a kernel newer then 5.10, you should use blksnap, otherwise use the veeamsnap package. However, some distributions backport fixes into older kernels. For example, at the time of writing, if you are using SLES 15 SP3 with kernel 5.3.18-150300.59.106, you can also use blksnap. Therefore it is always recommended to check KB article 2804 if you are manually or directly installing the Veeam Agent for Linux v6.

Independent of which module you use, Veeam uses 2 distribution techniques, depending on the Linux distribution.

DKMS: This is the main way of distributing the snapshot kernel module. In this scenario, Veeam ships the source code of the driver via deb or rpm package. Whenever a new Kernel is installed, DKMS will automatically recompile the required kernel module providing a module that is 100% compatible with the new kernel. The downside to this flexibility is that this approach requires DKMS and all of it’s dependencies like linux headers and compiler tools to be installed. For example, you can only use DKMS packages for Ubuntu as there are not prebuilt packages for it available.
Prebuilt: The Veeam Agent can also ship precompiled Linux kernel modules. These are ready to be used without compiling and thus do not dependent on DKMS and all of the dependencies of DKMS. The downside is that if a distribution vendor releases a new kernel, the current package might not be up-to-date instantly. This is why in general, we ship these for stable Enterprise Linux distribution such as RHEL or SLES. For a full list, consult the the helpcenter.

In general, prebuilt packages use the naming kmod (e.g. kmod-blksnap-6.0.x.x.el9.x86_64.rpm in the RHEL 9 repository) or kmp (e.g. blksnap-kmp-default-6.0.x.x-sles15.5.x86_64.rpm in the SLES 15 SP5 repository) as a package name and mostly have indication for which platform they are compiled (e.g. x86_64 or i386). DKMS packages can mostly be identified with noarch in the name as they are source code and not compiled for any specific platform yet. If you install from Veeam Backup & Replication centrally, Veeam Backup & Replication will choose the best matching method. If you install manually, the Veeam package will typically prefer prebuilt packages if they are available.

If installation of 3rd party kernel modules is for some reason not possible, there is an option to use the veeam-nosnap package which does not have any dependency on any of the kernel modules packages.

See the System Requirements for Nosnap Veeam Agent for Linux in the user guide for more details and limitations.

Creating bare metal recovery media

To perform a bare metal recovery, the required recovery media cannot be created within the VBR console. Instead, you can either

Patch the default recovery media on the agent computer directly after installation, by following the instructions outlined here, or
Start the creation process on the agent computer by entering the agent’s UI and pressing M to get to the “Miscellaneous” submenu and select “Patch Recovery Media”, or
Download a pre-built recovery ISO file from Veeam’s website: On this page, select your Linux distribution/architecture and you will find corresponding recovery media ISO files in the “Additional Downloads” section.

It is highly recommened to test your Bare Metal Recovery Media after you have created it to verify if it can detect your IO devices (Network, Disk, etc.). This will avoid any unnecessary troubleshooting during a real restore.

Secure Boot

UEFI Secure Boot allows the system to load only Linux kernels and Linux kernel modules that are signed by a known vendor. Typically the kernel and the modules are signed by you distribution, eg RedHat, Debian, Canonical, Suse, … and load without problems after installation.

To verify if your system is using Secure Boot, execute

$ sudo mokutil --sb-state

If it is enabled, it should respond “SecureBoot enabled”

When you use blksnap or veeamsnap, you use a kernel module that is not signed by the distribution. If you use Secure boot, you should make sure the public key that corresponds with the private key that was used to sign the kernel modules after compilation is known by your system. If it is unknown, then the kernel modules will not be able to load (the kernel will refuse to load them). If you want to read more about how Secure Boot and Linux work, refer to the UEFI Secure Boot / SHIM page of Debian.

It is important to understand that this public key and private key depends on who has compiled and signed the code.

If you are using prebuilt kernels, the modules are compiled and signed by Veeam R&D. The private key is kept secret by Veeam but the public key is distributed using the ueficert package. You should install and enroll the veeam public key BEFORE you run Veeam for the first time. Please refer to the user guide.
If you are using DKMS, DKMS can sign the packages after they are compiled with private key generated locally. The public key needs to be enrolled into the system. It is important to understand that in this case, the key enrolled in UEFI is not Veeam Specific but specific to your system and DKMS. It also means that it depends on your distribution. Eg if you install blksnap on Ubuntu, it will trigger mokutil automatically as explained in the user guide and in the Ubuntu DKMS documentation.

Enrolling the key requires rebooting the system and approving the new key during a boot cycle. Therefore, if you use Secure Boot, you have to plan for downtime.

Finally, Secure Boot also has an impact on the recovery media (for Bare Metal Recovery)

If you have Secure Boot enabled, you can only use the prebuilt iso downloaded from the Veeam website. Custom recovery media are not supported and will not load.
Alternatively, you can disable Secure Boot if you need to use custom drivers. It is critical that you test your recovery media beforehand to make sure you do not miss your SLA during the recovery process as mentioned in the previous section.

Veeam Agent for Linux

Veeam snapshot kernel module

Creating bare metal recovery media

Secure Boot

References