Veeam and Backblaze B2 – Testing and Tuning

 Phase 5 – Testing and Tuning
So running the Veeam job, we can see that we are only writing backup data to a local SMB share so, in theory, it should complete successfully just like any other normal job.

We can monitor the status of CloudSync from the Synology management interface, unfortunately, the status/progress information offered in CloudSync appears to be quite limited. The status/progress appears to be either, in-progress or completed.
Below is the incomplete summary, note: no progress or detail regarding the amount of data remaining to be uploaded.
We can, however, see a history of completed files.
For those interested, this is how it looks when it is up to date.
Scheduling
I recommend configuring scheduling to avoid syncing during backup job operations.
The first test of the backup job resulted in the Synology Cloud Sync uploading files that were still being written to by Veeam, this is an issue because these have been uploaded before Veeam has finished with them, meaning the file is likely incomplete or corrupt. Because of this we need to schedule CloudSync not to upload until after the backup job is completed.
Tuning
Some parameters we can look at to improve the performance would be the Part Size, as covered earlier in this article, we can increase the size on reliable links to improve performance. However, this should be set low if your link experiences frequent disconnects to avoid having to upload large amounts of data from scratch again.
Traffic throttling can also be configured by defining the max upload and download rate in KB/s.
Browsing files in B2
Browsing and retrieving files is as simple as you would expect with your typical object storage service, files can be browsed via the web interface and file versioning can be easily accessed.
B2 Snapshots
Creating snapshots of files and folders in B2 adds an extra level of protection against ransomware. In the example below, I have selected the folder created by Veeam for the backup job and initiated a B2 snapshot. I’ve searched high and low through the web interface but I couldn’t find a way to schedule the snaphots, I’m confident they can be triggered through scripts leveraging the APIs though.
What is cool though is that snapshots can be retrieved using several options, either by download, USB Flash Drive or even a USB Hard Drive. It’s always useful to have extra options for recovery large amounts of data especially if WAN links are slow.
File Lifecycle Management

By default, each Backblaze B2 bucket will keep all versions of all files forever, this is not a bad strategy but you may end up paying for storing backup data outside your retention requirements.

Because of the way Backblaze B2 works, when the same file is uploaded more than once to the same bucket, Backblaze B2 automatically keep both versions of the file but will “hide” any older versions from the B2 Web GUI. You can tell if the same file has been uploaded multiple times a (2) or any other number closely follows at the end of the file name when browsing files via the B2 web interface.

Using the B2 lifecycle rule, we can define how long we wish to keep file versions that are not the current version. File versions are be automatically hidden whenever a newer file with the same name is uploaded. These ‘hidden’ files are simply the older version of a file that has been replaced by a newer copy. Using the B2 lifecycle settings we can define our online retention outside of Veeam for how long we wish to keep our backup files in the bucket regardless if they have been overwritten or deleted from the Veeam repository.

  • ‘Days Till Hide’ defines how long to keep file versions that are not the current version.
  • ‘Days Till Delete’ defines how long to keep that file version until it is deleted.

When a Veeam backup job runs, it creates a backup file based on the following properties, %JOBNAME%,%DATE%,%TIME%. This results in essentially a unique file for each restore point for each job created by that Veeam BnR server. Meaning when we leverage B2 using Cloud Sync, we should find that backup files avoid any versioning, this includes .VBK, VBI and .VBR file extensions. The .VBM file, which only contains the %JOBNAME% in the filename, on the other hand, will be versioned several times as that file is constantly overwritten each time the backup job runs. This means unless we define a lifecycle rule for the .VBM file it could become very cluttered.

So for a file like the .VBM file we can create a lifecycle rule that keeps the old version for 30 days, and then deletes them:  

Regarding our .vbk files and .vbi files, as these are unique file names being uploaded we won’t see any file versioning occur. If no file versioning occurs then by default these files will remain forever in our B2 bucket. If we want to eventually purge the old restore points to keep our data consumption fees lower then we’d need to configure the following based the offsite retention requirements.

For me, I’m happy for 120 days for my incremental and 360 days for my fulls.

Lifecycle rules are applied once a day. Rules will be applied at the next daily run after that number of days has passed.

 

We have tested and tuned your buckets, congratulations, on to the summary.

4 thoughts on “Veeam and Backblaze B2 – Testing and Tuning

  1. Jay B Weinshenker

    Really liked the series of articles – did want to point out a typo on this page

    We can monitor the status of GoodSync from the Synology management int

    You mean CloudSync I believe.

    Reply
  2. Jay

    Hi, thanks for this really helpful guide. Is it possible for you to extend on the last check about Lifecycle Settings?

    It is not clear to me what file extensions to keep to comply with the restore points I have setup on the backup job.

    Thank you.

    Reply
    1. admin Post author

      Hey Jay,

      Thanks for your comment, I read over the lifecycle setting section and I agree it wasn’t very clear.
      So I’ve spent a bit of time this morning rewriting and expanding this section so hopefully, it makes a bit more sense now.

      Cheers

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *