I was fortunate enough to attend VeeamON Forum in Sydney last week, the company I work for (Data#3) actually sponsored a booth so I was armed with a scanner and had the task of talking to as many potential customers as possible. Usually, I would shy away from such a task but given the subject of the day was Veeam I had a great time. I even had one person mention he knew of this blog!
During the event, I had a great chat with Vish Venkatesh (short for Vishwajeeth) from Veeam also based in Sydney. Vish spent a year and a half as a Support Engineer before changing roles to an SE so I got a chance to ask about the inner workings of Veeam support. Continue reading
Something that all Veeam administrators should consider is how secure the underlying servers running your Veeam software really are. To help improve security I always try and run through a few recommendations with each Veeam administrator I work with,
- Inbound connectivity to backup servers from the Internet must not be allowed (3389 anyone?)
- Any accounts used for RDP access must not have Local Administrator privileges on jump servers, and you must never use the saved credentials functionality for RDP access or any other remote console connections.
- Ensure timely guest OS updates on backup infrastructure servers
A good resource for keeping up to date on Veeam security recommendations is here. I like to check it out every 3-6 months to ensure I’m still making the right recommendations to my customers.
One other thing I like to recommend in addition to the best practices above is enabling 2FA (Two-Factor Authentication) for all login sessions to underlying servers running Veeam components such as the VBR server, proxies and especially repositories. With 2FA, even if an attacker illegally acquires the correct username and password, the attacker is also required to gain access to the device used to receive the 2FA verification code. Often this device is a mobile phone or a security token which can easily be disabled if lost or stolen.
It must be noted that 2FA for Veeam consoles is currently not possible (it is a heavily requested feature though) and even with 2FA for login sessions into any Veeam servers there is still a risk that an attacker can access Veeam infrastructure via a Veeam Console running from another machine. This is why off-site/offline backups are so so critical in today’s world of ransomware. Leveraging Veeam Cloud Connect Backup with it’s Insider Protection feature is a great way to easily protect against this kind of risk.
This post will go into detail on how to quickly and easily and enable 2FA for RDP and local logon sessions connecting to your Veeam server.
A customer recently reached out to me with the issue below, while I hadn’t seen this issue before I thought I would check to see what I could dig up before they opened a ticket with Veeam support. “Error: Failed to call RPC function ‘StartAgent’: Timed out requesting agent port for client sessions.”
Veeam KB 1922 to the rescue, the cause of this issue is the ‘configuration of a Windows server within the Veeam console being set to have a limited number of ports to use‘ which thankfully can be resolved quite easily. To resolve simply go to the ‘Backup Infrastructure’ section in your VBR (Veeam Backup and Replication) console, go to the properties of any Windows servers that are being used by the job that is failing. So in this customers case, we can start with the backup proxy, then the backup repository, then if the problem still persists we can increase the port range on the VBR server as well. Once the port range is increased we simply click OK to apply the changes, I recommended we start with a relatively small number of ports (50) and increase if the problem still persists.
I haven’t figured out why in this customers case they encountered this port exhaustion issue, I find it curious as I’ve worked on much larger Veeam deployments before that didn’t encounter this issue. Ill need to perform some investigation and report back here once I learn more.
**UPDATE** Restarting the VBR server has resolved the issue without having to increase the port range. If it continues to happen we’ll look at increasing the port range but until then the default settings are good to stay.
So my first go at the VMCE-ADO exam was way back at New Orleans during VeeamON 2017. That experience could quite easily be described as an A+ for attendance but an F for effort, it was a textbook case of the ‘7 Ps’ and I walked away with a measly 50% result. Thankfully I was fortunate enough to be using an exam voucher which included a free reattempt so I thought why not give it a go while I can. That exam showed me first hand just how tough it really is but more importantly, I saw what it was going to take in regards to study to make sure I was really ready for the next attempt.
Unfortunately, it has been over a year since VeeamON 2017, in fact, VeeamON 2018 has already come and gone, yet I couldn’t delay sitting the exam any longer as the reattempt voucher was just about to expire. Timing was not the best as we just sold/purchased/moved houses 3 weeks prior and it was my sons 3rd birthday 2 weeks prior. I think I gained a few grey hairs this month… Nevertheless, the exam was booked and I couldn’t reschedule it without getting out my credit card.
One of the lesser known features of Veeam ONE is its ability to divide the virtual environment into various groupings and categories, essentially creating a view that is easier to digest from a business standpoint. This view can be valuable when you consider that most tools we would use such as vSphere client or SCVMM/Failover Cluster Manager are often configured to present information and data, for say, a more a technical perspective, something which might not be relevant or even make a whole lot of sense for all business stakeholders.
Both Veeam ONE Monitor and Veeam ONE Reporter will use this categorization provided by Business View. Veeam ONE Reporter, enables us to generate reports and build dashboards based on the categorisation created in Veeam ONE Business View. While Veeam ONE Monitor, we can monitor Veeam ONE Business View groups of VMs, hosts, clusters and storage objects.
By leveraging Veeam ONE Business View to group and categorise these objects into a hierarchy that makes more sense from a non-technical perspective such as office departments, projects, SLAs and much more we can easily review and report on resource allocation and utilisation based on these groupings.
Now that part is out of the way, I wanted to demonstrate how Veeam ONE Business View can be configured to help automate this process. We can configure set & forget rules and policies that organise objects into these group and categories. I’ve written this article to dive further into how one can configure Veeam ONE to categorise our infrastructure.
It’s a message every IT manager dreads.
‘Your personal files are encrypted by CTB-Locker. To decrypt the files, you need to pay 3 bitcoin.’
Yet, unfortunately, getting locked out of your company’s own data – and then being expected to pay a ransom to get it back – is becoming more common as cybercriminals get craftier. Like pesky bed bugs that have become immune to deterrents, ransomware attacks such as CryptoLocker, CryptoWall, Locky, TorrentLocker and Virlock are constantly evolving to sneak past all the new defences that IT security experts are busy building up.
Recently I had the opportunity to deploy Veeam B&R utilising Cloud Connect Replication for a customer to replace their existing DR solution. We were running into an issue with a couple replication jobs that were sitting at 99% for longer than I would expect, in some cases for several hours.
I wasn’t sure what it was doing as there was no network traffic, CPU or even disk usage on the on the source that could be detected. The Veeam job showed no tasks currently underway and I didn’t want to speak to the Service Provider to check their end until I had verified everything was working as expected at the source so I kept digging.
Rick Vanover has posted on the official Veeam blog regarding the opening of the Veeam Vanguard nominations for 2018. This will be the third year of the Vanguard program for which the recipients receive a variety of awesome benefits, one of which is a trip to VeeamON.
Some Vanguards are bloggers, some are active on the Veeam Forums, some are active on Spiceworks sharing a lot of Veeam-specific information or even on the Veeam subreddit, the list goes on for all of the ways Vanguards have engaged with the Veeam community.
If you know anyone that you’d like to nominate, perhaps yourself even, I strongly recommend giving it a go.
Nominations will be accepted until Friday, Dec. 29. You can go through the nomination process here.
Phase 1 – Create the Backblaze B2 Bucket
Phase 2 – Install and Configure Synology CloudSync
Phase 3 – Configure Veeam Backup Repository
Phase 4 – Create the Veeam Backup Job
Phase 5 – Testing and Tuning
B2 Cloud Storage is an object storage service offered by Backblaze that enables users and organisations to upload files to their heart’s content billed on a per monthly basis using a pay for what you consume model. Backblaze has evolved this object storage service ‘B2’ out of the already successful $5 a month unlimited backup plan which was built from the ground up using Storage Pods. Storage pods are designed in-house by Backblaze, leveraging consumer grade hardware and hard drives in a purpose-built chassis designed to minimise costs, reduce footprint and yield the best dollar per GB possible. For example, using 4TB drives, they can achieve a cost per GB as low as $0.036.
These Backblaze pods, which are now up to revision 6, are literally filled to the brim with hard drives, over 60 of them in fact in a 4U chassis. I recommend that you go and check out more on these awesome units here.
So, Backblaze takes these Storage Pods a step further for B2, by grouping 20 at a time into a Backblaze Vault it enables them to optimise reliability and durability of the entire system.
Phase 2 – Install and Configure Synology CloudSync
Ok, we have created a B2 bucket and we are now ready to configure our Synology NAS.
Now, in my case, I am just reusing a previously configured shared folder which is fine for my homelab testing so I’ll be going straight into installing and configuring CloudSync with B2. However, it is recommended to create a new shared folder dedicated for storing Veeam backup files and lock it down with authentication anytime you are deploying into production.
REMEMBER: It is important to size your volume correctly so that it can handle your retention policy capacity and performance requirements.
So let’s get started, first we need to install the Synology CloudSync Package, this will allow integration with Backblaze B2. During the installation, it will ask where you would like the packaged to be installed, I just picked ‘volume 1’ as that is where my other packages have also been installed.