For the last few weeks, I’ve been helping a customer migrate from one datacenter to another. The new datacenter contains a shiny new vBlock and as part of the deal, a Data Domain (DD) 6300 with 73TB of available capacity was installed to serve as a Veeam backup repository. This is a nice bonus as we’re migrating away from an 8TB IOmega Storcenter server that was bursting at the seams and a Windows NAS server with approximately 7TB of space that was installed to relieve the space crunch of the IOmega. Additionally, there are VMs with large virtual disks functioning as backup servers with local repositories so there are Veeam backup servers and repositories all over the place….and as you might suspect, the Veeam/backup documentation in regards to servers and their functions is limited in that one server I was told is a backup server no longer exists. How can this be? And if a ticket came in to restore a file or a VM, what would that look like? Do I need multiple local repositories?
I was tasked with consolidating the various Veeam backup servers to one and to configure all backup jobs to use the DD as a deduplicated storage appliance backup repository and I’ve done that. And to be honest, the DD has performed beyond my expectations, the backups are fast and I’ve not had a single failure since moving the backups to it, no leftover snapshots, etc. though those may come in time. The biggest “problem” so far, is that the DD dedupe and compression has been so effective. In fact, after moving all backup jobs to the DD repository, I’m only using 5TB of the 73TB capacity. Thus, this customer has paid for 68TB of unused capacity, perhaps capacity they’ll never use. Granted, to use this space they could set each backup job to retain 250 restore points or they could deploy and then backup 800% more VMs. Perhaps the excess capacity helps them to rest easy knowing that their Veeam backup repository will never run out of space again, and can you put a price on this kind of peace of mind? I bet somebody in the Accounting department can….
But if you think about that for a moment, is purchasing a 73TB DD system to support 5TB worth of backup data the best way to experience the comfort of knowing your backup repository won’t run out of space? Is there a better way to design Veeam backup repositories so as to minimize the amount of wasted space? Is there a means to allow organizations to purchase “smaller” local repositories while also providing additional capacity when you need it without having to purchase more hardware?
The Veeam Cloud/Capacity Tier
The answer to that question can be “Yes” via the Veeam Cloud/Capacity Tier. During his Veeam Cloud Tier presentation at Cloud Field Day 5, Anthony Spiteri (twitter: @anthonyspiteri) stated that the Cloud Tier capabilities now available in Veeam 9.5 “fundamentally changes the way you architect the local repository because most of the data CAN sit on object/cloud storage.”
But what is the Cloud Tier? The Cloud Tier is simply a capability that allows a scale-out backup repository (SOBR) to use object-based storage, such as AWS S3 buckets, as SOBR extents to provide virtually unlimited storage capacity and on-demand, pay-as-you-use flexibility.
Using the Cloud Tier capability gives you options. For example, you could purchase a “smaller” hardware appliance to serve as the local repository or perhaps you could even recycle some of the equipment you already have and possibly avoid buying anything altogether. Why? Because the cloud tier allows you to minimize the amount of local space required for backups since the bulk of your backup data will reside on object storage.
In the example I used above, the existing backup retention policies have consumed 5TB of available 73TB of a DD system. Thus this organization is paying for 68TB of space they are not using. Would it have been a better idea to purchase a smaller 5-10TB system for a local repository and then use object storage as needed to minimize wasted space? In this case, many of these decisions and purchases were made long before cloud tier was generally available, but going forward Veeam Cloud Tier will undoubtedly have an impact on the design of local backup repositories and likely the hardware purchased to support them.
A link to Anthony’s Cloud Tier CFD5 is below and I encourage you to watch it to gain more insight into the Veeam’s Cloud Tier capability, but I’d like to spotlight a few points from his presentation:
- Veeam has an agnostic approach to object storage. Though I mentioned AWS S3 earlier, the Cloud Tier will work with Azure Blob Storage, IBM Cloud Object Storage, or other S3-compatible storage.
- Understand that Cloud Tier is a tiering solution, not an archive. Veeam is not sending big chunks of data to object stores, but tiering little blocks.
- Tiering supports intelligent uploading and recovery. When uploading data, Veeam has the smarts to only upload unique blocks to object storage; the same block will not be uploaded twice. Also, when performing restores Veeam will look first to see if the blocks required for a restore exist locally and only downloads blocks from the object store when necessary. This ensures that restores are completed as quickly as possible AND saves on egress charges.
- Cloud Tiering does not create a copy of the data, meaning there is only a single copy of data that is either in the local repository or the object store. Cloud Tiering DOES NOT replace the functionality of backup copy jobs.
- When tiering is enabled, an Offload SOBR job is created and launched every 4 hours. This job is responsible for determining what data can be moved and then starts veeamagent.exe to perform the move.
- There may be the temptation to copy all backup data to the cloud tier but resist the urge. Retain at least some measure of local backups (I’d suggest at least 1 week but it’s your call) to ensure restores are done as quickly, and cheaply, as possible.
Perhaps the best thing about the Cloud Tier is this powerful feature is simple to implement. Whether you’re running out of space in your existing repositories or preparing to design a new Veeam infrastructure, at least explore its capabilities as you attempt optimize your Veeam backup infrastructure.
Anthony Spiteri’s Cloud Tier presentation at CFD5