Introduction

Today we are backing up on-premises vSphere VMs with Veeam direct to a Minio S3 bucket, then restoring that VM to a VMware Cloud Director ORG in the cloud. Once complete, we will then back that VM up direct to a Minio S3 bucket and recover it back to the on-premises vSphere deployment.

All this is powered by Veeam v12, Minio, and of course VMware vSphere & VMware Cloud Director

Let’s say you have an on-premises vSphere environment. You need a solid offsite backup option. You also want to be able to recover your VMs to a cloud provider in case you run into a disaster at your primary site. How can you recover a VM to a new production environment running in the cloud for days/weeks/months in the event you run into a disaster, but get your data *back* to your on-premises infrastructure? Ideally, you would choose to replicate your VMs, but maybe your budget doesn’t allow for that. This would be the next best thing – keeping in mind your RTOs are not going to match a full replication setup.

Disclaimer – Some Object Storage Services may charge data Ingress & Egress charges. Beware of ‘hidden’ charges such as these.

Add your S3 Storage

First things first, you want to connect your on-premises VBR server to your Minio bucket.

Backup Infrastructure -> Add Repository -> Object Storage

Chose S3 Compatible from the next screen.

Name the bucket appropriate to your needs, and enter the data provided to you by your object storage vendor. In this case, we’re using Minio S3 provided by Stage2Data.com. Make sure you add the credentials as provided.

Keep track of the bucket and folder you are using here – you’ll need that later!

The rest of the steps are pretty straightforward – I used the defaults as presented by Veeam.

Create a Backup

Creating a backup directly to Minio S3 is the same as creating a backup to any other block storage device, so I won’t go over it in detail. Just make sure you select the repository you created above!

However, one thing I want to mention is block sizes! Block sizes on Minio matter, not so much for performance – Minio is a rockstar when it comes to performance. That said, if you administrate your own Minio cluster, you will eventually need to replace a failed disk. Healing can be negatively impacted by block sizes (eg: it will take a Long Time ™ to heal a failed drive the more smaller blocks you have).

While Minio does support Veeam’s SOSAPI, Veeam does not yet do anything with the block size recommendations that object storage vendors such as Minio may present through Veeam’s own SOSAPI.

Going off on a Veeam SOSAPI Tangent

To explain further, Veeam has an SOSAPI that object storage vendors can utilize to provide Veeam with information not regularly returned by object storage. Information such as Capacity, Free Space, and recommended block sizes! However, utilizing that information to control Veeam backup jobs default block sizes are not yet implemented. Hence, Minio tells Veeam what size blocks to use, Veeam just doesn’t do anything with that information… yet. Hopefully it will be implemented soon!

Until then, increase the block sizes on your Veeam backup jobs that go offsite to Minio S3. Your vendor (and if that’s you) will thank you.

SOSAPI Tangent Ended

Under the Storage section of your Veeam backup job, select Advanced.

Set your Storage optimization to 4MB. Note: It is possible to unlock 8MB blocks through a registry change, but Minio recommends a 4MB block with their SOSAPI implementation, so that’s what we’ll use. I’ll elaborate further on block sizes with Veeam jobs in another post.

Once at least one full backup has been completed, you can now proceed to recover these backups to a cloud provider. Note: You’ll need your cloud provider’s support on these steps since this involves a lot of back-end work to get the VM recovered to vSphere and subsequently imported to VMware Cloud Director.

Recover your Backup

As mentioned above, these are steps your service provider will need to perform. If you are working with a company like Stage2Data they will ensure they perform these steps to get you back up and running quickly!

Deploy (or utilize an existing) Veeam Backup and Replication Server that has the cloud provider’s VMware Cloud Director already added to its inventory.

Add the Minio bucket that was used above to create the backups. Your Minio provider can either create a new user to access your Minio bucket, or they can re-use your existing user credentials.

Note, when attaching to an existing Minio bucket, Veeam will pop up a warning.

You want to select “Yes” here. The recovery VBR will take ownership of that Minio bucket so that no other Veeam server can use it – this is to prevent data corruption. However, in this case, we know the original VBR won’t be writing to that bucket anymore because the primary site is suffering a disaster. Hence, we don’t need to worry, and when we bring the original VBR back up, we can take ownership of the Minio bucket back.

Recovery to VCD

From here you will run a VM recovery to the Cloud vSphere environment. Unfortunately, Veeam does not yet support a restore from a vSphere VM direct to VMware Cloud Director, so we need to do an intermediate jump to vSphere. Just as with a backup, I won’t provide a detailed walkthrough of the steps to recover a VM, but it’s the same as you would do for any other VM. Note: for the purpose of this article I didn’t attempt to do an Instant VM recovery, I did a full VM restore.

Once the restore completes, you now have a VM available in the Cloud Provider’s vSphere environment. What do you do to get the customer access to their VM? VMware Cloud Director of course!

Log into VMware Cloud Director, select the Org VDC you want to import the VM into and open the tenant portal.

From the tenant portal, select your vApps and select “Import from vCenter”.

Locate the VM you restored in the steps above and import this into your VMware Cloud Director tenancy. Feel free to select the option to Move Virtual Machine (delete source). You won’t need the vSphere version after its restored to VCD.

Post Restore Steps

Once the import is complete, you now have a VM that’s ready to go. However, you will likely need to fix the networks on the VM itself to match your ORG VDC networks. Keep in mind, this will get you running, I never said it would get you running fast!

Ideally, Veeam will improve upon its restore process to allow a person to recover a vSphere VM directly to Cloud Director. That can cut down the time required to get a VM up and running significantly. Veeam has had this request before and is “considering it for future improvements”. Could show up in the next release or it could show up never. C’est la vie.

Once the networking is fixed, you are free to power on the VM and get cracking! Obviously, you’ll want to do these steps for as many VMs as you need to recover from, but the steps are there!

Your disaster has been recovered from. Your VMs are live. Life is good again.

Oh nice. You’ve recovered your on-premises servers. Your environment is ready again… but wait, you’ve been running in the cloud all this time. What do you do?

Back Again

At this point, you have a vApp and a VM. We need to create a backup job to take that VM that was imported into vSphere and back it up back into the Minio bucket.

These steps are fairly similar to creating any other VM backup.

Unfortunately, VMware will create a new unique ID for the VM. Hence, backing up that VM and mapping to the existing backup job will not have the desired effect of updating the existing VM backups with the new data. Just to keep things clean I’d suggest creating a new backup job and target the VCD VM. You can restore a VCD VM backup direct to vSphere, the only difference is that Veeam keeps the VCD metadata along with the backups.

Once this backup completes, you can repeat the same steps above – attach your (new or restored) VBR back to your Minio bucket, rescan the repository to reveal all backup jobs, then perform a restore back to your on-premises environment.

Conclusion

Veeam v12 combined with Minio Object Storage can be a great tool to utilize to ensure your backups are safe and secure offsite. With a little extra work, you can consume Minio to provide that extra Disaster Recovery safety by using Minio’s robust, performant storage as an additional stepping stone in your disaster recovery plan.

Keep in mind I would always recommend following the industry standard 3-2-1 backup rule. While this article showcases going straight to object storage, the same principle can be followed by using backup copy jobs instead.