Refreshing Storage For Backup? Time For A Change

By Don Foster

It’s comical how habits are formed through time

Johnny was cooking with his mother over the holidays.  As a curious 7-year-old, he decided he wanted to learn how to cook the family roast. He watched closely to everything his mother did – pulling out the spices, selecting the right pans and, of course, the preparation of the roast. He took everything in and listened closely. “OK,” said his mother, “the first thing we need to do is cut the end off the roast.”

His mother cut off the end and tossed it in the garbage.

Seemed odd. Why waste the perfectly good end of the roast? He spoke up. “Mom, why are we cutting off the end of the roast first?” His mother had no good answer and simply said, “Well, that’s how your grandfather always taught me. He is in the other room. Why don’t you go ask him. I think it is for the flavor and spices?!” She shrugged.

So off to the living room Johnny went. “Hey grandpa, mom says the first thing you do when cooking a roast is to cut off the end, for better flavor and stuff. Why?”  His grandpa turned from his crossword puzzle with a confused look. He said, “Flavor?! Why is your mother cutting off the end of a perfectly good roast? The only reason I did that was because my oven was too small!”

Sometimes a change in how you do things is for the better.

Software-defined storage (SDS) is proving to be a key driver for change in IT. This is one of the many reasons we acquired Hedvig and the Hedvig Distributed Storage Platform. As a transformational technology, Hedvig can abstract away underlying hardware infrastructure, provide storage services in a scale-out model, consolidate protocols, deliver site resiliency and perform all maintenance non-disruptively. We see the IT breakthroughs customers can achieve through the transformation and integration of storage and data management.

In fact, we just received the 2020 Breakthrough Award as the Data Transformation Solution of the Year.

We have been working hard to integrate the world’s most comprehensive data platform together with an innovative and leading SDS platform. Integrating Hedvig storage with your typical Commvault solutions creates the next dimension of Intelligent Data Management, providing more automation, control and access to your data. Last quarter, we released the next major milestone for our integrated vision, and I want to double click into the core details of what we have achieved within our product portfolio and what it means for you.

Hedvig provides a robust set of storage capabilities throughout the distributed platform. The ability to provision storage with specific attributes based on workload requirements provides the flexibility customers need. Because it is software defined, you can run the platform using off the shelf x86 components and scale-out in a predictable fashion. Need 50 or 100 TBs more usable capacity, add another node and do it non-disruptively. It is that easy, which is important when you look at the backup and recovery environment. This is one area where space can get consumed quickly and scale-out storage is a MUST.

Enter Commvault and Hedvig together.

Commvault Complete™ Backup & Recovery now fully supports the Hedvig Distributed Storage Platform as a backup target. Feature Release 11.19 together with Hedvig Release 3.9.9.8 will allow you to execute backups directly to the scale-out software-defined platform with all of the bells and whistles of Commvault Complete Backup & Recovery enabled. Specific integration work and development was done to ensure the best user experience and optimized interaction to keep your jobs running fast and your restores even faster.

How? Well, we streamlined the Hedvig NFS offering specifically for Commvault to ensure the underlying Hedvig storage cluster would handle Commvault workload files in an optimized manner. We quickly recognized that many of the file and storage management functions that run via the traditional NFS implementations could use an intelligent transformation to eliminate operational overhead.  Our ability to integrate the storage technology together with Commvault’s data management portfolio enabled an outside of the box approach to deliver an elegant and efficient solution. By implementing REST-based APIs, Commvault Media Agents and the Hedvig storage system are able to talk natively to handle the necessary data management functions and to allow less overheard to improve performance and scale.

As a part of this integration, we also put our heads together to optimize how calls to the Hedvig storage platform are handled. Instead of forcing the traditional directory listing calls, we improved upon the solution by using these REST-based APIs to create simple and fast file lookups across the distributed platform. This also meant a greatly enhanced function for retrieving computed sizes of the disk library, its volumes, sub directories and files. Hedvig is innately Commvault-aware with the ability to store additional metadata to optimize Commvault processes like pruning and cleanup operations. All of this together with the benefits of the Hedvig Distributed Storage Platform means a winning combination for refreshing your backup storage.

So what do you need to leverage Hedvig as target storage behind your Commvault Complete Backup & Recovery solution?  Let’s go through a few installation requirements:

  1. A Commvault Media Agent(s) running Linux
  2. A 3 node or larger Hedvig Distributed Storage Platform Cluster
  3. A Hedvig Storage Proxy enabled on each of the Commvault Media Agents

Since we are using Hedvig as a target, we need to ensure that each Media Agent leveraging this storage has a storage proxy loaded. This means no single points of failure, and direct native communication to the storage platform. We need Linux as the operating system of choice as Hedvig will provide storage as an NFS export to these servers (physical or virtual). As a part of the integration work, enhancements have been completed to ensure an optimized experience is available over the data management lifecycle (backup, restore, mount, retain, prune).

Presenting Hedvig Storage to Commvault Media Agents is simple through virtual disks in Cluster Watch. Clicking on “Add disk” from inside Cluster Watch brings you here:

Name the virtual disk, set the size and ensure NFS and Backup are selected for the disk and Workload type. Leave deduplication and encryption off (we recommend letting Commvault handle this inline) and select Replication or Erasure Coding “RS_4_2.” Replication will allow for RF3 distribution across sites and the cloud for the data stored. Erasure coding will use Reed Solomon erasure coding algorithms to provide data resiliency within the cluster. Once you have made your choice click on RUN.

Now you must provide access and export the disk to the Commvault Media Agents through “Manage Exports” in Cluster Watch.  A simple command on each Media Agent will then persistently mount the storage.

          # /usr/local/hedvig/scripts/mount.sh <vdiskname>

At this point you are ready to add the storage in Commvault Command Center™. Just like that, you have moved away from tradition scale-up storage that requires downtime and maintenance windows for upkeep, not to mention performance degradation when drives fail, to a scaleable, self-healing, non-disruptive and predictable storage platform that can grow as you need more capacity. This fits perfectly into your current Commvault configuration and should be in your capacity plans as your older traditional backup storage comes up for refresh. At a primary site, secondary site, or both – this direction adds to your overall IT flexibility and will get your feet wet with software-defined storage.

By the way, I know not all of you reading this JUST use Commvault. As hard as it is to admit, I know it is true. But guess what? We can support Veeam, Veritas, HYCU and many others with our Hedvig Distributed Storage Platform with the same outcomes and benefits. So if you DO use multiple vendors for data management, at least you could consolidate storage platforms.

Don’t cut the end off of your holiday roast just because that is what you have always done. Ovens are larger, and the technology has gotten a lot better.

Don Foster is Commvault’s VP, Storage Solutions.