Commvault IT uses Distributed Storage for Microsoft Hyper-V

Challenge

  • Enable seamless storage upgrades and better data portability for mission-critical applications
  • Provide flexibility to support multiple storage technologies and modern workloads
  • Provide transparent application and services failover to local, remote, or cloud datacenters

Solution

  • Commvault® Distributed Storage
  • Commvault® Backup & Recovery software
  • Microsoft® Hyper-V™ for virtual environment set up as a failover cluster

Result

  • Reduced management overhead/intervention since writes are distributed across locations
  • Displaced traditional storage solution with modern software-defined distributed storage architecture that enables greater resiliency across sites/locations
  • Improved application availability and data portability, including performing a non-disruptive upgrade to Commvault Distributed Storage LIVE during business hours while the applications were running
  • Actively investigating deploying to a second location

Industry

Enterprise data management and protection software

Location

Tinton Falls, NJ

URL

http://commvault.com

In numbers

  • 131TB storage cluster (118TB usable storage)
  • Six industry standard x86 servers deployed as distributed storage cluster
  • Two datacenters: one in Tinton Falls, NJ and one in Chicago, IL (Commvault Distributed Storage initially deployed in NJ datacenter)

Commvault is a leading intelligent data services vendor, enabling innovation with a dependable IT backbone designed for flexibility. Hyper-V serves numerous business-critical workloads such as web services, data warehousing, and intelligent communication platforms.

Server virtualization with Hyper-V enables organizations to deploy a flexible disaster recovery and availability solution. When combined with the benefits of back-end shared storage, these solutions provide transparent application and services failover to local, remote, or cloud data centers, thereby protecting against widespread disasters.

Commvault IT was looking for a solution that allowed them to maintain high availability for their mission-critical virtualized applications, while enabling them to perform non-disruptive upgrades. Additionally, they wanted a solution that gave them flexibility to support multiple storage technologies and modern workloads, including containers. Fortunately, Commvault IT didn’t have to look very far to find a solution that met all their needs.

The previous production setup (prior to Commvault Distributed Storage) included two sites (Tinton Falls, New Jersey and Chicago, Illinois) with NJ as the primary site replicated to Chicago using Hyper-V, Commvault backup software, and SAN storage.

This stack included the following challenges:

  • Management overhead to independently failover the application, network, and storage
  • Upgrades that frequently required taking the storage offline
  • Risk of not meeting the SLA after a disaster

After some investigation, Commvault IT realized that all these challenges could be addressed using Commvault Distributed Storage. The analysis also included evaluating the distributed storage application-aware and declarative data placement architecture, as well as the software-defined storage (SDS) deployment model and performance testing.

Commvault Distributed Storage is a scalable, multi-protocol, distributed storage solution that supports highly available Hyper-V environments. Specifically, Commvault Distributed Storage provides:

  1. A highly resilient platform that can survive a disk, node, rack and/or datacenter failure
  2. Support for “datacenter-aware” replication, with near-instantaneously replication of data to additional sites (private, public, or hosted cloud)
  3. Software-based end-to-end encryption for data in flight and data at rest
  4. A storage services engine for a distributed redundancy model to ensure a full fidelity data stack that protects against single points of failures (SPoF)
  5. Independent and elastic scale of compute and storage when deployed in “Hyperscale” mode, enabling a pay-as-you-grow consumption model

“Using distributed storage, we were able to migrate from our previous solution to Commvault Distributed Storage live during normal business operations, while our applications were still running. We’ve never been able to do that with any other solution.”

– Reza Morakabati, Chief Information Officer

Migration Process

Commvault IT developed a project to migrate one workload to distributed storage at a time starting with the backup environment followed by the Hyper-V environment. The previous environment consisted of a 4-node Hyper-V cluster using NetApp AFF and FAS.

For a systemic migration Commvault IT considered:

  • Prerequisites
  • Live migration with zero downtime
  • Post migration setup and test

Prerequisites

Commvault IT considered the following categories in the pre-migration playbook:

  • Hyper-V Failover Cluster: Commvault IT setup a group of similar Hyper-V nodes as a cluster to deliver HA to the VM, services, and processes if a node goes down. This action is triggered by Failover Cluster Services when needed.
  • Hedvig Proxy: Each Hyper-V host has a one-to-one mapping with a Hedvig proxy. The proxy intelligently routes I/O to the backend storage and optimizes performance to the application host. For full redundancy, proxies are paired with transparent failover. Since each proxy is stateless, this is a high-performance operation yielding read/write access to the associated storage with no data integrity risk.
  • Shared storage: “Hedvig Storage Services” is a patented distributed systems engine that enhances commodity servers to function as highly available, resilient storage assets. Unlike traditional arrays, distributed storage provides application-aware shared storage with tunable replication policies and advanced enterprise storage services, high storage efficiency, and high performance per application.    

Commvault software-defined distributed storage setup:

  • Commvault Distributed Storage can run on bare metal, VMs, and containers.

Live migration with zero downtime

The migration was carried out with no downtime during business hours, which was was a huge win according to Reza Morakabati, Chief Information Officer. “Using distributed storage, we were able to migrate from our previous solution to Commvault Distributed Storage live during normal business operations, while our applications were still running! We’ve never been able to do that with any other solution.” To enable this, Commvault IT used:

  • Deployment Server: It is a light bastion server used to automate cluster deployment and to conduct physical and logical moves/adds/changes to the underlying cluster topology. The deployment server was deployed as a small virtual machine and requires a network connection that is routable to the storage network used by the storage cluster. 
  • Storage Setup: The proxies reside on Linux using available SSDs as edge cache to optimize performance. Two ethernet interfaces were configured to run as a pair for failover. 
  • The Distributed Systems Engine: It is deployed on six Dell EMC PowerEdge R740 servers with 3.84TB SSD cache and 12 drives of 8TB storage. Each storage server is part of global processes and cluster operations but otherwise operates as a shared nothing architecture. 
  • Hyper-V setup: setup as failover cluster, with each Hyper-V host accessing the storage via its local proxy.

Post migration setup and test included validating and shut down with Failover Cluster Manager, verifying legacy storage was released and conducting performance testing.

Conclusion

Deploying this solution delivered:

  • Reduced management overhead since writes are distributed across locations.
  • Higher availability and better application portability for mission-critical applications, including performing a nondisruptive upgrade – while applications were actively being used – during normal business hours.
  • A modern software-defined distributed storage architecture that supports multiple storage protocols, application-aware storage, and additional workload support.

Based on the early results, Commvault IT is actively investigating expanding the footprint to a secondary datacenter and/or public cloud for secondary and tertiary data protection. In addition, other workloads are being evaluated to be added to the storage cluster to improve economics, simplify management, and onboard new applications to serve as a flexible foundation.

Microsoft Hyper-V

With Commvault, you can extend that simplicity to data management and recovery with our proven, Hyper-V-integrated solution.