- Aggressive organizational growth driven by acquisitions led to isolated data silos and disparate backups
- Enterprise deployment of Epic EHR system required responsive, seamless data protection
- Automated backup and recovery of converged environment with centralized management and reporting was needed to simplify overall data management
- Highest levels of data integrity were required to ensure complete regulatory compliance
- Commvault software with integrated backup, recovery, Commvault IntelliSnap snapshot management, virtual server protection, enterprise search and eDiscovery
- Commvault’s world-class data protection architected from the ground up to automate and ease all aspects of backup and recovery
- Commvault software outperformed other systems in backing up Epic EHR data and recovering entire file systems and databases to qualify for Epic maintenance discounts
- Commvault fuels DR success, with 15-minute failover to alternate site and fully restored production services in 30 minutes
- Maintaining multiple levels of backup with Commvault ensures expedited recoveries without impacting any clinical, patient or employee data
- Built-in search provides eDiscovery capabilities across file systems, ensuring comprehensive regulatory compliance
Prime Healthcare prescribes Commvault to protect critical clinical applications and patient records while ensuring heightened data security
Prime Healthcare is an award-winning hospital management company that operates 44 acute care hospitals with more than 7,700 patient beds across 14 states. With hospitals that are routinely recognized among the 100 Top Hospitals in the nation by Truven Health Analytics, Prime Healthcare is the largest for-profit operator of hospitals in California and one of the five largest and fastest growing for-profit hospital systems in the nation. As part of its mission to save hospitals and improve community healthcare, Prime continually acquires financially distressed hospitals and restores operations. Founded in 2001, the Ontario, Calif.-based organization has never closed or sold a hospital, preserving more than 43,500 jobs and providing care nationwide.
In 2015, 16 Prime hospitals were recognized as a “Top Performer on Key Quality Measures” by The Joint Commission, the leading medical accreditation organization in the country.
According to Erik Milham, Senior Director of Enterprise IT Operations and Infrastructure for Prime Healthcare, this overarching focus on quality extends to robust technologies and best practices used to rapidly assimilate new hospitals while safeguarding patient data and other information assets.
“Protecting our recently purchased assets as quickly as possible is a critical business priority,” he explains. “It’s imperative to ensure there is no disruption or degradation in patient care.”
Protecting our recently purchased assets as quickly as possible is a critical business priority. It’s imperative to ensure there is no disruption or degradation in patient care.Erik Milham, Senior Director of Enterprise IT Operations and Infrastructure | Prime Healthcare
Finding a cure for data silos
Prime Healthcare’s strategy of acquiring and turning around under-performing hospitals requires fast-paced assessments and deployments.
“Keeping pace with aggressive growth is challenging as we don’t have the luxury of time to do a lot of local hand-holding,” says Clint Monchamp, enterprise storage engineer for Prime Healthcare. “We continually look for ways to standardize on centralized, automated solutions.”
To streamline data management and mitigate risk, Prime Healthcare wanted to replace isolated systems with a unified solution built to deliver robust, scalable performance for all environments. To that end, the organization standardized on Cisco Unified Computing System (UCS) to manage resources across approximately 2,500 physical and virtual servers and launched an initiative to migrate standalone MEDITECH Electronic Health Records (EHR) systems to an enterprise-wide Epic EHR implementation.
Similarly, the IT team wanted to centralize management and protection of roughly 400 TBs of clinical, patient and operational data.
“We plan on bringing up to 10 new hospitals on the new Epic system per year,” notes Milham. “Trouble-free and fast deployments are crucial for ensuring our doctors and other healthcare providers can care for patients without delay.
“To ease our Epic migration, we needed a backup and recovery solution that was automated and could be launched and expanded easily. We needed a centralized, vendor-agnostic solution that could scale seamlessly across converged, virtual and cloud environments.”
Not only did Prime Healthcare require a fully integrated system for backing up and recovering crucial Epic patient records, hundreds of unique clinical and operational applications needed accommodation.
“No two hospitals are alike, so we had to break down data silos, simplify data management and maintain the highest levels of patient information safety and privacy,” says Monchamp.
Architecting faster, simpler EHR backup and recovery
In building a best-of-class data protection platform, Prime Healthcare put a premium on responsiveness of restoring a critical application, like Epic, that had the potential of being impacted in a disaster event.
“We had to ensure our clinical applications were protected from any type of disaster or corruption,” adds Milham. “Providing the highest levels of data integrity and compliance were paramount.”
In addition to seamlessly integrating with Epic and other core business practices, Prime Healthcare wanted to increase both the efficiency and effectiveness of regular backup operations. Minimizing tape storage while increasing use of both local and remote disk backups were on the short list of requirements, as was picking a vendor-agnostic solution.
“Avoiding vendor lock-in was key, as we have to support whatever storage comes our way,” says Monchamp. “This requirement eliminated a lot of products that were too closely tied to specific hardware.”
Additionally, data compression and deduplication on disk backups were desired to boost efficiency and reduce storage hardware costs. To help fulfill the organization’s DR vision, data replication and automated snapshot management were other requisites.
“We were entrusted by Prime Healthcare’s board to architect a solution with multiple levels of protection — from local and remote backups to full disaster recovery,” notes Milham. “Anything less would have been a disservice to our board, patients, physicians, nurses and employees.”
Equally important was centralized management and reporting to streamline implementation while facilitating ease of use.
“We needed to make it super simple for a local backup administrator to perform backups and restores,” Milham adds. “As only four full-time people oversee data protection for the entire organization, we had to mitigate the complexities encountered with previous backup software.”
As the review process evolved, the laundry list of requirements grew longer while the number of viable solutions shrank.
“There were very few products that could do everything we needed immediately as well as the things we wanted to do later,” says Monchamp. “Commvault stood out as the only solution built from the ground up to deliver enterprise-level data protection that matched our business practices. Commvault’s single pane of glass and intuitive GUI were as important as its cutting-edge feature set. We were confident Commvault would simplify our Epic EHR backup and recovery.”
Initially skeptical that Commvault could be deployed with Epic EHR as easily as other legacy platforms, Prime Healthcare’s integration partner was impressed with how much easier it was to capture Epic data as well as recover entire file systems and databases.
“When the rubber met the road, Commvault outperformed other platforms with the speed and simplicity of setting up hardware-based snapshots,” Monchamp recalls. “Commvault enabled streamlined backups and more flexible recovery and storage options as well as a simpler path to regulatory compliance.”
Commvault ensures healthy Epic EHR
Following a straightforward, initial deployment, Prime Healthcare has expedited Commvault rollouts at its various hospitals.
“The ability to fully configure the Commvault server and perform a backup in about 90 minutes has exceeded our expectations,” notes Monchamp. “Seamless integration with our virtualized environment makes it easy to accommodate VMware snapshots while application-consistent recovery points enable snapshot rollback for rapid data recovery.”
Commvault software’s ability to restore data quickly and seamlessly also exceeded Prime Healthcare’s expectations. During the rollout, the organization encountered an unrelated server failure that wiped out 150 servers and the hardware snapshot on a critical EMC flash storage array.
“During our first live test of Commvault’s recovery capabilities, we successfully restored 150 servers,” recalls Monchamp. “This gave us immediate confidence that we could offer a higher level of data protection, which helps me sleep better at night.”
Commvault software’s record of success is measured mostly in highly reliable performance, highlighted by Prime Healthcare’s 98 percent backup success rates and expedited recoveries following a failure. With Commvault’s Epic-validated reference architecture, Prime Healthcare can restore a full production image in minutes, which has let the organization qualify for certain Epic maintenance discounts.
“Previously, we struggled to meet those stringent recovery requirements, but we have not experienced any problems with Commvault,” Milham notes. “Commvault can take an image of any vendor’s storage system — whether it’s flash or spinning disk — and protect us against data loss.”
Real-time Disaster Recovery remedy
Several times a year, Prime Healthcare relies on Commvault to drive successful DR testing. As a result, the organization can failover its entire data center to its alternate site within 15 minutes while providing users with ready access to all data within 30 minutes. Thanks to Commvault, Prime Healthcare also maintains multiple backups to safeguard critical clinical and operational data at all times.
Secondary copies, maintained in a deduplicated format known as DASH copies, are pulled automatically across Prime Healthcare’s network from local sites to the DR site.
“Commvault enables us to be fully prepared should anything happen as we can recover data from multiple backup copies stored in different locations,” says Monchamp.
“We have all the right controls in place to ensure comprehensive information governance,” adds Milham. “With Commvault, we can perform eDiscovery on file shares and pull all relevant information because the search technologies have been built in. With Commvault, we can use one platform to protect vital hospital systems from unforeseen outages while ensuring comprehensive regulatory compliance.”
Prime Healthcare will continue to work closely with Commvault to ensure the highest levels of data protection for its far-sweeping clinical applications patient data.
“We’ll continue to build upon this best-of-class platform and innovate with Commvault to ensure that if something happens, pulling in Epic and other hospital systems will be as automatic as breathing,” concludes Milham.