Dissecting Ovum’s Decision Matrix

Posted 11/08/2016 by Miranda Lenning

Posted in

Selecting a Data Availability and Protection Solution for the Cloud Era 

When the industry analyst firm Ovum released its Decision Matrix: Selecting a Data Availability and Protection Solution for the Cloud Era, 2016-17 last week, a few things stood out to me: 

  • There are some questionable technical elements to consider. Most importantly, Ovum included functionality from Veeam’s 9.5 release in its evaluation of the vendor. While Veeam conducted some external marketing activities in August to “announce” the timeline of availability for their new release, much of this functionality remains unavailable. Notably, support for physical environment restoration to the cloud remains unavailable to customers today and untested in its effectiveness.

  • The report is incomplete and does not accurately reflect a comprehensive analysis of the space. For example, Veritas and EMC were not included in the report. Per Ovum, neither responded to the questionnaire in a timely manner and thus, were left out of the report. However, can we really fully evaluate data availability solutions without talking about Veritas, the market share leader, and EMC, a long-time player in the space? Both companies have product documentation and marketing materials available on-line that could have been reviewed by Ovum, so they could have been included in the matrix.

I highly respect this group of industry experts and have always enjoyed working with them. However, this particular analysis requires a deeper look at its accuracy and completeness, which my colleague Bill Wohl discusses in his blog, I want to drill down into some of the technical aspects of the report and take a closer look at the competitive landscape. 

As background, vendors often have visibility into reports like this to provide a pre-publication “fact check.” In advance of Ovum’s publication, Commvault voiced concerns around the report’s incompleteness, and more importantly, its technical accuracy specifically around Veeam. Commvault has always taken the high road when responding to industry analyst reports. What does this mean? We respond based on functionality that is either available at the time of the response, or, with what we know will be available upon the report’s publication. It’s frustrating when other vendors don’t put this level of diligence into their responses, and perhaps even more so when an analyst firm doesn’t hold vendors accountable for the accuracy of their response.

 As part of our discussions with Ovum, we learned that they were aware that Veeam had included yet-to-be-delivered functionality in their response, however Ovum did not alter the results of the report accordingly. While not specifically naming Veeam, Ovum did however include the following language that makes it clear the assessment of Commvault is based on current product releases, and not some currently unavailable future release version.

“In terms of capability, Commvault’s submission is based on currently available solutions, not to-be-released solutions like some of its competitors.” 

Of note, Commvault is confident that even with a complete release of 9.5, Veeam will remain technically inferior in many areas, which we’ll dig into as we analyze the components of the report: SWOT, the radar diagrams, and Veeam’s assessment:


The SWOT narrative does mention Commvault’s unique platform approach, which enables Commvault to outpace the competition in terms of breadth of coverage with storage vendors, application support and public cloud support. Unfortunately, the impact on TCO is not calculated and factored into the SWOT analysis. For example, Commvault’s breadth of support and vendor agnostic approach allows end users to maximize their existing infrastructure investments and, provides them the flexibility to choose their preferred hardware, application or cloud vendor. We’ve also done a great deal of research that proves Commvault’s TCO is lower than Veeam, in part due to our ability to leverage commodity hardware at scale, unlike Veeam which requires deduplication hardware.

 Ovum also notes Commvault’s progressive approach to helping customers move to the cloud—a space where Commvault is not only outpacing Veeam, who as of today offers limited and disjointed cloud support, but also the broader industry (we work with more than 38 public cloud storage platforms.)

Radar Diagram Comparison

The radar diagram has three evaluation points: market impact, execution and technology.

It’s clear that market impact was an area of strength for Veeam on the radar diagram evaluation. Veeam spends big on marketing. They use puffery, misrepresentation and misdirection (aka smoke and mirrors) to boost their relevance. The problem is that once you grow and face modern IT complexities, you need substance that delivers real value; hype and hope will not help customers recover their ERP systems.

There are several components of the execution analysis that we also call into question. While Ovum doesn’t detail the complexity behind the conclusions, two components are based on revenue numbers provided by the vendor. We’ve already called into question the accuracy of Veeam’s response, so it’s natural that we’d question this as well.

The technology section also brings into question the Veeam submission. We expect that Veeam leaned heavily on their claimed ease of deployment which would potentially influence TCO, and with new disaster recovery features set for delivery in 9.5, their DR/Replication score is unlikely to rank as high in its current form. While they can demonstrate decent performance in certain configurations, this does not translate into consistent levels over a wide range of environments, or at scale.

The Veeam ODM assessment

As previously stated, we know this analysis includes yet-to-be released functionality, which even when in market will remain inferior to Commvault’s innovation. Key applications based on common enterprise software such as SAP running on HANA or MaxDB will remain unsupported, let alone other key databases such as DB2, Sybase, MySQL, PostgreSQL and more. We aren’t talking about just legacy or physical systems; many of these are important enterprise applications that run on virtual servers too. Veeam’s application support is limited to only Exchange, SQL, SharePoint, Active Directory and Oracle.

There are also gaping holes in hypervisor support. Currently, Veeam supports only VMware and Hyper-V, with Azure VM’s (via an agent) being added in v9.5. This could leave you with a number of VM types that require even more backup software and infrastructure, such as those in AWS plus OpenStack, Citrix Xen, RHEL and Nutanix Acropolis. Veeam’s 9.5 release consolidates some cloud tools for Azure, but not for other clouds—such as AWS—and will still lack the provisioning tools required for a rounded solution. 

We have already mentioned that the report is about backup and availability, so many of the Commvault platform benefits are just not factored in, and while Veeam received a mention for their flexible licensing and service provider model, Commvault’s offerings—like the more robust and equally simple VM Backup and Recovery solution set—remain unmentioned.

Overall, while we respect Ovum and appreciate their expertise in this space, we’re disappointed with the completeness, accuracy and overall accountability of this report.