Big Data Management: Controlling the Beast

Posted 11/16/2016 by Commvault

Fans of author James Patterson recently saw a hit television show, "Zoo," based on one of his novels. The premise is that animals rule the world, uncontrolled, until a cure is found for a mutant strain that has turned all of them into vicious predators. When it comes to managing Big Data, have you ever felt you’re the one in the zoo and Big Data is running rampant outside? To control the Big Data beast, you need to have in place a multi-faceted management structure that can handle both structured and unstructured data, supports thousands of Big Data nodes, understands compliance and governance requirements and can shuttle data to the cloud, all while separating the critical data with legal implications from less essential data. About now, the zoo might be sounding good!

Getting control of Big Data is the first step in organizations realizing the full spectrum of benefits from advanced business and prescriptive analytics, and even real-time insights via streaming data. The International Institute for Analytics predicts, that by 2020, businesses using data effectively will see $430 billion in productivity benefits over competitive organizations that fail to use data.1

Since this isn’t fictional television, or even ‘reality’ TV, we’re not going to invent a magical cure for animals running amuck, but we certainly have technical tools at our disposal to keep Big Data under control; and in an organizational form and readiness that helps us increase competitiveness by extracting data knowledge and analytical intelligence. Here are a few key technical issues to consider:

Putting a Leash on Unstructured Data: Email and videos are examples of unstructured data, a category of Big Data greatly increasing as content marketers and other professionals find video, for example, an effective way to quickly communicate in a medium most people like. In an organization, video and other forms of unstructured data are subject to the same legal compliance requirements and corporate governance criteria as structured data. It is only a matter of time before video eDiscovery becomes as routine as basic document discovery. Your organization needs to have in place a backup, archive and discovery structure that equally supports structured and unstructured data.

Muzzling the Nodes: Organizations deal with the Big Data behemoth by running multi-nodal systems that enables them to query a specific node, discover what information it contains and then, based on its level of importance, assign retention, recovery and disaster recovery support. When further developing your Big Data management plan, look for a solution that enables you to investigate, node by node, the relative importance of this data so that IT and data protection resources are used for those data sets that add business value and are compliance-sensitive.

Harnessing the Power of the Cloud: Your organization most likely is now using a combination of storage and recovery platforms, including the cloud, on-premises and maybe even still, offsite tape. The cloud is an attractive option for moving Big Data sets of such large magnitude offsite due to ease of scalability and cost control. However, you will need to ensure there is a solid plan in place for disaster recovery and business continuity for the critical assets residing in the cloud. This will ensure compliance and efficient eDiscovery as needed. Also, be sure your data management solution can provide data portability between the cloud and other platforms, such as on-premises backup.

Controlling the Big Data beast is essential to realizing the full business value of this extraordinary amount of data organizations are generating and receiving every day. It is still early in the game for most organizations as they contend with the issues of managing, storing and protecting these large data sets, and extracting business intelligence from this data. A good start is addressing the growing amount of unstructured data and ensuring it has adequate protection and eDiscovery capability, determining how the cloud fits in your solution for Big Data recovery, and making sure you have a solution that enables portability between platforms. 

Ready to take control of your Big Data beast? Read more in the whitepaper, Set a Place for Big Data at the Adults’ Table. It covers the five reasons it’s time to take Big Data protection and recovery seriously.

117 Predictions About The Future Of Big Data Everyone Should Read, Forbes, March, 2016