Don't View Big Data as a Silver Bullet
If companies learned anything about Big Data this year, it’s that it is not a panacea. It’s a complex ecosystem unto itself, the value of which derives from its integration with all of an organization’s other operations.
"Some organizations are realizing that the investment in Big Data, though it promises many benefits, is only a piece of a much larger pie, and you need to have some of the other pieces in place"
For too long, CIOs and other data scientists have operated on the premise of “data first, story later.” In other words, capture as much data as possible, because if you capture everything, you can find out everything about the story you want to eventually tell. More often than not, it doesn’t work out that way. Some organizations are realizing that the investment in Big Data, though it promises many benefits, is only a piece of a much larger pie, and you need to have some of the other pieces in place.
First, understand what business problem you’re trying to solve. Too often, CIOs and their staffs have believed that capturing tons of information would give them all the insights they needed. But everyone has to figure out on their own how to analyze the data in order to get to the insights.
Second, CIOs must make sure that the skill sets of the company match the challenges at hand. Big-Data platforms are high tech and not very well grounded in mature technologies such as relational database platforms. “Mature” here means that people who buy, license and use the technology have been giving active feedback to the vendors who make these tools so that the vendors can improve their functionality. This refinement happens over years, sometimes decades, making that particular technology “mature” and able to handle a lot of requests. Experienced staffers also can analyze data with speed and accuracy to get to the meat of the issue.
Less mature software systems have the advantage of speed, in that they can process billions of data points in real time. The downside is that they don’t have the history of having been vetted and tweaked for improved performance over many years based on feedback from seasoned data scientists, who know how to make end runs around these gaps in non-mature technologies. Less experienced workers struggle with dazzling new, but less mature, technologies.
Data scientists and analysts who can handle not only mature technology, but also the newer “non-mature” software that’s flowing into the market, are hard to find. An organization’s ability to process data requires an understanding of the data lifecycle and the right professionals spread across departments who know how to leverage it.
The sophistication of the data analysis also hinges on the ability to evaluate other, complementary technologies. New software and tools are entering the market at a very fast pace so organizations need people who can quickly evaluate them and their applications without getting caught up in “the buzz” around the products.
Third, you must be well able to communicate the value of the Big-Data investment and all the tools, technologies and staffers needed to support it, to the executives who write the checks.
Many companies struggle to draw a straight line from investment to ROI with Big Data. Understandably, the C-suite wants to know the value of million-dollar purchases in newer technologies. It’s no secret that some data professionals make technology and software purchases because of trends and not necessarily based on the value they’ll add. A lot of CIOs need Hadoop, but understandably are asked what the organization is expected to get out of it. If you’ve identified the business problem you want to solve and have the proper staff in place to execute on it, you’re in a better position to ask for investment dollars.
And don’t forget the cost factor involving data storage and its use. Having to know what happened two years ago with certain data sets has a separate use case, and there’s real cost in how your data is stored and accessed. You don’t want to keep data around that you’re not using, or pay more to do so.
And, in addition to understanding internal business needs and expertise, it’s critical for CIOs and their teams to evaluate the prospective software, as well as the level of knowledge, support and maintenance capabilities you can expect from the vendor who makes it, before asking for that check.
Remember: Right plan, right people, and right ecosystem. Any one piece that is missing will certainly cause problems in execution. Big Data in and of itself, is not a silver bullet.