The need for speed, part 1: Big data analytics and flying blind


Big data and the need for speed
If CIOs and CFOs wholeheartedly agree on just one thing, it’s the value of in-depth data analyses delivered at the speed of thought.  Being able to reduce the analysis of billions of data records – a process that previously took hours to complete – to just seconds, or in some cases to a mere fraction of a second, is the promise of ‘big data’ and the future of IT.
Of course, it is very easy to expound the benefits of that kind of real time analysis, but actually putting in place the infrastructure to deliver it is, of course, a lot more complicated.  It’s a challenge that is testing the minds of CIOs and CFOs the world over – and it is a vitally important one.
It is not an overstatement to suggest that having the right business analytics solution in place is as vital to the overall health of today’s enterprise organisation as having air and water is to human beings. Simply put, you just can’t live without it.
Put it this way.  If pilots flew planes the way most corporate executives run their businesses, they wouldn’t know they had taken off until well after the plane reached 10,000 feet and the seatbelt sign had been turned off.
The larger and more sophisticated the organisation, the more data has to be processed to actually see where it is going. Comprehensive cost calculations at large complex organisations, for example, could easily take the better part of a day to execute.
And that’s under normal conditions. Introduce a stock market crash, a supply chain interruption or a manufacturing problem, and executive management is reduced to flying into the unknown by the seat of its collective pants.
Knowledge is power, and not having it for critical blocks of time is the stuff of CFOs’ nightmares. With millions at risk every minute in large organisations, the lack of timely business analytics can lead to logistical, manufacturing, and, ultimately, financial disaster, literally in less time than it takes to pull a comprehensive report.
As we all know, the data is there.  Information technology today has made it possible to collect staggering amounts of data, both structured and unstructured. The world’s technological per-capita capacity to store information has roughly doubled every 40 months since the 1980s. By 2012, 2.5 quintillion bytes of data were being created every day.
But turning these vast repositories of data into intelligence is no small task.  Fundamentally the key is to stop simply collecting data and begin actually connecting with that data in meaningful ways that will enhance the company’s most critical decision-making processes – and that boils down to putting in place the right analytics infrastructure.
In part 2, I will outline just such a solution…