Monday, April 30, 2012

The new frontier in Business Analytics: Big Data


Most of the organizations have access to tons and tons of information, but they don’t know how to get value out of it because most of the information is either in unstructured or semi structured format.  But most of these organizations who witness the data explosion around them have also started to realize that analyzing this large data sets (or Big Data) will become a key basis of competition and growth, sooner or later.

Now-a-days, every organization has taken the first step in data analytics, having data warehouses in place. The traditional data warehouses are mostly for analyzing structured data from various systems and producing insights with known and relatively stable measurements.  What gets unaddressed in the traditional solution is the unstructured or semi-structured data, which forms more than 70% of data generated in an organization such as email communication, server logs, customer service call logs, status updates in social media sites etc...

So what characteristics should the data sets have to qualify itself as Big Data? According to IBM, Big Data has three important characteristics: Volume, Variety and Velocity, the amount of data involved, the types of data and speed at which these are getting generated for the analysis.

Big data requires exceptional technologies to efficiently process large quantities of data within tolerable elapsed times. There are many and to begin with, let us get familiar with NoSQL and Hadoop, which is synonymous with Big Data in my subsequent blog.