A big part of my role at EMC these days is engaging with our customers and partners to discuss the industry trends and EMC strategy. Over the past couple months just about every customer wants to talk about their Big Data project. In addition to my anecdotal market analysis, IDC predicts the IT market for Big Data Solutions will grow from $10.2 billion in 2013 to over $53 billion by 2017. That is a 5x growth. No question that Big Data is one of the hottest IT opportunities today.
Before thinking about solutions, I think it is important to understand the enablers of next generation analytics or Big Data to use the popular term. Analytics has been an important IT capability for many years. Who doesn’t have one or more data warehouses, and data marts to support manufacturing, finance, sales, and marketing today? So what is new and enabling the next generation Big Data analytics opportunity?
I believe there are four enablers that have combined to reach maturity to create the perfect storm. The first enabler is the return on investment for storing and maintaining complete data sets for analytics is positive. The continued reduction of storage hardware costs, and improved data efficiency capabilities like de-duplication, and compression have become pervasive. In the most recent EMC Digital Universe research, IDC predicts the amount of digital data generated in each of the next two years will double, and will continue to double every two years for the rest of the decade.
The second enabler is the rapid growth of data from network-connected devices such as sensors. The EMC Digital Universe study predicts that 10% of all digital data will be generated by network connected devices by the end of the decade. Today, network-connected devices generate about 2% of all digital data. Smart companies like GE, are leveraging this data to provide differentiated services. A GE Wind Turbine contains about 20,000 sensors that generate 400 data points per second. The sensor data is analyzed in near real time to maximize the efficiency. This data is also stored and used for deeper analytics to improve maintenance and parts replacement. A GE Wind Turbine is more efficient and has higher availability enabled by next generation analytics.
The third enabler is the increase in unstructured data types such as images, video, and audio. The EMC Digital Universe study predicts 30% of all digital data by the end of the decade will be security related with the majority being images, video, and audio. Traditional analytics have not leverage these data types.
The forth enabler is a new set of analytics tools designed specifically to analyze large amounts of data, both structured (i.e. log data) and unstructured (images). Tools such as Hadoop, and Splunk were designed to analyze large data sets. These new analytics tools have been created through the Open Source community and have a low initial cost to get at least get started. ext generation analytics has become affordable for big companies but smaller companies are also using these tools to find new revenue, and provide differenaited services.
These are not the only four enablers of next generation analytics but I think they are the most impactful. I only see next generation analytics gaining more momentum in the coming months. I will share my thoughts on the IT requirements for Big Data services over the next few days.
It's unbelievable how much potential and money is there in Big Data.
Posted by: Tom Voyager | 08/27/2014 at 10:03 AM