Data Revolution The Gold Rush of The 21st Century

By Sethuraman Janardhanan On 28 Sep 2015

This blog is published by

The 1800s were all about mining – gold, iron and other metals. The 1900s were all about drilling – oil, natural gas and shale oil. The 2000s are again going to be focused on mining, but of a different kind – data.

Everything in the modern world from humans to machines is a data factory. The total data accumulated in 2012 and 2013 was more than nine times of the total data created till 2011. By 2020 this data is expected to reach 44 Zettabytes.

Data is being stored in servers across the globe, creating veritable data mines across multiple locations. North America leads the group in this data storage followed by Europe, Japan, China, Middle East, India and South America in that order.

Enterprises around the world have realised the value of these data mines and the technology for its proper mining and use is evolving every day. Proprietary algorithms are being developed to comb this data for trends, patterns and hidden nuances by enterprises around the world.

A closer look at the gold, oil and data rushes throws up certain interesting similarities. In the case of metal or oil, the technology and the methods of refining are as important as the ore/crude itself. We cannot use the ore or crude in the raw form. Without refining, it has far lesser utility and value.

A lot of investment is needed to set up these processes, methods and infrastructure. Similarly, process and the methods to extract usable insights from data needs extensive investment and research.

New model

Data is fundamentally transforming the way people do business, how they communicate and how they make decisions. It is turning the traditional business models on its head and bringing new unused resources to the market place.

Take the examples of Uber and Airbnb. In both these cases under-utilised resources have been brought to the market. Uber brought personal cabs into the taxi network and Airbnb included personal property that was under-utilised. One promoted taxi sharing, another personal properties and rooms that the owner was willing to rent, but didn’t know how.

The new market places are getting created where none existed before. In fact, the reliance on capital for doing business and securing capital itself is coming down with concepts like crowdfunding.

New technologies like the Internet of Things (IoT) are enabling the insertion of sensors in machines so that they can communicate. According to Cisco, IoT will be generating 400 Zettabytes of data every year by 2018. Capturing and analysing this data is going to be a huge challenge and an immensely profitable and critical activity in the very near future.

Consumer choice today is being decided by what data is created by other users. The internet-savvy generation hardly takes a buying decision without checking out the reviews for the seller and the product online. All these new business channels are heavily dependent on the analysis and interpretation of continuously generated consumer data.

This has further led to transition from traditional statistical models of data analysis, which was once a favorite, to models that incorporate automated techniques giving rise to machine learning processes for analytics.

Politics today is as much about real-life campaigning and touch points as about Facebook, Twitter and YouTube. Politicians from the US, Middle East and India all have dedicated online wings which are creating data and analysing it to understand voter opinion and predict voter leanings.

Communication, which was once the soul preserve of words and voice, is now moving towards video and images. People are communicating more and more through images.

All these changes have led to the rise of tremendously huge volumes of data – mining this data and using it to get the advantage gold diggers obtained in the 1800s forms the crux of an organisation’s success.

Mining an avalanche

Data is being converted into actions and customer impact across consumer behavior, supply chain efficiency, healthcare, scientific research, agriculture, logistics, urban design, energy, retailing, crime reduction, business operations, and many other aspects of business performance.

Almost any company that grows, makes or sells any product can use big data analytics to ensure that their production and manufacturing processes are more effective and efficient, their marketing efforts are better targeted and their business processes are more cost effective.

Big data analytics not only changes the predictive modeling and forecasting concept by closely connecting with machine learning algorithms, but further brings about a transformation in knowledge extraction and interpretation.

Machine learning algorithms lent scalability and predictability across various big data analytical procedures using combination of fully automatic and generic concepts. A study by Bain and Company shows how important big data analytics can be for companies in terms of competitive differentiation.

When 400 large corporations were examined, they found that the companies with the best big data analytics capabilities were doing better than their competitors by a huge difference. They were two times as likely to be in the top quartile of financial performance within their industries and to use data very frequently when making decisions, three times as likely to executive decisions as intended, and five times as likely to make decisions much faster than market peers.

This is going to become increasingly prominent in both scale and scope with every passing day. The transformation of physical and digital worlds becoming one is irrevocable, irreplaceable and exponential.

With the data driven economy here to stay, organisations that are cognizant of the rise of data economy will feature in the list of successes while others merely continue to exist.