The Prescribed Value of Data over Time.
by Anirudh Todi (Senior Software Engineer at Twitter)
There is no dispute; data is growing exponentially…and it will continue to grow, both in volume and complexity. In general, the more of a resource there is, the less value any individual item holds – like a commodity. Due to the economic laws of supply and demand, the more of a commodity exists, the lower its price or value. In this way, individual data points can be viewed as a commodity, similar to oil or gold. And similar to fuel or precious metals, data has been used as a means of doing business, hedging risks, investing and even speculating for decades or even hundreds of years.
This commoditization of data did not happen overnight. Data comes in many sizes and shapes and is formed from a vast range of sources. Advancements in technology have contributed to the data deluge, with unstructured data volumes increasingly generated not only by the human workforce, but by machines, generating petabytes of data on a daily basis. As we look at this trend of growing data stores, each individual data point is effectively undifferentiated and, on its own, conveys limited value – just like a commodity. The typical rational approach to managing undifferentiated goods is to reduce their cost as low as possible. This is often the approach taken by storage professionals as they focus on the cost side of the data equation.
However, something unique happens when you take all the individual data points, and consolidate them into a centralized repository. By aggregating individual data points, the value becomes greater than the sum of its parts. By delivering a holistic view of the sum knowledge of the organization and exposing it to statistical and predictive analytics tools, strategic competitive value can be gained from the data. The aggregated data suddenly becomes so crucial and proprietary that its value is unparalleled and irreplaceable to the organization.
For instance, companies like Twitter and Facebook collect data on every single action taken by the user on their websites. While the value of any given data point can often be negative due to the cost of acquiring and storing the data, by providing tools to drill down into that data at the content level to understand its meaning and extract useful insights from it, organizations can exploit it and ultimately realize the value that it embodies – the crown jewels of the firm.
So although individual data points have relatively minimal value by themselves, when consolidated into a data centric repository, substantial value and a competitive edge can be gained. Having this central repository of information at your fingertips allows for increased productivity and the ability to make a good timely decision based on good data, one that will bring enormous value. Better to focus on the value side of the data equation; the costs are just at the surface.