Evolution of Data
Growth in data is attributed to vast improvements in internet technologies. Internet infrastructure enabled expansion across several industries including telecommunications, social media platforms, streaming services, and Internet of Things (IoT) applications.
The Limits of Traditional Data Warehouses
Traditional enterprise data warehouse (EDW) systems proved inadequate for managing this explosive data expansion:
- The conventional EDW architecture was engineered specifically for relational data structures and lacked capability to process unstructured or semi-structured information
- The EDW infrastructure suffered from scalability constraints. Compute and storage layers remained inseparably coupled, preventing independent scaling
The Big Data Revolution
The Big Data revolution introduced transformative innovations addressing these limitations. A pivotal advancement involved decoupling storage from computational resources. This era also fundamentally transformed how organizations approached data consumption, expanding beyond structured datasets to encompass diverse data formats.
Storage Scale
The brontobyte is the metric following the yottabyte. One brontobyte equals one quadrillion terabytes. A thousand brontobytes constitute one geopbyte.
Cost Evolution
- 2010: Storing one gigabyte on traditional hard disk drives cost approximately ten cents ($0.10/GB)
- Current: This expense has declined to one penny per gigabyte ($0.01/GB)
This cost reduction catalyzed a fundamental business philosophy change: organizations now adopt a “store first, analyze later” approach rather than selectively curating data for warehouse inclusion.