Data Temperature and Digital Universe

Kaizhong Gao, updated in August, 2019

Data storage technologies have tremendous growth over the past few decades. Both hard disk drive (HDD) and Tape recording products have areal density growth for more than 1000 times within the past 25 years. Over the past decade, the size of the digital universe has grown exponentially and will continue to grow at approximately 40% per year in the next few years (according to IDC/EMC). One of the primary driving factors is the rapid adoption of mobile devices and network enabled sensors (or the internet of things, IOT). The low cost of computational chips and storage devices enabled rapid adoption of various devices across many industries and consumer markets. Many storage technologies have emerged such as NAND based solid state drive (SSD) began to take over part of the traditional HDD market. The most recent announcement of 3D NAND based SSD has the device capacity well exceed 100 Terabytes (TB), far exceeding current HDD product at 16 TB/unit. The storage class SSD became mainstream that taking over traditional high performance HDD data storage market.

Data Temperature in a Storage Device

To understand how to manage big data storage and where each data storage technology fit into the big picture, relevant data storage theory and a universal standard needs to be established. Here we give a standard “data temperature” definition:

where one year is approximately equals to 31,557,600 seconds. This data temperature, in unit of degree G (ºG), can be utilized as a standard measure to determine one aspect of data property, i.e. how often the data needs to be assessed, in storage devices/systems. The access is defined as either a read or a write operation. Here we do not differentiate read and write operations. Under this standard data temperature definition. if a file is accessed yearly, daily or in every second, then its data temperature is at approximately 0 ºG, 25 ºG and 75 ºG respectively.

With this definition, we can identify universal evolution of the digital universe. In certain ways, this evolution of digital universe shares the same trait as the physical universe, which leads to profound impact to human society. Some of the discussions have been shared with google talk.