Blog
Comment
iniewski
Are we getting lost in this huge sea of humongous data? Is storage capability of ...
giraffedata
His company also uses disk drives and tape cartridges, and probably has a higher ...
The storage tipping point is upon us
Ambuj Goyal
2/27/2013 2:01 AM EST
It’s well known that the performance of computing has advanced at a much faster rate than the systems that store and retrieve the information they generate.
For many, this game of catch-up has existed since the first digital storage systems were introduced more than 50 years ago. At the time, the concern was no longer about the performance of computing, but about creating a digital storage system that could keep up with it.
Thus began the subsequent watershed moves from paper punch cards to magnetic tape and then hard disk drives—moments that revolutionized computing and ultimately the world in which we live.
Today we find ourselves at another critical technological juncture that once again is demanding a revolutionary approach to storage—an approach that will help it keep pace with not only computing, but with the information onslaught of Big Data.
To understand where the storage industry is headed, one need only look to the reason that computing has historically outpaced it. Unlike the storage industry, computing has continually leveraged and advanced semiconductor trends while storage systems have remained mechanical, with motorized wheels of tape or spinning disks. In fact, computing shifted from mechanical devices more than a hundred years ago, while digital storage, for the most part, remains tethered to technologies born out of the 1950’s.
Not any longer. We are at the tipping point of a new era of computer storage that will witness entire systems based on flash semiconductor memory to handle fast moving, operational data in real-time. Though flash has been utilized in a variety of capacities over the past 30 years and in hybrid storage systems over the past several years, complete flash systems will dominate the landscape in the coming months and years.
All-flash systems will not only provide exponential performance gains over mechanical and hybrid systems, they will help organizations dramatically lower data center energy consumption rates due to their inherent low power-consuming memory and lack of moving parts—no small feat. According to a 2011 study by Stanford University, data centers account for 2 percent of all the electricity consumed in the U.S.
For many, this game of catch-up has existed since the first digital storage systems were introduced more than 50 years ago. At the time, the concern was no longer about the performance of computing, but about creating a digital storage system that could keep up with it.
Thus began the subsequent watershed moves from paper punch cards to magnetic tape and then hard disk drives—moments that revolutionized computing and ultimately the world in which we live.
Today we find ourselves at another critical technological juncture that once again is demanding a revolutionary approach to storage—an approach that will help it keep pace with not only computing, but with the information onslaught of Big Data.
To understand where the storage industry is headed, one need only look to the reason that computing has historically outpaced it. Unlike the storage industry, computing has continually leveraged and advanced semiconductor trends while storage systems have remained mechanical, with motorized wheels of tape or spinning disks. In fact, computing shifted from mechanical devices more than a hundred years ago, while digital storage, for the most part, remains tethered to technologies born out of the 1950’s.
Not any longer. We are at the tipping point of a new era of computer storage that will witness entire systems based on flash semiconductor memory to handle fast moving, operational data in real-time. Though flash has been utilized in a variety of capacities over the past 30 years and in hybrid storage systems over the past several years, complete flash systems will dominate the landscape in the coming months and years.
All-flash systems will not only provide exponential performance gains over mechanical and hybrid systems, they will help organizations dramatically lower data center energy consumption rates due to their inherent low power-consuming memory and lack of moving parts—no small feat. According to a 2011 study by Stanford University, data centers account for 2 percent of all the electricity consumed in the U.S.
|
|
|
|
|
Navigate to related information
docdivakar
2/28/2013 2:53 AM EST
One thing abundantly clear to every one is the explosion of data in the last five years. If projects based on IoT forge ahead and we end up living with 50billion nodes networked by 2050, a storage environment where round trip latencies for accessing and fetching data have to scale exponentially and that is possible only with solid state drives. The question is how can be make the nonvolatile storage faster and more reliable, over and above what is afforded by redundancy today.
MP Divakar
Sign in to Reply
Work to Ride comma Ride to Work
2/28/2013 12:02 PM EST
At some point we'll have to stop hoarding data and do some old-fashioned spring cleaning. I see a horrible practice of multiple instances of the exact same files here on the corporate servers basically because people are in herently lazy and nobody forces the issue. We could easily contain this data explosion if people could just discipline themselves to keep only what they need. I dare say, 98% of the data is probably worthless (repeated, out-of-date, superseded, obselete, etc.).
Sign in to Reply
vlsi_guy
2/28/2013 1:00 PM EST
Sounds like an opportunity to make some money to me :-) (i.e., finding duplicate data and flagging it)
Sign in to Reply
docdivakar
2/28/2013 2:09 PM EST
Valid point! Most storage area networks and storage services will have 4X to 6X redundancy and my point above is this can get way out of hand with hardware limitations in the coming days. Solidstate drives have to improve their reliability drastically from where they are now.
MP Divakar
Sign in to Reply
Robotics Developer
3/1/2013 12:01 AM EST
I love the fact that I have 2 TB of at home storage! I also really like having alternate remote site backups! It is a wonderful time to be on a machine with so many options for storage/performance and capacity..
Sign in to Reply
iniewski
3/1/2013 10:52 AM EST
Not sure that few Tbs of storage I have helps me. I am at the point where I am not sure where everything is located. And don't have the time to sort thru 100 Gbs+ storage in family pictures not to mention other stuff. Help is needed. Start-up opps?
Sign in to Reply
resistion
3/1/2013 12:08 PM EST
Computing could become big data oriented if it were all based on big look up tables. But accessing memory to carry this out would have to be much more parallel than existing multi-core threads to be appreciably fast. Probably on the order of 1000x at least.
Sign in to Reply
DMcCunney
3/2/2013 11:10 PM EST
Of course the author will plug the advantages of flash memory, since his company uses it.
But the Invisible Hand stills plays a role. We are in the early stages of shifting to SSDs in PCs and the like, because old fashioned hard drives are simply a lot *cheaper*. I don't see the industry as a whole shifting till costs drop by an order of magnitude. It's the reason why an awful lot of backup is to tape: the cheapest cost per megabyte of storage.
Sign in to Reply
giraffedata
3/4/2013 1:10 PM EST
His company also uses disk drives and tape cartridges, and probably has a higher profit margin on them, so I think his saying solid state is about to take over carries real information.
Sign in to Reply
iniewski
3/4/2013 7:21 PM EST
Are we getting lost in this huge sea of humongous data? Is storage capability of our network start to become so much larger than ability of our brain to digest that data?? The tipping point might be that soon nobody will know what they have stored and where.
Sign in to Reply