The advent of digital pictures, music and movies promises a consumer media revolution that's just starting to drive the technology and generate hard-to-predict demands for new kinds of storage systems and services.

Makers of flash memory and hard disk drives both see a 10-year road map ahead that will enable them to respond to the growth. There are technical challenges, however. Hard drives in particular face a historic shift to perpendicular recording in the next two years, and the possibility that designers may be hitting a wall in their efforts to continue to shrink disk drives.

Dozens of dark-horse technologies are cooking in the labs. The most promising of them-probe storage-could insert itself into the hot spot of the emerging consumer markets within 18 months.

"Consumer applications for storage have been interesting for quite a while, but they've only kicked off in the last 18 months beginning with the personal video recorder (PVR), games and new kinds of systems," said disk drive veteran Mike Cordano, vice president of sales and marketing at Maxtor Corp. (Milpitas, Calif.).

"These markets are still very, very young, and most of the growth is still ahead of us," said Eli Harari, CEO and founder of SanDisk Corp. (Sunnyvale, Calif.), a leading flash maker that recently booked record quarterly revenue. "This new consumer market will be driven by cell phones and convergence devices with cameras and MP3 players and video."

Mark Kryder, CTO at leading hard-drive maker Seagate Technology and a former storage researcher at Carnegie-Mellon University, agreed. While computers are approaching 200 million units a year, cell phones will ship as many as 600 million units a year-and Apple Computer sold a whopping 150,000 units of its iPod mini the day it was released. "These are big markets," said Kryder.

"A few years ago, it was unusual for someone to own a hard-disk drive. But in a few years, it will not be unusual for homes to have perhaps 10 hard disks-in computers, set-top boxes, video recorders, MP3 players and even cell phones," said John Best, CTO of Hitachi Global Storage Technologies (GST), created by the merger of Hitachi and IBM's drive businesses.

"Clearly, the hard drive is getting into consumer devices such as the Xbox, TiVo, MP3 players, PDAs and automotive aftermarket navigation systems," said John Osterhout, business development manager for the 1-inch Microdrive at Hitachi GST. "This is very significant in signaling the trend of hard drives in the consumer space."

Still, Best said, it's important to keep the consumer opportunity-the new and somewhat unknown factor in the storage game-in balance with the traditional and still-vast market for computer storage.

"A lot of the excitement is in the consumer segment, which is the fastest-growing area, though it's still very small," said Best, formerly the head of the IBM Almaden Research Center. "Consumer will be a technology driver, but not the only one. The enterprise will continue to push performance and reliability, and nothing is going to change that."

"It won't be one size fits all. Different criteria will prevail for different market needs," said Alan Niebel, chief executive of market watcher Web-Feet Research Inc. (Monterey, Calif.).

Today, a gigabyte marks the unofficial dividing line above which drives are generally better and below which flash usually makes more sense. In three to five years, the line is likely to shift to 4 Gbytes.

There's less bang for the buck trying to pack three or more bits per cell in a flash chip, said Harari of SanDisk. For example, a 3-bit cell requires eight levels of signal differentiation, and that means 16 to 32 levels of granularity in what can effectively be detected.

"That's possible in the lab, but it's not practical when running at very high speeds," said Harari.

Nevertheless, the basic dynamics of CMOS scaling provide plenty of headroom for flash. Today's 500-Mbyte chip made in a 90-nanometer process will morph into a 4-Gbyte (32-Gbit) chip when the industry gets to 40-nm technology in about five years, Harari estimated.

"We've gone from 4-Mbit to 4-Gbit chips in 13 years. I don't think we will go to 4 Tbits in the next 13 years. Still, we can go quite a long ways before we hit some significant brick walls," he said.

"We're at the threshold of things getting really difficult. Our view is it doesn't make sense to go less than a 1-inch drive," said Best of Hitachi GST.

Seagate's Kryder disagreed. "I think the 0.85-inch drive probably makes sense for cell phones, if you can get it into an Secure Disk [flash card] form factor. It opens up some doors, he said. "The question is, when is the time right to get into that space? IBM launched its [1-inch] Microdrive about five years ago, and it's just now finding significant markets."

The trend currently favors the hard disk in MP3 players, thanks to the format's lower costs at high capacities. But flash appears to have the inside rail in the megaconsumer market for storage in the cell phone because of flash's smaller size, low power and greater durability for moderate capacities.

Rather than using traditional magnetic or electronic means, IBM's millipede uses thousands of nano-sharp tips to store more than 3 billion bits of data in punch indentations the size of single holes of a computer punch card developed some 110 years ago.

Meanwhile, the emergence of video on some cell phones may call for greater storage capacity over the next four years, opening the door to hard drives. "But by then flash will be so pervasive in phones it will be difficult for hard drives to get a broad hold," Niebel said.

Osterhout hopes drive makers can leverage their strength in MP3 players into design wins in handsets. "I wonder how many MP3 players will become cellular phones. Even getting a segment of the mobile phone market is significant," said Osterhout.

A smaller but significant consumer market in automotive storage for navigation and entertainment systems is just getting started. However, car makers are asking for drives with an operating range from - 40 degrees C to 85 degrees C, something no drive maker currently offers but many are working to develop.

Another emerging consumer market is what Maxtor calls a "shared home storage" box that could be a networked media repository for a set-top box, PC or digital TV.

"Today it's mainly about sharing data files, but this will evolve to sharing entertainment media," said Cordano, whose company is developing middleware to manage multiple digital video streams on its consumer drives. "You will see versions of this begin to reach the market in 2005 and get more sophisticated over time."

The other big change ahead for drives is a shift from storing magnetic charges longitudinally on a disk's surface to storing the charges vertically. Benefits stem from the fact that it's difficult to push horizontal charges close together, whereas vertical charges will naturally couple, opening the door to smaller recording tracks or more bits stored per track area.

"It's a big change. The heads, media, signal processing and servos are all different. It's probably the biggest change since the hard drive was invented in 1956," said Seagate's Kryder, who worked on the technology while at Carnegie-Mellon.

Experts expect the shift to perpendicular recording will commence late next year.

"There isn't that much extensibility left in longitudinal recording," said Kryder. "You can do 160 Gbytes/platter with longitudinal, but probably not 200 Gbytes and certainly not 300 Gbytes, so you will see some variance across the industry for when people introduce perpendicular recoding."

The technology "gets us close to a terabit per square inch. When that level is reached, a single 3.5-inch disk will store over 1 terabyte of information," Kryder added.

Best of Hitachi GST said perpendicular recording "in its first two generations probably provides a fairly modest areal density increase, on the order of a factor of two or three."

Bigger advances will come when perpendicular recoding is linked to a new generation of so-called patterned media, Best said. Rather than magnetize several grains on a platter, patterned media promises a move toward magnetizing a single grain at a time, as long as drive makers can find economical ways to read and write the patterns.

"I'm a bit of a patterned-media enthusiast myself, but I couldn't predict the time frame it might happen," Best said. It might come before, after or at the same time as work in so-called thermally assisted storage, in which spots on a disk are heated as part of a refined read/write process.

"We are doing a lot of work in both [patterned media and thermally assisted recording], but they will not become commercial in five years," said Kryder.

A smaller but growing priority in hard-drive R&D is reliability-for both business and consumer products.

"We'd like the drives to have 10 times the reliability they have today, and we think that's feasible. That has not been a metric for the industry in the past, but it's a major one now," Kryder said.

In March, 10-person startup Nanochip Inc. (Austin, Texas) secured $20 million in second-round funding to help commercialize its probe storage technology. The company has already lined up two foundries and believes it could have products ready to ship before the end of 2005.

Worldwide production of original information, if stored digitally, in terabytes, circa 2002. Upper estimates assume scanned data; lower estimates assume compression.

IBM, with its much larger resources, has demonstrated prototypes but still describes the work at its Zurich, Switzerland, research lab and the Almaden lab in San Jose, Calif., as a highly promising research project. With densities of a trillion bits/square inch demonstrated in mid-2002, the much-publicized Millipede approach is capable of densities 20 times higher than more conventional storage technologies (see www.research.ibm.com/resources/news/20020611_millipede.shtml).

IBM declined requests for an interview on the status and outlook of the Millipede project.

HP Labs has also had an active R&D project on probe storage for several years. Hewlett-Packard is actively trying to license its approach, called Atomic Resolution Storage. A spokesman said the company has decided the technology is not core to HP's business, and the lab hopes to shift resources to other projects.

"Assuming [probe storage] is successful, it will probably fit somewhere in between flash and hard disk," said Seagate's Kryder, who has a small internal team studying probe storage.

"Probe storage has to be cheaper than hard drives and have a capacity more than flash. If that pans out, I think it will have a reasonable market," Kryder said. "The key to doing this well will be to find the right read/write head/media mechanism. This is the area where there is no general agreement about how to move forward."

IBM's Millipede uses an indented polymer scheme that uses heat as an erase mechanism to push back out the indentations. By contrast, Nanochip uses a phase-change technology not unlike the approach used in optical disks, Kryder said.

Hitachi GST also has a small probe R&D effort. Best, however, said he harbors doubts about the technology: "Probe storage doesn't scale well to products of multiple square inches in size. I am skeptical it could replace hard disks or even flash for most applications."

Harari of SanDisk was even more dubious, calling probe storage "a lab curiosity."

"I have no problem accelerating the demise of our [flash] technology," he said, "but for the next five to 10 years, it looks like we have the best technology in town, and we know how to make it."

One of the most hyped, MRAM, cannot scale to the capacities of flash or hard disks, Niebel said. The often-discussed concept of holographic storage has faced a lack of suitable materials for years, said Best. "I don't think there is a viable dark-horse technology out there to replace the hard disk for the next 10 years," he said. "Any dark-horse technology is facing a moving target in hard disks, and that's traditionally a big problem for establishing a new technology."

"If you ignore the optical space, I don't see anything on the horizon that's a serious competitor to the hard-disk drive," added Kryder.