In a keynote speech here Tuesday (Sept. 25) at the International Conference on Computer Design (ICCD), Shen said microprocessor researchers are at a crossroads, moving toward deeper pipelines with higher frequencies, a trend which can impact instruction-level parallelism and overall processing efficiency.

Since the days of the 50-MHz Intel 486, processor performance has improved by 75 times as frequencies have multiplied by about 50 times. Improvements in process technology, supporting faster clock speeds, have accounted for 13 of that 75x improvement, and improved microarchitectures another 4x improvement. Instruction-level parallelism has shown only a 1.5x improvement over that time, as companies have emphasized deeper pipelines and higher clock frequencies.

Instruction-level parallelism requires a wider pipeline, which increases the complexity of each stage, rather than the thinner, multiple stages of a deep pipeline. "As you slice the pipeline thinner, you increase the latency and lower the instruction-level parallelism," Shen said, noting that his views were personal and did not represent Intel Corp.'s future product road map. Shen taught at Carnegie Mellon University for 18 years, heading up the microarchitecture research team there before joining Intel Labs about a year ago. Based in Santa Clara, Calif., Shen manages Intel Labs staff located in Santa Clara, Hillsboro, Ore., and in Austin, Texas.

The Pentium 4 processor has struck a good balance, he claimed, with a pipeline depth of around 25, frequencies moving past 2 GHz, and support for instruction-level parallelism as well.

"With the P4, Intel spent a lot of time incorporating instruction-level parallelism. Intel made the right decision: The higher frequency resulted in higher performance," he said.

Performance is usually defined as frequency times the instructions per cycle. "There is a point at which we don't want to pipe that deep," Shen said, adding that researchers are questioning, how deep can you go?

One answer, according to a predictive model by Intel processor architect Ed Grochowski, is that microprocessor pipelines may go to 57 stages before performance gains start to fall off. The 57-stage number — a take-off on ketchup maker Heinz' 57 Varieties — is not to be taken as a hard number. "It is 57, plus or minus 20, but the key is that we may not be able to go out past 50 pipeline stages without losing performance," Shen said.

"There are a number of approaches, and perhaps the best way of saying it is that there are tensions between each of these approaches. For example, there are tensions between hardware complexity and doing more in the compiler. But while it may take several years to develop new hardware, to recompile all of the existing software may take much longer. Between a new microarchitecture and a new compiler, there may be a kind of impedance mismatch" in terms of how long it takes to tune existing applications to a new compiler, Shen said.

Other tensions exist. Some argue that putting several processors on the same die will prove advantageous as several billion transistors become available. Others argue that a single massive core can run multiple threads, but the challenge then is to develop applications that support multi-threading.

Chip-level multiprocessing (CMP) and simultaneous multi-threading (SMT) can be combined to process multiple threads on multiple cores, but then validation time may increase. And since microprocessor teams must hit a certain level of performance within a certain time window, the time to validate the processor is growing more critical.

Also, in an era when power consumption may dictate the course that MPU design teams take, Shen said, "It is not clear whether SMT or CMP is more efficient in terms of power. In the end the solution may come from all different angles."

Many engineers are working to improve the memory pre-fetch capabilities, as well as improving the efficiency of the branch prediction engine. It may be possible to append pre-fetch threads to support what Shen called "speculative pre-computation."

"Forces are pulling us in various directions, and it is not clear what is the obvious path," Shen said.

Microprocessor designers have engaged in a decade-long discussion about the relative merits of pushing frequency or instruction-level parallelism, known as the "speed demon versus Braniac" debate.

"My point is that it is not so much one against the other. We need both instructions per cycle and frequency, and it is a real delicate balancing act. The question I am raising is, in what new and clever ways can be combine the two?"

As the debate continues, power dissipation may become an overriding concern. Shen said that as processors evolve over the rest of this decade into huge chips incorporating several billion transistors operating at frequencies of 10 to 30 GHz, power consumption will increase exponentially, putting greater emphasis on processor efficiency rather than brute frequency increases.