"Limits related to energy and power appear much looser and leave more room for improvement," says Markov. "And there are powerful ways to circumvent the assumptions behind these limits, which raises further research questions."
According to Markov, there are dozens of limits that, in fact, are unsurmountable but are so loose that they will not necessary affect computing at all — physics principles like Planck's length (which sets the fundamental limit on measurement), the Bekenstein limit (the maximum amount of information required to describe a physical system down to the quantum level), and the Schwarzschild radius (the radius of a sphere, which if smaller becomes a black hole).
"Some are relevant, but have some gotchas. For example, P!=NP" — whether problems whose solutions can be quickly verified by a computer can also be quickly solved by a computer. (P!=NP is one of the seven Millennium Prize Problems with a $1 million prize for the first correct solution.)
Using biological computational models like the brain reveals that it distributes memory (synapses) between every computing node (neuron) and is interconnect-limited. However, it compensates by being much more energy efficient -- by using slower switching speeds, low supply voltages, liquid cooling, and a very different power network.
(Source: Human Connectome Project)
As Markov told us:
But P!=NP is conjectured, not proven, and is focused on worst-case behavior. So, one can get around it through domain-specific computing and application-specific optimizations. Comparing Amdahl's Law [which predicts the theoretical maximum speedup using multiple processors] and Gustafson's limits [that computations involving arbitrarily large datasets can be efficiently parallelized] in parallel computing, and observing ongoing industry developments, one can see a type of natural selection, the survival of applications fittest for parallelism and those less affected by fundamental limits.
Other limits affecting chip building are not directly related to the size of devices, but to how they can be more efficiently interconnected — called the "tyranny of interconnect" by Markov.
Based on the speed of light, minimal physical size of a computing element, and the number of available dimensions, it bounds the speed-up you can get from parallel computing if you pack computing elements into the available space, and shows that many promises in parallel computing are unattainable if you happen to live in two or three dimensions. But it also shows that you can do more in three dimensions than in two dimensions, and the improvement is asymptotic. The important part is that stacking 2D layers does not give you such an improvement — you have to scale in the third dimension, just like you scale in previous dimensions.
Another aspect of speeding up computations is new materials, which can make interconnects faster and more energy efficient.
For example, carbon nanotube transistors provide greater drive strength. Even with metallic wires, this can simplify interconnect buffering, reduce wire delay, decrease energy consumption and the footprint of the entire circuit. On the other hand, fundamental limits tend to equally apply to new and existing technologies, so it is important to understand them before promising a new revolution in power, performance, etc.
Another approach that may forgo Moore's Law is emulating natural systems, which seem to often work better than engineer-designed semiconductors despite having limitations that are at odds with Moore's Law.
Biological systems are also subject to fundamental limits. For example, we know that human brain connectivity is 3D, and individual "devices" are quite big and slow. Just like modern integrated circuits, brains are interconnect-limited. The brain is much more energy efficient. It uses lower switching speeds, low supply voltages, liquid cooling, and a very different power network. It also needs to rest and chemically clean itself — the significance of this to computing is unclear.
We also know that the brain is very disappointing as a general-purpose computer. It can't multiply many 64-bit numbers per second, can't copy stored information in bulk, and can't be thinking a hundred thoughts at once (texting while driving is illegal for that reason). However, the brain is a great multimedia processor, handles uncertainty well, and is capable of intuition, creativity, and other types of high-level reasoning. Figuring how this is done leaves researchers more than enough work.
In the end, Markov suggests that when a specific limit is approached, the key to circumventing it is understanding its assumptions. For example, the International Technology Roadmap for Semiconductors (ITRS) should add the analysis of limits to make its predictions more accurate. After all, the ITRS initially predicted that the 45 nanometer node would run at 10 GHz speeds, a blunder that Markov suggests could have been circumvented by paying attention to the fundamental limits on energy resources, power dissipation constraints, and energy waste.
— R. Colin Johnson, Advanced Technology Editor, EE Times
While we could probably improve current manufacturing techniques, i personally set my eyes on one of the crazy options promising orders of magnitude improvment: Quantum computing, Optical computing(like optalysys does) , Analog brain like computing, etc.
The first problem is that Moore's law is not a law, but only an observation. We have reduced the cost of integrated circuits over the years and we will continue to do that with many inovations. But there is no guarantee that it will continue forever and we all knew that.