News & Analysis
Comment
Zuben
You seem to be confused regarding the difference between programming a computer ...
allinenriq
Say thanks a lot for your time and effort to have put these things together on ...
IBM demos cognitive computer chips
R Colin Johnson
8/18/2011 12:01 AM EDT
PORTLAND, Ore.—By replicating the functions of neurons, synapses, dendrites and axons in the brain using special-purpose silicon circuitry, IBM claims to have developed the first custom cognitive computing cores that bring together digital spiking neurons with ultra-dense, on-chip, crossbar synapses and event-driven communication.
IBM's effort is the crowning achievement of a "phase zero" and "phase one" contract with the Defense Advanced Research Project Agency (DARPA) to build Systems of Neuromorphic Adaptive Plastic Scalable Electronics (SyNAPSE), IBM and its university partners—Columbia University, Cornell University, and University of California-Merced, and the University of Wisconsin-Madison—now enter "phase two," which extends their efforts for another 18 months with a new infusion of $21 million in funding. DARPA funding the project has received thus far, including the new funding, amounts to $41 million in total.
The eventual goal is to create a brain-like 10 billion neuron, 100 trillion synapse cognitive computer with comparable size and power consumption to the human brain.
"We want to extend and complement the traditional von Neumann computer for realtime uncertain environments," said Dharmendra Modha, project leader for IBM Research. "Cognitive computers must integrate the inputs from multiple sensors in a context dependent fashion in order to close the realtime sensory-motor feedback loop."
Though IBM claims its custom cognitive computing cores are the first of their kind, a rival European program using conventional ARM cores called SpiNNaker— for spiking neural network architecture—was announced last month.
Traditional von Nuemann computers are ill-equipped to deal with the multiple simultaneous data streams coming in from sensors today, but brains handle these easily by distributing processing and memory among its neural networks. In particular, sensors feed neurons down input lines called dendrites.
IBM's Cognitive Computing Chip, at about 3-mm wide, has demonstrated the ability to play (and win) against a human in the game "Pong" and can also read a written letter 7, even when written in various ways.
The neuron integrates over these inputs until a threshold is exceeded, at which point it fires a pulse down its output axon, which is weighted by the synapses connected to other neurons. Pattern recognition is accomplished by the synapses "learning" which connections are used most often, which causes them to grow stronger, while seldom used connections wither away. In this way, the neural network closes the sensory-motor feedback loop, since once a pattern is recognized from the sensor inputs, the output motor neurons mobilize a response.
IBM replicates the brain's architecture by using a crossbar array to hold the synapses, which then learn which sensory patterns correspond to which desired motor control outputs. The crossbar array connects the neurons to sensor inputs by integrating over a large fan-in of dendrites, then firing output pulses down axons which feed individual synaptic connections to the other neurons in the network.
"Synapses are realized with a crossbar array, in which the vertical lines are the input dendrites and horizontal lines are the output axons," said Modha. "Each neuron fires in order to communicate with the other neurons which fully integrates memory with processor, instead of separating them like von Neumann."
IBM's effort is the crowning achievement of a "phase zero" and "phase one" contract with the Defense Advanced Research Project Agency (DARPA) to build Systems of Neuromorphic Adaptive Plastic Scalable Electronics (SyNAPSE), IBM and its university partners—Columbia University, Cornell University, and University of California-Merced, and the University of Wisconsin-Madison—now enter "phase two," which extends their efforts for another 18 months with a new infusion of $21 million in funding. DARPA funding the project has received thus far, including the new funding, amounts to $41 million in total.
The eventual goal is to create a brain-like 10 billion neuron, 100 trillion synapse cognitive computer with comparable size and power consumption to the human brain.
"We want to extend and complement the traditional von Neumann computer for realtime uncertain environments," said Dharmendra Modha, project leader for IBM Research. "Cognitive computers must integrate the inputs from multiple sensors in a context dependent fashion in order to close the realtime sensory-motor feedback loop."
Though IBM claims its custom cognitive computing cores are the first of their kind, a rival European program using conventional ARM cores called SpiNNaker— for spiking neural network architecture—was announced last month.
Traditional von Nuemann computers are ill-equipped to deal with the multiple simultaneous data streams coming in from sensors today, but brains handle these easily by distributing processing and memory among its neural networks. In particular, sensors feed neurons down input lines called dendrites.
IBM's Cognitive Computing Chip, at about 3-mm wide, has demonstrated the ability to play (and win) against a human in the game "Pong" and can also read a written letter 7, even when written in various ways.
The neuron integrates over these inputs until a threshold is exceeded, at which point it fires a pulse down its output axon, which is weighted by the synapses connected to other neurons. Pattern recognition is accomplished by the synapses "learning" which connections are used most often, which causes them to grow stronger, while seldom used connections wither away. In this way, the neural network closes the sensory-motor feedback loop, since once a pattern is recognized from the sensor inputs, the output motor neurons mobilize a response.
IBM replicates the brain's architecture by using a crossbar array to hold the synapses, which then learn which sensory patterns correspond to which desired motor control outputs. The crossbar array connects the neurons to sensor inputs by integrating over a large fan-in of dendrites, then firing output pulses down axons which feed individual synaptic connections to the other neurons in the network.
"Synapses are realized with a crossbar array, in which the vertical lines are the input dendrites and horizontal lines are the output axons," said Modha. "Each neuron fires in order to communicate with the other neurons which fully integrates memory with processor, instead of separating them like von Neumann."
|
|
|
|
|
Navigate to related information
stixoffire
8/18/2011 4:30 AM EDT
Aye'll Be Back.
Sign in to Reply
rbtbob
8/18/2011 7:42 AM EDT
Someone there at EE Times should ask Ron Neale about this part; "...the more conductive it becomes..."
Sign in to Reply
Volatile Memory
8/18/2011 8:56 AM EDT
Naah, even Apple II's 1Mhz processor could beat humans at "Pong" and read a written letter 7! This is just another scam on the taxpayer, and IBM should be ashamed! The so-called cognitive computer is nothing more than an underpowered curve-fitting device that gets stuck in local extrema, as IBM knows very well. A multi-core CPU with enough DRAM beats IBM's monstrosity in any task, any time.
Sign in to Reply
Zuben
1/17/2012 12:49 PM EST
You seem to be confused regarding the difference between programming a computer to beat people at Pong (which obviously happened when the Pong console came out) and having a computer learn to do the same thing. That's fine; being confused is fine. Matched with your arrogance, however, and your confusion becomes tiresome.
Sign in to Reply
R G.Neale
8/18/2011 9:29 AM EDT
Mr rbtbob-I am aware, and I tried to cover at least one application of a programmable resistance device, the PCM, in neural applications with the work I reported in:-
http://www.eetimes.com/design/memory-design/4218114/PCM-Progress-Report-No-4--Brains
Here is my quote from PCM PR#4 that I think is applicable in light of the present stagnant state of commercial PCM product development
“If, going forward, the dreams of neural network emulation are to be fully realized, the challenges to PCM device designers in terms of precision, discrimination and scaling will exceed, by far, anything that has been accomplished to date.”
A quote that is also applicable to all programmable resistance devices, including, CBRAM and ReRAM. Also, with respect, I think you should also be reminded that for the synapse, timing between pre- and post-synaptic pulses as well as conduction change as a function of usage is important.
Sign in to Reply
Volatile Memory
8/18/2011 11:43 AM EDT
PCM brings nothing to the so-called cognitive chip. Even if the cognitive chip made sense (which it does not), its value would be in the connectivity per sq inch (i.e., number of "synapses"), not the storage/counting media.
Sign in to Reply
rbtbob
8/18/2011 1:43 PM EDT
On reading your explanation of what Stanford had demonstrated, I was amazed that they could perform a resistance change using such a large number of pulses AND, if I understand your analysis, get the device to repeat the cycle enough to demonstrate a workable functionality.
Now the question is whether IBM is pursuing Stanford's scheme or some other? Also, could directional current pulses be used to enable both additive and subtractive resistance changes?
Sign in to Reply
mdkosloski
8/18/2011 10:09 AM EDT
I can't help but find it a bit odd that IBM is still trying to duplicate probabilistic, over-complete, non-orthogonal, impulse integration systems using perfectly ordered and organized grids of binary devices. Might as well write the whole thing in software at that point. Biological neurons don't send signals in one or two defined routes, rather many directions randomized from neuron to neuron often including back to the neurons that originated the signal. It is interesting to note, though, that their learning algorithm does strengthen or "prune" pathways based on use.
Sign in to Reply
rbtbob
8/18/2011 10:29 AM EDT
Just to keep things straight, almost all neurons send OUT only one signal along one axon. The axon branches at the end and connects to the dendrites of many other neurons. Neurons may have thousands of dendrites receiving signals from other neurons (or receptors) Axons and synapses are like PCM, there is no possible way they could actually work :-)
Sign in to Reply
Volatile Memory
8/18/2011 11:45 AM EDT
rbtbob: Axons and synapses work. PCM doesn't. As demonstrated by JPL/NASA, Samsung, and even Numonyx/Micron.
Sign in to Reply
R_Colin_Johnson
8/18/2011 1:23 PM EDT
This is IBM's first generation device, intentionally created to transfer its supercomputer simulations to a hardware platform. As their simulations become more detailed, IBM will have to deal with all the mentioned issues going forward. (And yes, it is relatively easy to program a computer to play Pong or recognize a numeral, which is why these were good metrics for a very simple chip learning a task on its own.)
Sign in to Reply
pixies
8/18/2011 4:55 PM EDT
$20M for Phase 0 and Phase I combined, not bad. I wonder what's the goal for Phase II. It will be exciting to see a machine with 10B neurons built and find out what it can do.
Sign in to Reply
Code Monkey
8/18/2011 4:58 PM EDT
To the lab guys: If you drop the chip and it breaks, don't substitute the one marked "Abnormal - Do Not Use".
Sign in to Reply
Bob744
8/18/2011 5:07 PM EDT
Will the device be made out of a platinum/iridium alloy?
Sign in to Reply
BrainiacV
8/18/2011 5:15 PM EDT
I dunno, it's hard to tell what you've taught a neural circuit. It's easy teaching it the difference between a bruised and unbruised apple. I once heard the story of how they tried to teach a neural program to detect tanks on a field. They showed it fields with and without. When it came for a field trial it failed spectacularly. Going back over the input data, the best guess is that they had taught it how to tell a sunny day from a cloudy day, which turned out to be the difference when the tanks were on the field or not.
Sign in to Reply
R G.Neale
8/18/2011 5:39 PM EDT
Mr Rbtrob- There were two parts to the Stanford synapse experiment. The first was to demonstrate 100 level resolution. The second to demonstrate that the synapse characteristics could be reproduced, for this they used 15 pulse trains. I reduced it to fewer for the purpose of my explanation and illustration. I think the use of epitaxial regrowth of the same crystals, as illustrated in one of my figures is the way to obtain reproducibility and 100 level resolution. However, that path tend to lead to the conclusion that PCRAM might offer a superior solution.
Sign in to Reply
Frank Eory
8/18/2011 5:44 PM EDT
A pure digital chip that is supposed to emulate neurons, synapses and dendrites? As mdkosloski said, they might as well just do the whole thing in software. Oh wait, they did.
So I guess the point of building the chip was just so they could eventually build a 10B neuron/100T synapse machine the size of a shoe box? They could simulate that in software too...it just takes more computers to do it.
Sign in to Reply
markhahn
8/18/2011 7:39 PM EDT
I'm disappointed eetimes doesn't have something more technical about the architecture here. are the weights stored in registers? the device is said to be all digital, so doesn't an update cycle require a lot of flops? isn't a fan-out of 10,000 pretty hard to drive? why is 10Hz an adequate update rate (it certainly doesn't match real neurons.)
if this thing is structured like an xbar, does that mean there's some kind of multiplier at each crossing? (the weights and summation are digital, right?)
Sign in to Reply
resistion
8/19/2011 2:21 AM EDT
I think I missed something, such as how the neurons are implemented. Phase change? Charge accumulation?
Sign in to Reply
resistion
8/19/2011 9:21 AM EDT
Ok it's just SRAM and CPU, specifically wired, apparently.
http://www.scientificamerican.com/article.cfm?id=inside-ibms-cognitive-chip
Sign in to Reply
entity279
8/19/2011 3:19 AM EDT
Implementation aside, why is this any better than any regular (software, hardware or.. FPGA) ANN?
Sign in to Reply
R G.Neale
8/19/2011 1:14 PM EDT
I have just been asked to explain the final sentence in my posting above, there was a typo. It should have read: "However, that path tends to lead to the conclusion that CBRAM might offer a superior solution." I meant the Conducting Bridge RAM, that offers, by electrochemical action the precision to add and remove individual layers of atoms. Without that is the need for melting, high temperatures and high current densities associated with "reset" in conventional PCM. Apologies.
Sign in to Reply
resistion
8/19/2011 6:03 PM EDT
That's an interesting implementation there, Ron. I wonder if there is also any overlap there with memristors.
Sign in to Reply
Charles.Desassure
8/19/2011 4:27 PM EDT
Thanks for this article. I think IBM is on to something big here.
Sign in to Reply
TFCSD
8/20/2011 5:25 PM EDT
There are a few morons (managers) who could use a few more neurons. Could these chips be used on the pointy haired boss so he can understand Dilbert the engineer?
Sign in to Reply
Amir Rahat
8/21/2011 4:41 AM EDT
So little progress has been made since 1957 and the Perceptron! See the original papers or check out Wiki http://en.wikipedia.org/wiki/Perceptron
Sign in to Reply
allinenriq
8/24/2011 2:57 AM EDT
Say thanks a lot for your time and effort to have put these things together on this blog. Mary and I very much loved your ideas through the articles on certain things.
http://www.elektronische-zigarette24.de
elektrische Zigarette
Sign in to Reply