Short schedule "answers a lot of questions"
SAN JOSE, Calif. — Google’s new processors for accelerating machine learning were built under a tight schedule, said Norm Jouppi, a veteran chip designer and distinguished hardware engineer who led the effort.
“I joined Google 2.5 years ago, and there were already some efforts underway…The emphasis was on getting something out quickly,” said Jouppi in an interview with EE Times.
A decision was made about three years ago to make a custom chip to accelerate Google’s machine learning algorithms. Jeff Dean, a top Google software engineer, helped drive the project in its early days.
The chips, called tensor processing units (TPUs) after the Google TensorFlow algorithm they accelerate, have been running for more than a year in Google’s data centers. “The sked was pretty challenging, the team did really well on that—a short schedule can help focus you, it answers a lot of questions,” Jouppi said.
Before Jouppi arrived in September 2013, Google engineers had evaluated using CPUs, GPUs and FPGAs. “They decided the benefits of customization were great enough that they wanted to go straight to custom silicon,” he said pointing to a paper from Mark Horowitz of Stanford suggesting custom silicon can deliver improvements of an order of magnitude
Google may reveal some details about the TPUs in the fall, but for now it is keeping mum on their inner workings. For example, Jouppi would not comment on whether they handle training of neural networks. He would only say they are more efficient than standard parts because they use “lower precision only using as many bits as needed.”
Google may reveal details about its TPU in the fall. (Image: Google)
Next Page: Calling your chip in the cloud
Calling your chip in the cloud
The back of the PCI Express board carrying the TPU (Image: Google)
The good news given the tight schedule is Google’s chip designers hit their marks.
“It was just 22 days from when we first got tested parts to having them running in the data center. That doesn’t just mean powered on, but running apps—that was really exciting,” he said. In addition the chips use the “first mask set. We never needed any mask changes—the design team was really great,” he added.
The new accelerators are part of an increasingly diverse set of chips inside Google’s data centers.
“There’s more heterogeneity than you might think…we also have a lot of GPUs in that infrastructure—we have all kinds of things now, it’s not like in the old days,” Jouppi said referring to the more homogenous data centers once associated with the search giant.
Jouppi was part of the original Stanford team that designed the MIPS architecture. He holds 90 patents and is known for his work on memory subsystems at Digital Equipment, Compaq and later Hewlett-Packard where he worked on server design.
The chance to work at Google was a welcome return to chip design, he said in an online interview. “A lot of the basic [chip design] issues are still the same [today] but they have gotten more complicated, there are more rules of various types, and the numbers are a lot bigger – it’s sometimes a shock to talk about a million of something,” he told EE Times.
Jouppi said he cannot talk about any other semiconductor projects at Google, but the company is still hiring chip designers.
“We’re always looking for great engineers,” he said. “A lot of people are still unfamiliar we are doing these chip efforts.”
For Jouppi, one of the benefits of the work is making an impact. “As an engineer, it’s really great when you can pick up your cellphone and talk to your chip in the data center,” he said.
— Rick Merritt, Silicon Valley Bureau Chief, EE Times