News & Analysis
Slideshow: Facebook remakes the server
Rick Merritt
10/3/2012 12:00 PM EDT
.
MENLO PARK, Calif. – Facebook showed EETimes its first working prototype of Open Rack, an energy efficient server that's intended to power data centers which can theoretically scale to more than 100,000 physical machines.
Amazon, Google and Microsoft also specify low cost, energy efficient servers and other data center gear. But they generally keep those designs private, aiming to get a proprietary advantage. Facebook's "open" angle isn't related to a Linux-like use of the word. Rather, it indicates use of modular power supplies, standard mechanical form factors and other elements any vendor can make. This means everything can be swapped out, without replacing the entire server.
[Designing with ARM? Get a 10% discount on ARM TechCon 2012, Oct. 30 - Nov. 1 in Santa Clara. Click here to register for the year's biggest and best live event for the ARM community. ]
OpenRack is very much a work in progress. The current rack uses Facebook’s third and so-far most compact generation of a two-socket x86 server. Engineers here already envision a fourth generation that shrinks the server design yet again. Ideally, the board would contain only processors and a PCI Express fabric, no Ethernet or other components—but that will require a variant of Express that is only on the drawing board today.
Each OpenVault tray has control electronics on a slide-out board. In future, the board could host an ARM-based controller to handle special storage functions. Facebook believes such an application could become its first use of an ARM SoC.
In the meantime, Facebook engineers are working on an interim step where hard disk boot drives are replaced by smaller, more robust solid-state drives. Specifically, they expect to move from 250-500 Gbyte hard drives—the smallest they can now get—to 128 Gbyte serial ATA SSDs which have enough capacity and might even be cheaper.
For storage at the rack level, one of the big milestones of the current design is OpenVault, a new design for an ultra low cost hard disk drive array.
Despite its big ambitions, Facebook’s small server engineering team is expanding just a bit but will stay lean, probably less than 40 people. "Our philosophy is to keep small fast teams--huge teams move slowly," said Matt Corddry, senior manager of hardware engineering who formerly held a similar job at Amazon. "We typically have less than ten people per team. We love to go fast and have small teams with everything they need to break rules and build things quickly, leveraging partners."
Click through to see the rest of the slideshow.
Next: The latest server sled
MENLO PARK, Calif. – Facebook showed EETimes its first working prototype of Open Rack, an energy efficient server that's intended to power data centers which can theoretically scale to more than 100,000 physical machines.
Amazon, Google and Microsoft also specify low cost, energy efficient servers and other data center gear. But they generally keep those designs private, aiming to get a proprietary advantage. Facebook's "open" angle isn't related to a Linux-like use of the word. Rather, it indicates use of modular power supplies, standard mechanical form factors and other elements any vendor can make. This means everything can be swapped out, without replacing the entire server.
[Designing with ARM? Get a 10% discount on ARM TechCon 2012, Oct. 30 - Nov. 1 in Santa Clara. Click here to register for the year's biggest and best live event for the ARM community. ]
OpenRack is very much a work in progress. The current rack uses Facebook’s third and so-far most compact generation of a two-socket x86 server. Engineers here already envision a fourth generation that shrinks the server design yet again. Ideally, the board would contain only processors and a PCI Express fabric, no Ethernet or other components—but that will require a variant of Express that is only on the drawing board today.
Each OpenVault tray has control electronics on a slide-out board. In future, the board could host an ARM-based controller to handle special storage functions. Facebook believes such an application could become its first use of an ARM SoC.
In the meantime, Facebook engineers are working on an interim step where hard disk boot drives are replaced by smaller, more robust solid-state drives. Specifically, they expect to move from 250-500 Gbyte hard drives—the smallest they can now get—to 128 Gbyte serial ATA SSDs which have enough capacity and might even be cheaper.
For storage at the rack level, one of the big milestones of the current design is OpenVault, a new design for an ultra low cost hard disk drive array.
Despite its big ambitions, Facebook’s small server engineering team is expanding just a bit but will stay lean, probably less than 40 people. "Our philosophy is to keep small fast teams--huge teams move slowly," said Matt Corddry, senior manager of hardware engineering who formerly held a similar job at Amazon. "We typically have less than ten people per team. We love to go fast and have small teams with everything they need to break rules and build things quickly, leveraging partners."
Click through to see the rest of the slideshow.
Next: The latest server sled
|
|
|
|
|
Navigate to related information
iniewski
10/3/2012 12:09 PM EDT
Cool slide deck Rick...rare case of being able to see inside a hardware box!
Sign in to Reply
SandunDhammika
10/3/2012 3:32 PM EDT
This is the same technology that used in backbone servers.
Sign in to Reply
tb1
10/3/2012 3:38 PM EDT
I used a link checker: dklafjs's link takes you to http://applehaters.blogspot.com/
Sign in to Reply
tb1
10/3/2012 3:42 PM EDT
I wonder if you could make a box that could take these opensource PCs, and make a cheap desktop PC.
Sign in to Reply
steveDS
10/3/2012 7:17 PM EDT
I would like to see the EMI test results...
Sign in to Reply
rick.merritt
10/4/2012 2:40 AM EDT
Check out the Open Compute Project for more details on Facebook's designs.
Are these designs something you plan to use?
Sign in to Reply
steveDS
10/4/2012 10:57 AM EDT
No. Just curious how much they would emit. I wonder if Facebook tested them? Since they don't sell them FCC certification would not be an issue.
Sign in to Reply
ttt3
10/4/2012 9:51 AM EDT
Would be nice to see a block diagram of the system.
Sign in to Reply
rick.merritt
10/5/2012 4:09 AM EDT
@tt3: Great idea. I'll check on that
Sign in to Reply
naperlou
10/5/2012 12:13 PM EDT
Actually, none of this is necessary. The drive toward higher density per rack is a problem. By doing that cooling problems are made worse. I was at an event on data centers at the University of Illinois at Urbana-Champagne and a speaker from Microsoft discussed their findings. For one thing, space is not an issue, except in Tokyo and Hong Kong. Thus, you are better spreading the system out a bit.
Another thing they should consider, since they are using their own software, is another processor architecture. There are many very effecient processors based on the ARM and POWER architectures that might run their code faster and use a lot less power.
Sign in to Reply
MrGeek
10/5/2012 1:49 PM EDT
Google has also done there own server computer system. You would think that the large server hardware companies would have a better solution for the data center issues. Heat generation is a major issue in data centers. The colder climates have a real advantage.
Sign in to Reply
GREAT-Terry
10/5/2012 11:16 PM EDT
Cool pictures! I like to see more on the system block diagram and more on the hardware related design. Is that really revolutionary?
Sign in to Reply
tb1
10/8/2012 12:28 PM EDT
"There are many very effecient processors based on the ARM and POWER architectures that might run their code faster and use a lot less power."
Facebook is reportedly playing with ARN and even Tilera processors in their servers.
Sign in to Reply
tb1
10/8/2012 12:28 PM EDT
(Oops: ARM not ARN)
Sign in to Reply
help.fulguy
10/8/2012 6:59 PM EDT
Hey Rick,
I am sick of your Pre-ARM articles. For once, can EETimes write un-biased article. You even have ARM Tech conference ad in every one of your article. Talk about unbiased reporting.....
Helpfulguy
Sign in to Reply
help.fulguy
10/8/2012 6:59 PM EDT
pro-ARM
Sign in to Reply
Battar
10/9/2012 11:16 AM EDT
A simple algorithm which automatically edits out extra exclamation marks in idiotic facebook posts could probably save them a few terabytes of storage. And thats just the beginning.
Sign in to Reply
BobsUrUncle
10/10/2012 7:52 AM EDT
Cool science project.. until someone gets electrocuted. Metal grills serve a purpose that these junior engineers will someday "discover".
I guess tearing down a product is considered innovation these days. I expected better from EETimes. I guess it's just a tabloid now. Facebook innovation? Don't make me laugh.
You can make your car get better gas mileage by ripping out the seats, airbags and brakes, but don't try selling it to anyone else.
Sign in to Reply