News & Analysis
Comment
Wobbly
That is the whole point. They are redesiging the entire processor to be on a ...
david.may
what's people's opinion on this new Ram cube option given the new membership ? ...
Facebook likes wimpy cores, CPU subscriptions
Rick Merritt
6/21/2012 2:17 PM EDT
CPUs by subscription
In Facebook’s ideal future, it will be able to upgrade the CPUs in its systems every 18 months or even sooner, letting server DRAMs and network controllers remain in place for longer cycles. Today all three of those elements as well as power supplies are upgraded at once every three hyears because they are part of the same motherboard.
A new OpenRack specification the OpenCompute group launched recently would move at least the power supply from the board and into the rack. Facebook expects to get its first production OpenRack design into its data centers in July.
By the first half of next year, Facebook hopes to have a proof-of-concept system ready for test that will put on separate upgradable “sleds” the boot hard drives and network controllers. “The hardest thing to segregate is the CPU from the DRAM, but there are some technologies on the horizon to do that,” Frankovsky said.
In the medium term, Facebook foresees pulling CPUs in sockets on similar sleds with DRAM. The Web company expects to replace the CPUs where socket-compatible upgrades are available.
“The next phase is to use an edge connector that will not introduce incremental latency outside our timing boundaries,” said Frankovsky. “That could use a high-speed, low-latency optical interconnect, but no one knows what it would look like,” he said.
At that point, Facebook has proposed using a subscription model of buying processors “by clocks and cores per year.” The company would reuse as many old CPUs as it could but would sell or ask the processor vendor to re-sell the extra chips to another end user.
“This notion is one we just recently started sharing so we have had no real reaction to it yet,” said Frankovsky. “This is a thought that hasn’t graduated to an idea yet, but it could be win-win scenario,” he said.
Meanwhile, Frankovsky said he would like to see today’s network switches move from modules proprietary to the blade servers of various vendors to an open approach based on an interconnect such as PCI Express. I would love to see PCI Express switching and management get more mature,” he added.
In Facebook’s ideal future, it will be able to upgrade the CPUs in its systems every 18 months or even sooner, letting server DRAMs and network controllers remain in place for longer cycles. Today all three of those elements as well as power supplies are upgraded at once every three hyears because they are part of the same motherboard.
A new OpenRack specification the OpenCompute group launched recently would move at least the power supply from the board and into the rack. Facebook expects to get its first production OpenRack design into its data centers in July.
By the first half of next year, Facebook hopes to have a proof-of-concept system ready for test that will put on separate upgradable “sleds” the boot hard drives and network controllers. “The hardest thing to segregate is the CPU from the DRAM, but there are some technologies on the horizon to do that,” Frankovsky said.
In the medium term, Facebook foresees pulling CPUs in sockets on similar sleds with DRAM. The Web company expects to replace the CPUs where socket-compatible upgrades are available.
“The next phase is to use an edge connector that will not introduce incremental latency outside our timing boundaries,” said Frankovsky. “That could use a high-speed, low-latency optical interconnect, but no one knows what it would look like,” he said.
At that point, Facebook has proposed using a subscription model of buying processors “by clocks and cores per year.” The company would reuse as many old CPUs as it could but would sell or ask the processor vendor to re-sell the extra chips to another end user.
“This notion is one we just recently started sharing so we have had no real reaction to it yet,” said Frankovsky. “This is a thought that hasn’t graduated to an idea yet, but it could be win-win scenario,” he said.
Meanwhile, Frankovsky said he would like to see today’s network switches move from modules proprietary to the blade servers of various vendors to an open approach based on an interconnect such as PCI Express. I would love to see PCI Express switching and management get more mature,” he added.
|
|
|
|
|
Navigate to related information
david.may
6/21/2012 3:49 PM EDT
"“This notion is one we just recently started sharing so we have had no real reaction to it yet,” said Frankovsky. “This is a thought that hasn’t graduated to an idea yet, but it could be win-win scenario,” he said."
i don't understand why it took they so long to realize its good , i assume these Facebook execs didn't even realize that generic ARM on SODIMM has existed since 2009 or earlier even
for instance this as one single example
http://www.linuxfordevices.com/c/a/News/Direct-Insights-TritonTX51/?kc=rss
it cant be that hadr for facebook to make a rack of ARM on SODIMM carrier with 4 SOC per SODIMM and move the RAM on these to the main daughter server board instead of inline on the SODIMM next to the new ARM Cortex A15's they could use today.
why wait until late 2013 when they can have these ARM ON SODIMM made and delivered with a phone call today to start testing, ARM ON SODIMM is obvious for a massive parallel Server in a U2 case densely packed in to generic server SLED carriers
Sign in to Reply
green_is_now
6/24/2012 11:33 AM EDT
testing
Sign in to Reply
Mahanth Gouda
6/22/2012 6:55 AM EDT
Well said "david.may". I totally agree with you.
Sign in to Reply
selinz
6/22/2012 10:33 AM EDT
Let's not forget that facebook, so far anyway, doesn't make hardware. Google and facebook seem to do a lot of "remote computation" in their web experiences so I'm hoping that this will not increase user latency more.
Sign in to Reply
rick.merritt
6/22/2012 11:27 AM EDT
Facebook is the most vocal of all the big Web 2.0 data centers.
If they are doing this, I suspect, Google, Amazon, MSoft, Yahoo and others may be too.
Question is, in 2013, will it be Intel Atom (Centerton), Tilera, Marvell, Calxeda. That's list in order of the most likely candidates in my book. AMCC, AMD and Samsung probably mwon't have anything tested and ready to go until 2014, IMHO.
Sign in to Reply
MikeSmith2011
6/22/2012 2:55 PM EDT
Subscription model for CPUs based on clocks, allowing vendors to resell CPUs. Sounds too radical an idea. I may be the contrarian here but most of the ideas coming out of FB seem outlandish. Are there any TCO calculations that show how a more complex system design that can plug and play memory/CPU/NICs is better?
Sign in to Reply
rick.merritt
6/22/2012 4:55 PM EDT
Dunno about any calculations. But Frank said he'd ideally like to replace CPU in less than 18 months. Networking, memory and disk might last three years of more.
Sign in to Reply
samfuller
6/22/2012 6:14 PM EDT
As I was reading this I thought that the subscription model sounded like leasing a car. FB would get the first 18 months then you would pass it on to the next owner. Seems like this would work better as a virtual service than as actual hardware swapping.
Sign in to Reply
ttm123
6/22/2012 7:50 PM EDT
can't wait to see the flood of used processors on Ebay/Craigslist
Sign in to Reply
t.alex
6/22/2012 10:13 PM EDT
This is so fun:) will we Like or not like the core
Sign in to Reply
ese002
6/23/2012 1:43 PM EDT
I don't see this happening. Existing x86 servers already have socketed CPU's and they have for far longer than Facebook has existed. But CPU's are seldom upgraded except for some hobbyists. The reason is that replacing components (especially CPU's with their cumbersome heat sinks) is labor intensive and the performance boost is limited. It is almost always more productive to wait and replace motherboard, cpu, and memory all at once.
Now it is true that Atoms and ARM processors don't have standard sockets. It would be nice to be able to be mix and match low power cpus with motherboards in the same way we mix and match high power cpus with their motherboards today. But, once installed, I don't expect that Atmoms will move around any more than Xeons already do.
Sign in to Reply
Wobbly
9/10/2012 11:25 AM EDT
That is the whole point. They are redesiging the entire processor to be on a generic carrier, so that it can be easily swapped. On low power, high efficiency cores, the heat sink requirements are much less, so the physical design is easier.
Sign in to Reply
green_is_now
6/24/2012 11:40 AM EDT
Why replace? Why not expand with the new agnostic topology? (not just the cpu per say but the echosystem that surounds it as well.
Is the wimpy cpu just about serving up utube and facebook type web page feeds?
whats the need difference for a data streaming for a financial application for example v.s. a utube video?
Sign in to Reply
green_is_now
6/24/2012 11:50 AM EDT
Probably more about supporting upstream procesors that can be pin compatible in a family.
Quad ARMs are not ready except from Invidia.
Duals are abundant and will be priced accordingly.
If you can drop in a quad A9 in 18 months it will be cheap because everyone will have figured it out. The dual 15 with dual A9 would be nice to be in a version that can drop into the same slot. Evn if it gives up some features of its bigger pin count brothers...and on and on. The CPU churn rate is mutch faster than the rack-server infrastructure evolves. so it makes sense to want to put a few releases of CPU's in the same infrastructure. Or rip it out each time you want to go to the next 10% improvement?
Sign in to Reply
jeremybirch
6/26/2012 12:00 PM EDT
surely the problem here is that if you upgrade a 1 core cpu card with a 2 or 4 core one, that you will be expecting 4 times the memory bandwidth. It is unlikely you would have designed your system to have 4 times the necessary memory bandwidth to start with, so just updating the cores will be less than effective as they will clash for memory resource. I presume that most of the work FB's servers are doing are very short transactions (eg when someone puts a short message on a page) so processors will be constantly having to get new data as they swap from serving one user to serving another. This implies huge disk and/or internal network bandwidth to fetch that user's page.
Unless of course they are planning to keep each of the 1billion accounts alive in DRAM in their farm 24/7 - in which case I expect the memory manufacturers will be rubbing their hands with glee.
Sign in to Reply
david.may
6/26/2012 3:16 PM EDT
that seems simple enough when you make these rack of ARM on SODIMM carrier with 4 SOC per SODIMM, and move the RAM on these to the main daughter server board
you make sure to use Wide IO as per Samsung's ARM speced Block.
the key is to keep as much generic kit as you can from the mobile space upgraded to the servers space requirement today, then slowly or even faster move all of today's dual/quad mobile and static SOC home devices into that same ARM on SODIMM + daughter board configuration...
presto facebook or their future agents can sell these older ARM on SODIMM SOCK + daughter boards to their end users :) every one wins and make a profit as one example.
Sign in to Reply
david.may
6/29/2012 2:04 AM EDT
what's people's opinion on this new Ram cube option given the new membership ? and the fact they say it has even greater potential speed than wideIO
http://denalimemoryreport.wordpress.com/2012/06/28/arm-hp-and-sk-hynix-join-hybrid-memory-cube-consortium-hmcc-first-spec-due-by-end-of-year/#comment-350
Sign in to Reply
Kevin Krewell
6/24/2012 3:53 PM EDT
Hi Rick,
A minor correction: Tilera's processor core has a "MIPS-like" instruction set, but it is not a MIPS licensed processor, so calling it MIPS-based is misleading.
Ironically, one of the first "wimpy core" server processor was the "Niagara" UltraSPARC-T1, but that line of servers has not gotten much attention of late.
Sign in to Reply
rick.merritt
6/25/2012 1:25 AM EDT
@Kevin Krewell: Thanks for the clarification re Tilera.
Think we will see Oracle or an Oracle licensee do anything with Sparc in this market?
Sign in to Reply
Kevin Krewell
6/25/2012 3:03 PM EDT
Rick,
I'm not covering Oracle/SPARC for The Linley Group, so I haven't talked to Oracle about it. Based on what I know about Oracle, I wouldn't think it's very likely.
Kevin
Sign in to Reply