@Rick Merritt: it was good to see you at GreenNet last week in the City.
Question: what is the Ethernet PMD for the switch? RJ-45? If that is the case, then your statement needs revision:
"Switch X consumes about 2W per 56 Gbit/s Infiniband port, 1.5W on 40GE ports and 0.62W on 10GE ports. That's about on par with competing Ethernet devices some of which require external physical layer components that the Switch X integrates."
Because the 10Gig RJ45 ports today consume more than 5W.
If Switch X supports other (than RJ45) PMD's for Ethernet (like SFP+), then the power numbers look reasonable; I can also see how Mellanox leveraged its effort in IB switches to develop a product for 100G/40G Ethernet.