SAN JOSE, Calif. — Hewlett Packard Enterprise (HPE) has built a working version of The Machine, its research prototype for a big-data server. It packs a whopping 160 terabytes of main memory to spawn new ways to program systems at a time when Moore’s law is slowing and a half-dozen new non-volatile memory architectures are in the works.
The system packs 40 32-core ARM-based Cavium ThunderX 2 processors linked to a giant DRAM pool. They are spread across four enclosures using photonics ASICs and FPGA-based memory controllers that HPE designed in-house. It marks the second time that the company has shown a major experiment with an ARM server after its Moonshot project in late 2014.
“It’s an order of magnitude bigger than anyone else has done … no one’s ever built a unified memory subsystem at this capacity before,” said Kirk Bresniker, chief architect of the machine and a research fellow at HPE. Each Cavium SoC “can reach any data on the memory fabric via byte-level transactions for executing its instructions.”
The memory fabric is based on an early draft of the GenZ specification, which won’t be final until late this year. The ad hoc standard was announced by a consortium last year as an agnostic interconnect for a variety of magnetic, resistive, and other persistent memories in the works.
Intel and Micron have started shipping novel 3D XPoint memories initially in solid-state drives, and Western Digital and startups have talked about ReRAMs mainly still in the labs. But so far, “no other memory is mature and available in quantity at this point in time, so we went with DRAM” said Bresniker, admitting that DRAM is too pricey for use in a commercial system at this capacity.
Initial tests show that the 160-terabyte system can run large-scale data-processing jobs faster than traditional computers. However, HPE did not release latency or other performance data because it is still validating the FPGA code that implements the GenZ fabric.
The 700-page GenZ spec spans uses cases from handheld and embedded systems to exascale supercomputers. Memory semantics for an ARM-driven system form part of HPE’s contribution to the spec.
Next page: Photonics chip one of system’s innovations
HPE's Machine spans four enclosures linked via GenZ and a proprietary photonics interconnect. (Images: HPE)