PCIe goes Clockless--Achieving independent spread-spectrum clocking without SSC isolation

Independent SSC Demonstration

Figure 5 shows the two five-slot expansion boards. One card is enclosed in a standard ATX CPU case, while the other is left open for clearer observation. There is no CPU in either expander card – only a Gen3 switch with upstream and downstream configurable slots. (Data is provided to each expander via the server platform reference card, shown in Figure 4.) To the left of the switch are two daughter-card host assemblies.  Either of these host assemblies can be configured to operate as an upstream port, with the remaining port useable for cascading of expansion cards.  

An interchangeable range of daughter cards, as shown below (Figure 6), make for a flexible demonstration vehicle to interconnect between alternate connection options. The SSC modulated clock is created from an external Texas Instruments CDCE925 evaluation board and inserted via SMA to the expander card configured for Molex Mini-SAS connectivity. The onboard CFC clock is disabled.   The second (optical) expander uses the on-board CFC clocking as system reference.  As a result, this demonstration shows three separate clocking domains operating independently through the upstream switch--CPU SSC, copper expander SSC, and optical expander CFC--all without separate clock-management configurations.  SSC is preserved on the copper cable.

What’s significant about this demonstration are:

• Low-cost Avago x2 McLink modules create simple 16Gbps PCIe bandwidth/ USB connector.
• Molex Mini-SAS-HD (SF8644 6Gbps) cabling is capable of supporting PCIe Gen3 (x4 lane width) at two meters and beyond.
• Independent clocking opens the door for wider, lower-cost connectivity options, which can benefit from cross-protocol economies of scale.
• Only the link between the expansion box and the server is required to carry traffic at Gen3 and will transmit/receive on separate SSC domains; while independent SSC is not an industry standard, this highlights the direction and standards to which PCIe devices must migrate.
• The clock-less link reaches Gen3 operation via the standard PCIe linkup progression from Gen1 to Gen3.
• All other PCIe cards inside the expansion bay or in the server can operate with the native spread clocking and at any PCIe (Gen1, Gen2, or Gen3) negotiated rate.

The Result

In the demonstration, both 0.5% down-spread and center-spread clock modulation was used at the expansion board and shown to have no difference in link integrity.  While the PCIe specification calls for modulation frequencies to not exceed 33 kHz, at 0.5% down-spread, the TI synthesizer had a fixed modulation of ~ 30Khz.  Interestingly, the CPU box was tested and found to have 0.5%, center-spread modulation.  While the PCIe specification only allocates for down-spread clocking, rather than look for another PC, this clocking was used (an indication of what can happen with real-world management of disparate systems).

The first two plots (Figures 7 and 8, above) show the SSC on each side of the link. The motherboard PCIe clock modulation was observed via SMA breakout card from a PCIe slot (not shown).  A spare output buffer on the expansion board shows the synthesizer SSC used on the copper configured expansion card.

In the absence of more costly time interval analyzer tools, the scope (Figure 9) plot shows a simple means to verify a modulated reference.  Here, the scope is set to delay trigger mode, with approximately 450ns of delay.  With delay enabled, the time between scope triggering and scope sampling result in random placement of the waveform.  In the absence of SSC, the delayed trigger output and the non-delayed output will have similar profiles because the deviation of the waveform is small.  In this example, the spread modulation has resulted in significant clock width increase, which is indicative of SSC being enabled.

The Result

Figures 10 and 11 are internal eye monitor measurements of the CFC optical receiver (USB) eye and the down-spread SSC copper cable (Mini- SAS-HD) eye, taken at the PCIe server card.  Because the server has a center spread SSC domain, multiple link domains are observed: center spread SSC to down-spread SSC and center-spread SSC to CFC.

With the three systems operating on separate spread clock domains, no change in the link error performance or significant eye quality reduction was observed.

Key points of this demonstration are:

• The link between the expansion box and the server is the only link required to carry traffic at Gen3 and will transmit/receive on separate SSC domains.
• The clock-less link reaches Gen3 operation via the standard PCIe linkup progression from Gen1 to Gen3.
• If operating over a copper link, EMI suppression would be preserved, thus reducing the need for higher-performance shielding or concerns about clock distribution and integrity.
• For the optical interface, Gen3 backchannel tuning isn’t needed; fixed equalization sets the electrical portion of the full optical path.
• All other PCIe cards inside the expansion bay or in the server can operate with the native spread clocking and at any PCIe (Gen1, Gen2, or Gen3) negotiated speed.
• While an x2 optical McLink was targeted in this application for consumer consideration--effectively doubling the PCIe bandwidth over Thunderbolt--SSC Free can be scaled to higher-density PCIe lane options, optical and alternate copper connectivity solutions.

PCIe vendors working to make Independent SSC operation a standard functional feature of next-generation of devices, with the goal that the entire PCIe ecosystem will join, and benefit from, this trend.  System clock management capabilities from for the PCIe market are now ready for prime time

About the Author

Reginald Conley is vice president of applications engineering at PLX Technologu.  He can be reached at rconley@plxtech.com.