PCIe goes Clockless--Achieving independent spread-spectrum clocking without SSC isolation

PCI Express (PCIe) has established itself as the IO interconnect of choice for communication within the server and PC environment.  Today, an emerging trend among designers is extending PCIe beyond the PC/server while maintaining the advantages of simplicity, bandwidth, scalability, low power and cost.  One of the major system-level challenges in extending PCIe outside the box has been clock distribution between separated domains.  

While many PCIe devices can operate asynchronously, these applications use constant-frequency clocking.  The challenge gets more complicated when spread spectrum clocking (SSC) is needed. In systems required to operate with SSC, the only available option has been clock isolation.  This method adds complexity in component count, clock fidelity, and media selection.  In addition, the cable itself must operate in a constant frequency clock (CFC) domain.  This CFC domain can still represent a significant source of electromagnetic interference (EMI).   

Let’s look at the importance of introducing independent SSC operation to the ecosystem of PCIe and some of the cross technology advantages in performance, simplicity and cost that it can deliver.

Spread Spectrum Clocking

Spread spectrum is the process by which the system clock is dithered in a controlled manner so as to reduce peak energy content.  SSC techniques are used so as to minimize EMI and/or pass Federal Communications Commission (FCC) requirements.  While the overall energy is unchanged, the peak (tonal) power is reduced. The amount of peak energy dispersion is dependent on the modulation bandwidth, spreading depth and spreading profile. 

In the case of PCIe, the typical modulation profile is a 30 KHz-33 KHz, 0.5% down-spread clock. The modulation profile can be several types, but typically ends up triangular.  When extending high-speed data outside of an enclosure, copper cabling can significantly increase the amount of peak radiated energy.  Systems designers must either modulate the data exiting the box or resort to more costly cables with a high shielding index.  In the case of PCIe, until now the option of modulating the data traveling outside the box was not available.

How PCIe Deals with SSC

PCIe, fundamentally, is a short-reach, point-to-point-protocol that is typically synchronous. Under these conditions, spreading the system reference clock has minor impact on the overall links; each device undergoes nearly the same frequency deviation in approximate lockstep. To extend modulated clock architecture beyond the confines of the box, cable provisions must be made for sending a clock signal as well as data.  In addition to added cabling cost, it increases complexity in terms of not just buffers for maintaining clock fidelity, but also clock timing correlation between the transmitting and receiving devices.  (The PCIe specification provides recommended relative trace delay timing so as to keep correlation between transmitter and receiver).  Additionally, if systems with separate master clock domains need to communicate--two independent servers each with their own CPU clock for example--passing a clock between two master devices will not work. 

As mentioned above, while PCIe devices from vendors such as PLX have been used in asynchronous applications for several years, separated domains require the use of constant frequency clocking or the use of a PCIe switch feature called SSC isolation. With this feature, the system runs with SSC, but the cable does not.

While users might be familiar with the SSC isolation capability, independent SSC is significantly different and a potential boon for systems designers.  In SSC isolation, the clocking structure is doubled on each side of the link. One domain is preserved for the system. A second domain is required to transition between system boundaries. As shown in the diagram above, this transition boundary typically consists of an additional CFC clock chip on each side of the external channel.  The constant frequency allows each side of the link to operate asynchronously. While this functionality has proven highly valuable, several disadvantages arise:

• In the case of copper connections, radiated EMI can be significant, precluding the use of lower-cost cable options to meet FCC requirements.
• Multiple clock domains must be managed – one for SSC, another for CFC, thus increasing design complexity and component count.
• Often in uncontrolled system environments, illegal SSC modulation is found; typically systems will be found to be center-spread, rather than down-spread, representing a significant compatibility problem and difficult system management when looking to define the proper SSC/CFC clock values.
• In systems that can operate as SSC only, increasing cable provisions for an additional clock signal means added cost in cable, buffering and clock conditioning.

Having PCIe with Independent SSC operation alleviates these concerns.  As suppliers of PCIe switches, endpoints and hosts move forward, it is critical that clock simplification takes place.  With PCIe features such as SSC Free PCIexpress, neither clock management, additional clock chips and/or buffers nor protocol translations are needed – just simple scalability and connectivity, lower connection costs and higher density.

A conceptual PCIe reference implementation highlights one of many potential flexible consumer designs, all geared towards extending storage, running remote graphics or providing an expansion platform for the vast number of PCIe devices on the market.  It reflects how independent SSC can build upon today’s connectivity solutions, by reapportioning two alternate connectivity solutions targeted at distance and cost for PCIe purposes.

With simplicity and high density as objectives, the implementation achieves connectivity via an optical port (in this case, a dual x2 Avago McLink modules with optical USB connectors providing 32Gbps) and a copper port (single Molex x4 Mini-SAS HD SFF-8644 connector and cable providing 32Gbps).  Neither of these solutions employ additional structures for reference clock transmission.  However, each represent the type of port density, cost and/or reach expansion that PCIe has begun to target.  As demonstrated by this basic reuse, clock simplification is critical to reaching the objectives of an expanded PCIe ecosystem and improved economies of scale.