What interconnects are used with memory for HPC and AI?

High-performance computing (HPC) for artificial intelligence (AI) applications places extreme demands on memory connectors. They must deliver high-speed connectivity, high levels of signal integrity (SI), and support high-density solutions. This article reviews connector needs for Peripheral Component Interconnect Express 6.0 (PCIe 6.0), Serial Attached SCSI (SAS) including SAS 3.0, 4.0 and SAS/PCIe 5.0, and Data Rate 5 Synchronous Dynamic Random-Access Memory (DDR5 SDRAM).

While SAS/PCIe 5.0 connectors support data transfer speeds of 32GT/s for PCIe lanes and 24Gb/s for SAS lanes, next-generation PCIe 6.1 achieves 64 GT/s. Several new technologies were used to reach that new performance plateau, including:

Pulse Amplitude Modulation with four levels (PAM4) signaling combines two bits per clock cycle for four amplitude levels to encode 00, 01, 10, and 11. This contrasts with the non-return to zero (NZR) modulation used in earlier generations of PCIe, which used one bit per clock cycle and two amplitude levels to encode 0 and 1 (Figure 1).

Figure 1. To enable higher transfer rates, PCIe 6.1 has replaced NZR modulation (left) with PAM4 modulation (right). (Image: Rambus)

Even with connectors that deliver superior SI, the move from NZR to PAM4 can result in a higher bit error rate (BER). To mitigate this increase, PCIe 6.1 has added a forward error correction (FEC) mechanism that coordinates with a cyclic redundancy check (CRC). The new approach adds less than 2 ns of latency and enables PCIe 6.1 to maintain the same reach as earlier generations.

SAS connectors

SAS connectors feature reduced cross talk and enhanced performance with enterprise solid state drives (SSDs) and PCIe Gen 4 and Gen 5 connectivity. There are three primary SAS connector standards: Storage Networking Industry Association (SNIA) small form factor 8680 (SSF 8680), SFF 8681, and SFF 8639.

SAS 3.0 connectors have 29 positions and handle data rates up to 12 Gbps. They are compatible with the SFF 8680 specification and support both SAS and SATA drives. Features include hot plugging, blind mating misalignment compensation, and a retention mechanism.

SAS 4.0 connectors also have 29 positions and support faster data rates up to 24 Gbps. They have features like SAS 30 connectors but are compatible with the SFF 8681 specification. They are often used for SATA drives in mission-critical applications like high-performance computing for AI and bulk storage systems.

SAS/PCIe 5.0 connectors are compatible with SFF 8639 and support U.2 and U.3 applications. U.2 supports standard SAS and SATA-based spinning disks and solid-state drives (SSDs). U.3 uses the same connector and is a “tri-mode” standard that combines SAS, SATA, and nonvolatile memory express (NVMe) support into a single controller. These connectors support data transfer speeds of 32GT/s for PCIe lanes and 24Gb/s for SAS lanes.

DDR5

DDR5 modules are available in three electrical formats: unbuffered (U), registered (R), and load-reduced (L). The Joint Electron Device Engineering Council (JEDEC) Standard PS-005B—DDR5 288 Pin U/R/LR DIMM Connector Performance Standard defines DDR5 connectors for U, R, and L modules that support channels with transfer rates up to 6.4 GT/S.

The JEDEC standard defines the form, fit, and function of DDR5 connectors, including the mechanical, electrical, and reliability requirements for a connector mated to a 1.27 mm-thick module. The standard supports mounting multiple DDR5 modules on a circuit board, usually with 288-pin connectors. As with earlier generations of DDR memory, DDR5 is available in several DIMM configurations.

A notch in the center of DDR modules aligns with the DDR socket for proper mating. The notch in DDR5 sockets also prevents non-DDR5 modules from being installed in the system. DDR5 notches differ between U and R module types (Figure 2).

Figure 2. Different module keys prevent DDR4 modules from being inserted into DDR5 sockets and vice versa. (Image: Kingston Technology)

Summary

Various interconnect types are required to support the memory needs of high-performance computing and AI systems. Increasing storage densities and speeds have introduced new connector types for PCIe 6.1, SAS 4.0, and DDR5 memory technologies.