SMC applicability

How it works

SMC runs parallel to the TCP/IP stack. Instead of processing packets in software, SMC offloads protocol processing to the underlying hardware, transmitting data by directly reading and writing remote shared memory. The result is lower latency, reduced CPU overhead, and higher network throughput.

SMC was open-sourced by IBM to the Linux community and is enhanced in Alibaba Cloud Linux 3. SMC is compatible with standard socket interfaces. By intercepting the socket() system call to change the socket family and protocol parameters, existing TCP applications switch to the SMC stack transparently — no modification or recompilation needed. For setup instructions, see Use SMC.

SMC comes in two variants:

• SMC over RDMA (SMC-R): Uses RDMA technology. Alibaba Cloud Linux 3 provides SMC-R based on elastic Remote Direct Memory Access (eRDMA). All content in this topic applies to SMC-R.

• SMC over Direct Memory Access (SMC-D): Uses internal shared memory (ISM) technology.

Connection establishment: SMC starts with a standard TCP three-way handshake. A special TCP option (Kind: 254, Value: 0xE2D4C3D9) in the SYN packet signals SMC capability to the peer. If both sides support SMC, they create corresponding RDMA resources and complete an SMC handshake. All subsequent data flows over RDMA. If the SMC handshake fails, both sides automatically fall back to the established TCP connection — communication continues uninterrupted.

After the first SMC connection is established, SMC creates an RDMA reliable connection (RC) — called an SMC link — over the RDMA network. Subsequent connections reuse the same SMC link to minimize RDMA resource overhead. When the reuse limit is reached, a new SMC link is created. Multiple SMC links form an SMC link group (LGR) for RDMA transmission and disaster recovery.

Prerequisites

To use SMC-R on Alibaba Cloud Elastic Compute Service (ECS):

• eRDMA-enabled instance: Configure an eRDMA network interface (ERI) on your instance. For supported instance types and configuration steps, see Configure eRDMA on an enterprise-level instance.

• Alibaba Cloud Linux 3 with kernel 5.10.134-16 or later: To check and update your kernel version, see Change the kernel version.

Use cases

SMC delivers the most benefit in these scenarios:

• No application modifications required: SMC's primary advantage is enabling TCP applications to use RDMA without any code changes. Note that as a kernel protocol stack, SMC has additional system call and kernel-mode/user-mode data copy overhead compared to native RDMA applications. SMC is ideal when seamless integration with existing applications is the priority.

• Latency-sensitive and throughput-sensitive workloads: Hardware offload reduces protocol processing latency and CPU overhead; direct memory access increases network throughput.

• Internal network communication: SMC requires both endpoints to support it, which is easy to achieve in internal networks. Short network links amplify the latency benefit.

• Network-intensive services: Services where network is the primary bottleneck see the most significant latency and throughput improvements.

When not to use SMC

Avoid SMC in the following scenarios. Where a workaround exists, it is listed in the Solution column.

Scenario	Why SMC performs poorly	Solution
Short-lived connections	SMC connection setup involves a TCP handshake, an SMC handshake, and RDMA resource creation — all before any data flows. This overhead makes SMC slower than TCP for short-lived connections.	Use smc-ebpf to limit which connections use SMC. See Enable and configure SMC.
High burst connection rates	The same connection-setup overhead causes queuing during connection bursts, leading to connection establishment timeouts.	Use smc-ebpf to limit which connections use SMC. See Enable and configure SMC.
Cross-network-element paths	Some network elements drop or incorrectly relay SYN/SYN-ACK packets that contain unrecognized TCP Experimental Options, causing handshake timeouts or connection failures. This can occur on internet-facing services.	Run aliyunsmc-check syn_check to verify the path. If the path fails, disable SMC on that link using smc-ebpf. See Check the communication link.
Public network communication	Public networks frequently involve network elements that interfere with SMC handshakes (same issue as above). Meeting SMC requirements on both sides is also difficult, causing fallback to TCP. Additionally, SMC has no security hardening and is unsuitable for untrusted environments.	—
Memory-constrained environments	Each SMC connection pre-allocates shared memory buffers. By default, one connection consumes sysctl net.smc.wmem + sysctl net.smc.rmem on each end.	Use smc-ebpf to restrict SMC usage, or reduce the default buffer sizes. See Enable and configure SMC and Enable and configure SMC.
Cross-zone communication	Cross-zone transmission adds hundreds of microseconds to a few milliseconds of latency, which reduces SMC's latency advantage. The higher bandwidth-delay product also requires larger buffers to fully use bandwidth, increasing memory pressure.	Use Unique Endpoint Identifiers (UEIDs) to prevent SMC from being used across zones. See Enable and configure SMC.

Automated check tool

Alibaba Cloud Linux 3 includes aliyunsmc-check, a tool for verifying SMC environment configuration, communication links, and end-to-end connectivity. It is part of the aliyun-smc-extensions package.

Install the package

sudo yum install -y aliyun-smc-extensions

Check the basic environment

Verify that the kernel version and eRDMA device are correctly configured:

aliyunsmc-check basic_check

Sample output:

Check the communication link

Verify that no network elements on the path interfere with SMC. Replace <url> with the target URL:

aliyunsmc-check syn_check --url <url>

Sample output:

If the link fails the check, use smc-ebpf to disable SMC on that link. See Enable and configure SMC.

Check SMC connectivity

Verify that SMC can establish connections between the current host and a remote host. The tool uses SSH to connect to the remote host.

Replace the placeholders:

Placeholder	Description
<peer ip>	IP address of the remote host
<usr>	Username on the remote host
<keyfile>	Path to the SSH private key file
<passwd>	Password for the remote host

SSH key file authentication

aliyunsmc-check connect_check --hosts <peer ip> --user <usr> --key <keyfile>

SSH password authentication

aliyunsmc-check connect_check --hosts <peer ip> --user <usr> --pwd <passwd>

Passwordless SSH authentication

aliyunsmc-check connect_check --hosts <peer ip> --user <usr>

Sample output:

What's next

• Use SMC — enable SMC, configure the smc-ebpf tool, and tune buffer sizes