Use vtoa to get the real client address after address translation-Alibaba Cloud Linux(Alinux)-阿里云帮助中心

In FullNAT scenarios such as Anti-DDoS Proxy and CDN acceleration, the forwarding node replaces the client's source IP and port with its own before passing traffic to the backend server. vtoa is a kernel module for Alibaba Cloud Linux 3 that recovers the original client IP and port from the TCP Option Address (TOA) embedded in the TCP header. Your application retrieves it with no changes to network infrastructure, using either getsockopt (recommended) or getpeername.

Both IPv4 and IPv6 are supported.

How it works

A client request arrives at the forwarding node (for example, an Anti-DDoS node).
The forwarding node performs FullNAT, replacing the client's source IP and port with its own, and embeds the original address into a TCP Option field (TOA).
The backend server receives the connection. vtoa extracts the TOA data from the TCP header in the kernel.
Your application calls getsockopt (recommended) or getpeername to retrieve the original client address.

Use cases

Anti-DDoS Proxy: Client requests undergo FullNAT at the Anti-DDoS forwarding node. Anti-DDoS embeds the real client IP and port into the TCP Option. The backend origin server uses vtoa to recover the real client IP.
CDN acceleration: CDN acceleration nodes forward requests to the origin server. The origin server uses vtoa to recover the real client address.

Supported systems

vtoa requires Alibaba Cloud Linux 3 with kernel version 5.10.134-15 or later.

Run uname -r to check the kernel version.

Warning

vtoa modifies the value returned by the getpeername system call. Confirm the following is acceptable before installing:

Network components that modify getpeername via eBPF—such as Cilium—may conflict with vtoa and cause abnormal behavior.
Applications that depend on getpeername may be affected.

To avoid these issues, use the getsockopt method, which retrieves the original client address without modifying what getpeername returns.

Install vtoa

Install vtoa with yum:

sudo yum install vtoa -y

After installation, vtoa starts immediately and is configured to start automatically on boot. No further configuration is typically needed.

To uninstall:

sudo yum remove vtoa -y

After uninstallation, vtoa is disabled.

Manage the vtoa service

Although vtoa runs automatically after installation, you can control it with systemctl:

Action	Command
Start	`sudo systemctl start vtoa`
Stop	`sudo systemctl stop vtoa`
Enable auto-start on boot	`sudo systemctl enable vtoa`
Disable auto-start on boot	`sudo systemctl disable vtoa`
Check status	`systemctl status vtoa`

Get the real client address

After vtoa is running, use one of the following system calls to retrieve the original client address. The caddr.sa_family field is AF_INET for IPv4 and AF_INET6 for IPv6.

Use getsockopt (recommended)

Important

This method requires kernel version 5.10.134-17 or later. Run uname -r to verify.

vtoa registers a socket option with optname value 1348. Call getsockopt with IPPROTO_IP and this optname to retrieve the original client address directly. This method does not modify the value returned by getpeername, making it safe in environments with eBPF-based networking tools.

struct sockaddr caddr;
int optlen = sizeof(caddr);
int optname = 1348;

getsockopt(fd, IPPROTO_IP, optname, &caddr, &optlen);
// caddr.sa_family can be AF_INET or AF_INET6, which correspond to IPv4 and IPv6 addresses.

Use getpeername

Important

This method is supported on kernel version 5.10.134-15 or later. It may be deprecated in the future. Use getsockopt when possible.

When vtoa is enabled, getpeername returns the original client address instead of the FullNAT address. No application code change is needed beyond calling getpeername as usual.

struct sockaddr caddr;
int caddr_len = sizeof(caddr);

getpeername(fd, &caddr, &caddr_len);
// caddr.sa_family can be AF_INET or AF_INET6, which correspond to IPv4 and IPv6 addresses.
// accept() can also be used to get the client address in a similar way.

Why getsockopt is preferred: getpeername globally overrides the peer address for all callers on the socket. This can break components that expect the NAT address, such as certain eBPF-based tools, and may be deprecated in a future kernel release. getsockopt retrieves the original client address without changing what getpeername returns, making it safer in mixed environments.

FAQ

What TCP option formats does vtoa support?

vtoa parses address information carried in TCP Options. If the format does not match, vtoa silently ignores the option. This has no effect on the application and is equivalent to vtoa being disabled.

Two formats are supported:

TOA (IPv4) — opcode 254

opsize = 8. Both ip and port are in network byte order.

0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |     opcode    |    opsize     |              port             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                              ip                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

TOA_V6 (IPv6) — opcode 253

opsize = 20. Both ip and port are in network byte order.

0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |     opcode    |    opsize     |              port             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   +                                                               +
   |                                                               |
   +                              ip                               +
   |                                                               |
   +                                                               +
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

How do I use vtoa in container environments?

vtoa does not support container-level isolation. Install vtoa on the host, not inside a container. Once installed on the host, it is effective for all containers on that node (for example, when using ACK with the containerd runtime).

vtoa is installed but I still get the FullNAT address — how do I troubleshoot?

Check the following in order:

Kernel version: Run uname -r and confirm the version is 5.10.134-15 or later (or 5.10.134-17 or later if using getsockopt).
Service status: Run systemctl status vtoa and confirm the service is active (running).
TOA format: Confirm your upstream forwarding node (Anti-DDoS, CDN, or custom FullNAT) is inserting a TOA with opcode 254 (IPv4) or 253 (IPv6). If the opcode or format does not match, vtoa silently ignores the option.
eBPF conflicts: If Cilium or another eBPF-based component is running on the host, check whether it also hooks getpeername. Conflicting hooks can produce unexpected results. If so, switch to the getsockopt method, which does not modify the getpeername return value.