Preserve client source IP

Method

For HTTP and HTTPS listeners, Global Accelerator (GA) uses the Preserve Client IP feature to retrieve the originating client IP. This feature is enabled by default.

GA stores the originating client IP address in theX-Forwarded-For field of the HTTP request header. You can configure your backend server to retrieve this IP address. The first IP address in the field is the originating client IP address.

X-Forwarded-For: <originating client IP, proxy server 1 IP, proxy server 2 IP, ...>

Procedure

This procedure uses an Nginx 1.20.1 server running on Alibaba Cloud Linux 3 as an example. The steps may vary depending on your environment.

1. Run nginx -V | grep http_realip_module to check whether http_realip_module is installed. Nginx uses this module to parse X-Forwarded-For records.

   If the output includes --with-http_realip_module, the module is installed. Proceed to the next step.

   Sample output if the http_realip_module is installed

   nginx version: nginx/1.20.1
   built by gcc 4.8.5 20150623 (Red Hat 4.8.5-44) (GCC) 
   built with OpenSSL 1.1.1k  FIPS 25 Mar 2021
   TLS SNI support enabled
   configure arguments: --prefix=/usr/share/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib64/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --http-client-body-temp-path=/var/lib/nginx/tmp/client_body --http-proxy-temp-path=/var/lib/nginx/tmp/proxy --http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi --http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi --http-scgi-temp-path=/var/lib/nginx/tmp/scgi --pid-path=/run/nginx.pid --lock-path=/run/lock/subsys/nginx --user=nginx --group=nginx --with-compat --with-debug --with-file-aio --with-google_perftools_module --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_degradation_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_image_filter_module=dynamic --with-http_mp4_module --with-http_perl_module=dynamic --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-http_xslt_module=dynamic --with-mail=dynamic --with-mail_ssl_module --with-pcre --with-pcre-jit --with-stream=dynamic --with-stream_ssl_module --with-stream_ssl_preread_module --with-threads --with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic' --with-ld-opt='-Wl,-z,relro -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,-E'
   

   Note

   • http_realip_module is available since Nginx 1.0.4 (2011). If you use an older version, back up your configuration and upgrade Nginx.

   • If http_realip_module is not installed, recompile Nginx with the module or reinstall Nginx using a package manager such as yum.

2. Edit the Nginx configuration file. Run nginx -t to find the file path. The default is /etc/nginx/nginx.conf.

   http {
     # Make sure that the log_format includes $http_x_forwarded_for. This variable records the value of X-Forwarded-For.
     log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                         '$status $body_bytes_sent "$http_referer" '
                         '"$http_user_agent" "$http_x_forwarded_for"';
     
     # ...
   }
   

3. Run the sudo nginx -s reload command to reload the Nginx configuration.

4. Retrieve the originating client IP address.

   You can check the Nginx access log to verify that the originating client IP address is successfully retrieved.

   Run thetail -n <number_of_lines> <log_path> command. In the returned log entry, check the field corresponding to the$http_x_forwarded_for variable. The first IP address is the originating client IP address.

   In this example, the default Nginx log file path is/var/log/nginx/access.log.

   [root@izxxx ~]# tail -n 1 /var/log/nginx/access.log
   172.20xxx - - [17/Oct/2024:17:46:49 +0800] "GET / HTTP/1.1" 304 0 "-" "Mozilla/5.0" "47.243.xxx"

   Note

   To prevent health check logs from overwriting entries that contain originating client IP addresses, view multiple log entries at a time or temporarily increase the health check interval.

Methods

To retrieve the originating client IP address, enable the Preserve Client IP feature and select a retrieval method.

Supported methods vary by backend service type.

• Automatic: GA forwards the originating client IP address directly to the backend service. The client IP address retrieved by the backend server is the originating client IP address.

• Proxy Protocol: The backend server must support Proxy Protocol to obtain the originating client IP address.

  What is Proxy Protocol?

  Proxy Protocol is a protocol for forwarding a client's original network connection information from a proxy server to a backend server.

  Typically, when a proxy server forwards a client request, it rewrites the request header and replaces the client's original IP address and port with its own. This prevents the backend server from obtaining the client's originating network information.

  With Proxy Protocol, the proxy server encapsulates the client's original connection information within a Proxy Protocol header and prepends it to the data forwarded to the backend server. The backend server can then parse this header to obtain the original information, such as the source IP address, source port, and protocol.

  This allows the backend server to perform more accurate logging, access control, and traffic monitoring.

  Important

  • Proxy Protocol requires support from both the proxy server and the backend server to function correctly. If your backend server cannot parse Proxy Protocol, enabling this feature will likely cause parsing errors and affect service availability.

  • GA listeners use Proxy Protocol to transmit original connection information (such as source IP, destination IP, source port, and destination port) by prepending it to the TCP data stream. This process does not discard or overwrite any existing data.

  • GA supports only Proxy Protocol v1, which supports only the TCP protocol. For more information, see The PROXY protocol.

Procedure

This procedure uses an Nginx 1.20.1 server running on Alibaba Cloud Linux 3 as an example. The steps may vary depending on your environment.

Method

To retrieve the originating client IP address, enable the Preserve Client IP feature. For UDP listeners, this feature supports only the Automatically Retrieve Client IP method, where GA forwards the originating client IP address directly to the backend service.

When you enable Preserve Client IP for a UDP listener, the following limitations apply:

• Only pay-as-you-go GA instances are supported.

  If your pay-as-you-go GA instance does not support enabling client IP preservation for UDP listeners, the instance version may not support this feature. To use this feature, contact your account manager to request an instance upgrade.

• This is supported only when the Backend Service Type is ECS , ENI , or NLB . For NLB-type backend services, the Preserve Client IP feature is not enabled by default. To use this feature, contact your account manager.

• Backend servers can only retrieve originating IP addresses from clients of the same IP version (IPv4 to IPv4 and IPv6 to IPv6).

Procedure

This procedure uses an Nginx 1.20.1 server running on Alibaba Cloud Linux 3 as an example. The steps may vary depending on your environment.

1. Enable client IP preservation.

   1. Log on to the Global Accelerator (GA) console.

   2. On the Instances page, find the target GA instance, and click Configure Listener in the Actions column.

   3. On the Listener tab, find the target UDP listener, and click Edit Listener in the Actions column.

   4. On the Configure Listener & Protocol page, click Next.

   5. On the Configure Endpoint Group page, select Maintain from the Preserve Client IP drop-down list, and then click Next.

      The Retrieve Client IP Address is set to Automatic by default.

   6. On the Configuration Review page, confirm the configuration and click Submit.

2. Check the Nginx configuration file.

   Run thenginx -t command to find the path of the configuration file. The default path is typically/etc/nginx/nginx.conf.

   Note

   To process UDP traffic, the Nginx server must support thestream{} module.

   Thestream{} module was added in Nginx 1.9.0 and is not installed by default. You must compile and install it to use the stream{} module.

   #...
   # Process UDP traffic (make sure to include this module)
   stream {
       log_format main_stream '$remote_addr - [$time_local] '
                           '"$protocol" $status $bytes_sent $bytes_received '
                           '"$session_time"';
       access_log  /var/log/nginx/stream_access.log  main_stream;
   # Example: UDP listener on port 4000
       server {
           listen 4000 udp;
           #...
       }
   }

3. Retrieve the originating client IP address.

   • Method 1: Check the Nginx log to verify the originating client IP address.

     Run thetail -n <number_of_lines> <log_path> command. In the returned log entry, the first IP address, which corresponds to the$remote_addr variable, is the originating client IP address.

     In this example, the log file path is/var/log/nginx/stream_access.log.

     [root@iZ2zxxx          ~]# tail -n 5 /var/log/nginx/stream_access.log
     47.251.xxx.xxx - [24/Jan/2025:13:37:43 +0800] "UDP" 200 0 12 "600.099"
     47.251.xxx.xxx - [24/Jan/2025:13:39:11 +0800] "UDP" 200 0 12 "600.088"
     47.251.xxx.xxx - [24/Jan/2025:13:39:28 +0800] "UDP" 200 0 14 "600.012"
     47.251.xxx.xxx - [24/Jan/2025:13:39:37 +0800] "UDP" 200 0 14 "600.009"
     47.251.xxx.xxx - [24/Jan/2025:13:41:04 +0800] "UDP" 200 0 14 "600.086"

   • Method 2: Capture data packets to verify the originating client IP address.

     Run thetcpdump udp port [listener_port] -n -X -s 0 command to check the data packets for the originating client IP address.

     [root@iz2xxx ~]# tcpdump udp port 4000 -n -X -s 0
     dropped privs to tcpdump
     tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
     listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
     14:04:06.047337 IP 47.251.xxx.xxx 55491 > 10.5.120.213.terabase: UDP, length 14
     0x0000: 4500 002a 6e1e 4000 4011 a221 2ffb 77ae  E..*n.@.@..!/w.
     0x0010: 0a05 78d5 d8c3 0fa0 0016 12fb 4865 6c6c  ..x.........Hell
     0x0020: 6f2c 2055 4450 3032 210a                  o, UDP02!.

Troubleshooting client IP preservation

• For an HTTP or HTTPS listener, this feature fails if the backend server cannot process the X-Forwarded-For header.

• For a TCP listener:

  • If the backend service is deployed on Alibaba Cloud, this feature fails if the backend server or the EIP-associated instance is outside a VPC.

  • If the backend service is deployed outside Alibaba Cloud, the backend server must support parsing Proxy Protocol to obtain the client IP address. Otherwise, this feature fails.

    If your backend server cannot parse Proxy Protocol, it cannot correctly process accelerated traffic.

  • If an IPv6 client accesses an IPv4 backend service, the backend server must support parsing Proxy Protocol to obtain the client IP address. Otherwise, this feature fails.

• For a UDP listener, this feature is not supported when an IPv6 client accesses an IPv4 backend service.

Disabling client IP address preservation

For HTTP or HTTPS listeners, client IP address preservation is enabled by default and cannot be disabled.

Retrieving client IP in IPv6-to-IPv4 translation

• For an HTTP or HTTPS listener: The client IP addresses of both IPv4 and IPv6 clients are available only in the X-Forwarded-For header of HTTP requests.

• For a TCP listener: The client IP address of an IPv6 client can be obtained only by using Proxy Protocol. This method requires the Proxy Protocol to support parsing Proxy Protocol. Otherwise, the access request fails.

• For a UDP listener: Retrieving the client IP address of an IPv6 client is not supported.