Android端HTTPDNS+HttpURLConnection最佳实践

通过Android SDK接入这篇文档,您已经了解了Android SDK导入、配置、解析IP、应用到网络库和接入验证的完整流程,本文主要介绍基于 HttpURLConnection 接入HTTPDNS的具体方案。

1. 前言

本文档主要介绍在 Android 端使用 HttpURLConnection 进行网络请求时,如何在 HTTPS(含 SNI) 场景下集成 HTTPDNS 并实现 “IP 直连” 的方案,如果您对原理感兴趣,可以参考IP直连接入HTTPDNS的原理

重要

当前 Android 端主流网络开发框架大多已转向 OkHttp。OkHttp 原生提供自定义 DNS 服务的接口,可更简洁、优雅地实现 IP 直连。推荐优先参考AndroidHTTPDNS+OkHttp最佳实践进行接入。以下内容仅在您业务场景无法使用 OkHttp 时,提供基于 HttpURLConnection 的替代方案。

2. 接入方案

根据是否涉及 SNI,我们可以分成两种场景讨论:

  1. HTTPS 场景(SNI) 则需要在创建 SSLSocket 时,将原始域名通过 SNI 方式传递给服务端,同时也要做好 HostnameVerifier 相关逻辑。

  2. HTTPS 场景(非 SNI) 可以通过 HostnameVerifier 接口,在证书校验时将 IP 还原为原始域名进行验证。

下面分别介绍这两种场景下的完整接入流程和参考示例。

2.1 HTTPS 场景(SNI)

对于部署了多域名证书、需要在握手前通过 SNI 提供域名信息给服务器的场景,除了 HostnameVerifier,还需在 创建 SSLSocket 之时 设置正确的 SNI 名称。为此,需要定制一个 自定义的 SSLSocketFactory,在 createSocket() 方法中进行以下操作:

  1. 替换为 HTTPDNS 解析后的 IP 进行连接。

  2. 调用系统或自定义的 SSLCertificateSocketFactory,在握手前通过 setHostname() 设置 SNI Hostname。

  3. 自行执行证书验证:将证书校验中的域名从 IP 改回原始域名。

在官方HTTPDNS Android Demo中,我们针对HttpsURLConnection,提供了在SNI场景下使用HTTPDNS的示例代码。

自定义 SSLSocketFactory 示例

class TlsSniSocketFactory constructor(conn: HttpsURLConnection): SSLSocketFactory() {
    private val mConn: HttpsURLConnection

    init {
        mConn = conn
    }

    override fun createSocket(plainSocket: Socket?, host: String?, port: Int, autoClose: Boolean): Socket {
        var peerHost = mConn.getRequestProperty("Host")
        if (peerHost == null) {
            peerHost = host
        }

        val address = plainSocket!!.inetAddress
        if (autoClose) {
            // we don't need the plainSocket
            plainSocket.close()
        }

        // create and connect SSL socket, but don't do hostname/certificate verification yet
        val sslSocketFactory =
            SSLCertificateSocketFactory.getDefault(0) as SSLCertificateSocketFactory
        val ssl = sslSocketFactory.createSocket(address, R.attr.port) as SSLSocket

        // enable TLSv1.1/1.2 if available
        ssl.enabledProtocols = ssl.supportedProtocols

        // set up SNI before the handshake
        if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.JELLY_BEAN_MR1) {
            // setting sni hostname
            sslSocketFactory.setHostname(ssl, peerHost)
        } else {
            // No documented SNI support on Android <4.2, trying with reflection
            try {
                val setHostnameMethod = ssl.javaClass.getMethod(
                    "setHostname",
                    String::class.java
                )
                setHostnameMethod.invoke(ssl, peerHost)
            } catch (e: Exception) {
            }
        }

        // verify hostname and certificate
        val session = ssl.session

        if (!HttpsURLConnection.getDefaultHostnameVerifier()
                .verify(peerHost, session)
        ) throw SSLPeerUnverifiedException(
            "Cannot verify hostname: $peerHost"
        )

        return ssl
    }
}
public class TlsSniSocketFactory extends SSLSocketFactory {
    
    private HttpsURLConnection mConn;
    public TlsSniSocketFactory(HttpsURLConnection conn) {
        mConn = conn;
    }

    @Override
    public Socket createSocket(Socket plainSocket, String host, int port, boolean autoClose) throws IOException {
        String peerHost = mConn.getRequestProperty("Host");
        if (peerHost == null)
            peerHost = host;

        InetAddress address = plainSocket.getInetAddress();
        if (autoClose) {
            // we don't need the plainSocket
            plainSocket.close();
        }
        // create and connect SSL socket, but don't do hostname/certificate verification yet
        SSLCertificateSocketFactory sslSocketFactory = (SSLCertificateSocketFactory) SSLCertificateSocketFactory.getDefault(0);
        SSLSocket ssl = (SSLSocket) sslSocketFactory.createSocket(address, port);

        // enable TLSv1.1/1.2 if available
        ssl.setEnabledProtocols(ssl.getSupportedProtocols());

        // set up SNI before the handshake
        if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.JELLY_BEAN_MR1) {
            // setting sni hostname
            sslSocketFactory.setHostname(ssl, peerHost);
        } else {
            // No documented SNI support on Android <4.2, trying with reflection
            try {
                java.lang.reflect.Method setHostnameMethod = ssl.getClass().getMethod("setHostname", String.class);
                setHostnameMethod.invoke(ssl, peerHost);
            } catch (Exception e) {

            }
        }

        // verify hostname and certificate
        SSLSession session = ssl.getSession();

        if (!HttpsURLConnection.getDefaultHostnameVerifier().verify(peerHost, session))
            throw new SSLPeerUnverifiedException("Cannot verify hostname: " + peerHost);

        return ssl;
    }
}

重定向处理示例

许多 SNI 场景下的请求可能会经历多次 HTTP 3xx 重定向。示例中也展示了如何在重定向时再次基于 HTTPDNS 解析新的 Host 并继续发起请求。

fun recursiveRequest(path: String) {
    var conn: HttpURLConnection? = null
    try {
        val url = URL(path)
        conn = url.openConnection() as HttpURLConnection
        // 同步接口获取IP
        val httpdnsResult = HttpDns.getService(accountID)
            .getHttpDnsResultForHostSync(url.host, RequestIpType.both)
        var ip: String? = null
        if (httpdnsResult.ips != null && httpdnsResult.ips.isNotEmpty()) {
            ip = httpdnsResult.ips[0]
        } else if (httpdnsResult.ipv6s != null && httpdnsResult.ipv6s.isNotEmpty()) {
            ip = httpdnsResult.ipv6s[0]
        }
        if (!TextUtils.isEmpty(ip)) {
            // 通过HTTPDNS获取IP成功,进行URL替换和HOST头设置
            val newUrl = path.replaceFirst(url.host.toRegex(), ip!!)
            conn = URL(newUrl).openConnection() as HttpURLConnection
            conn.connectTimeout = 30000
            conn.readTimeout = 30000
            conn.instanceFollowRedirects = false

            // 设置HTTP请求头Host域
            conn.setRequestProperty("Host", url.host)
            if (conn is HttpsURLConnection) {
                val httpsURLConnection = conn

                // https场景,证书校验
                httpsURLConnection.hostnameVerifier =
                    HostnameVerifier { _, session ->
                        var host = httpsURLConnection.getRequestProperty("Host")
                        if (null == host) {
                            host = httpsURLConnection.url.host
                        }
                        HttpsURLConnection.getDefaultHostnameVerifier().verify(host, session)
                    }

                // sni场景,创建SSLScocket
                httpsURLConnection.sslSocketFactory = TlsSniSocketFactory(httpsURLConnection)
            }
        }
        val code = conn.responseCode // Network block
        if (code in 300..399) {
            var location = conn.getHeaderField("Location")
            if (location == null) {
                location = conn.getHeaderField("location")
            }
            if (location != null) {
                if (!(location.startsWith("http://") || location
                        .startsWith("https://"))
                ) {
                    //某些时候会省略host,只返回后面的path,所以需要补全url
                    val originalUrl = URL(path)
                    location = (originalUrl.protocol + "://"
                            + originalUrl.host + location)
                }
                recursiveRequest(location)
            }
        } else {
            // redirect finish.
            val dis = DataInputStream(conn.inputStream)
            var len: Int
            val buff = ByteArray(4096)
            val response = StringBuilder()
            while (dis.read(buff).also { len = it } != -1) {
                response.append(String(buff, 0, len))
            }
            Log.d(TAG, "Response: $response")
        }
    } catch (e: MalformedURLException) {
        Log.w(TAG, "recursiveRequest MalformedURLException")
    } catch (e: IOException) {
        Log.w(TAG, "recursiveRequest IOException")
    } catch (e: java.lang.Exception) {
        Log.w(TAG, "unknow exception")
    } finally {
        conn?.disconnect()
    }
}
public void recursiveRequest(String path) {
    HttpURLConnection conn = null;
    try {
        URL url = new URL(path);
        conn = (HttpURLConnection) url.openConnection();
        // 同步接口获取IP
        HTTPDNSResult httpdnsResult = HttpDns.getService(accountID).getHttpDnsResultForHostSync(url.getHost(), RequestIpType.both);

        String ip = null;
        if (httpdnsResult.getIps() != null && httpdnsResult.getIps().length > 0) {
            ip = httpdnsResult.getIps()[0];
        } else if (httpdnsResult.getIpv6s() != null && httpdnsResult.getIpv6s().length > 0) {
            ip = httpdnsResult.getIpv6s()[0];
        }

        if (!TextUtils.isEmpty(ip)) {
            // 通过HTTPDNS获取IP成功,进行URL替换和HOST头设置
            String newUrl = path.replaceFirst(url.getHost(), ip);
            conn = (HttpURLConnection) new URL(newUrl).openConnection();

            conn.setConnectTimeout(30000);
            conn.setReadTimeout(30000);
            conn.setInstanceFollowRedirects(false);

            // 设置HTTP请求头Host域
            conn.setRequestProperty("Host", url.getHost());

            if (conn instanceof HttpsURLConnection) {
                final HttpsURLConnection httpsURLConnection = (HttpsURLConnection) conn;

                // https场景,证书校验
                httpsURLConnection.setHostnameVerifier(new HostnameVerifier() {
                    @Override
                    public boolean verify(String hostname, SSLSession session) {
                        String host = httpsURLConnection.getRequestProperty("Host");
                        if (null == host) {
                            host = httpsURLConnection.getURL().getHost();
                        }
                        return HttpsURLConnection.getDefaultHostnameVerifier().verify(host, session);
                    }
                });

                // sni场景,创建SSLScocket
                httpsURLConnection.setSSLSocketFactory(new TlsSniSocketFactory(httpsURLConnection));
            }
        }

        int code = conn.getResponseCode();// Network block
        if (code >= 300 && code < 400) {
            String location = conn.getHeaderField("Location");
            if (location == null) {
                location = conn.getHeaderField("location");
            }

            if (location != null) {
                if (!(location.startsWith("http://") || location
                        .startsWith("https://"))) {
                    //某些时候会省略host,只返回后面的path,所以需要补全url
                    URL originalUrl = new URL(path);
                    location = originalUrl.getProtocol() + "://"
                            + originalUrl.getHost() + location;
                }

                recursiveRequest(location);
            }
        } else {
            // redirect finish.
            DataInputStream dis = new DataInputStream(conn.getInputStream());
            int len;
            byte[] buff = new byte[4096];
            StringBuilder response = new StringBuilder();
            while ((len = dis.read(buff)) != -1) {
                response.append(new String(buff, 0, len));
            }
            Log.d(TAG, "Response: " + response.toString());
        }
    } catch (MalformedURLException e) {
        Log.w(TAG, "recursiveRequest MalformedURLException");
    } catch (IOException e) {
        Log.w(TAG, "recursiveRequest IOException");
    } catch (Exception e) {
        Log.w(TAG, "unknow exception");
    } finally {
        if (conn != null) {
            conn.disconnect();
        }
    }
}

2.2 HTTPS 场景(非 SNI)

对于仅使用单域名证书(即不涉及多域名部署或不需通过 SNI 指明域名)的场景,最主要的改动在于 hostname 验证。具体原理是:在证书校验流程的第二步,通过 hook 或自行实现 HostnameVerifier,将用来验证的域名从 IP 改回原始域名即可。

重要 该方案仅适用于非 SNI 场景。如果业务部署了多证书、多域名,请确认是否确实 不需要 SNI。如果遇到类似 SSLHandshakeException: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.请优先排查目标站点是否需要 SNI 支持。

以下示例基于 HttpURLConnection,演示如何利用 HostnameVerifier 接口完成证书校验。

try {
    val url = "https://140.205.XX.XX/?sprefer=sypc00"
    val connection = URL(url).openConnection() as HttpURLConnection
    connection.setRequestProperty("Host", "m.taobao.com")
    if (connection is HttpsURLConnection) {
        connection.hostnameVerifier = HostnameVerifier { _, session ->
            /*
            * 关于这个接口的说明,官方有文档描述:
            * This is an extended verification option that implementers can provide.
            * It is to be used during a handshake if the URL's hostname does not match the
            * peer's identification hostname.
            *
            * 使用HTTPDNS后URL里设置的hostname不是远程的主机名(如:m.taobao.com),与证书颁发的域不匹配,
            * Android HttpsURLConnection提供了回调接口让用户来处理这种定制化场景。
            * 在确认HTTPDNS返回的源站IP与Session携带的IP信息一致后,您可以在回调方法中将待验证域名替换为原来的真实域名进行验证。
            *
            */
            HttpsURLConnection.getDefaultHostnameVerifier().verify("m.taobao.com", session)
        }
    }
    connection.connect()
} catch (e: java.lang.Exception) {
    e.printStackTrace()
}
try {
    String url = "https://140.205.XX.XX/?sprefer=sypc00";
    HttpURLConnection connection = (HttpURLConnection) new URL(url).openConnection();

    connection.setRequestProperty("Host", "m.taobao.com");

    if (connection instanceof HttpsURLConnection) {
        connection.setHostnameVerifier(new HostnameVerifier() {
    
           /*
            * 关于这个接口的说明,官方有文档描述:
            * This is an extended verification option that implementers can provide.
            * It is to be used during a handshake if the URL's hostname does not match the
            * peer's identification hostname.
            *
            * 使用HTTPDNS后URL里设置的hostname不是远程的主机名(如:m.taobao.com),与证书颁发的域不匹配,
            * Android HttpsURLConnection提供了回调接口让用户来处理这种定制化场景。
            * 在确认HTTPDNS返回的源站IP与Session携带的IP信息一致后,您可以在回调方法中将待验证域名替换为原来的真实域名进行验证。
            *
            */
            @Override
            public boolean verify(String hostname, SSLSession session) {
                return HttpsURLConnection.getDefaultHostnameVerifier().verify("m.taobao.com", session);
            }
        });
    }

    connection.connect();
} catch (Exception e) {
    e.printStackTrace();
}

3. 总结

  1. HTTPDNS 在 HTTPS 下的挑战

    • 证书校验需要域名匹配;

    • SNI 需要在握手之前提供域名给服务器。

  2. SNI 场景

    • 必须自定义 SSLSocketFactory,在握手前设置 SNI Hostname,并在 HostnameVerifier 中处理证书校验域名的替换。

  3. 非 SNI 场景

    • 只需通过 HostnameVerifier 把验证用的域名从 IP 改回原始域名。

  4. 优先使用 OkHttp

至此,您已经了解如何在 Android 端通过 HttpURLConnection 实现基于 HTTPDNS 的 HTTPS(含 SNI) “IP 直连” 方案,希望本文能帮助您顺利完成集成。