Android端HTTPS(含SNI)业务场景:IP直连方案

背景说明

本文主要介绍HTTPS(含SNI)业务场景下在Android端实现“IP直连”的通用解决方案。

如果您是Android开发者,并且以OkHttp作为网络开发框架,由于OkHttp提供了自定义DNS服务接口,可以优雅地实现IP直连。其方案相比通用方案更加简单且通用性更强,推荐您参考Android端HTTPDNS+OkHttp接入指南接入HTTPDNS。

HTTPS

发送HTTPS请求首先要进行SSL/TLS握手,握手过程大致如下:

  1. 客户端发起握手请求,携带随机数、支持算法列表等参数。

  2. 服务端收到请求,选择合适的算法,下发公钥证书和随机数。

  3. 客户端对服务端证书进行校验,并发送随机数信息,该信息使用公钥加密。

  4. 服务端通过私钥获取随机数信息。

  5. 双方根据以上交互的信息生成session ticket,用作该连接后续数据传输的加密密钥。

上述过程中,和HTTPDNS有关的是第3步,客户端需要验证服务端下发的证书,验证过程有以下两个要点:

  1. 客户端用本地保存的根证书解开证书链,确认服务端下发的证书是由可信任的机构颁发的。

  2. 客户端需要检查证书的domain域和扩展域,看是否包含本次请求的host。

如果上述两点都校验通过,就证明当前的服务端是可信任的,否则就是不可信任,应当中断当前连接。

当客户端使用HTTPDNS解析域名时,请求URL中的host会被替换成HTTPDNS解析出来的IP,所以在证书验证的第2步,会出现domain不匹配的情况,导致SSL/TLS握手不成功。

SNI

SNI(Server Name Indication)是为了解决一个服务器使用多个域名和证书的SSL/TLS扩展。它的工作原理如下:

  1. 在连接到服务器建立SSL链接之前先发送要访问站点的域名(Hostname)。

  2. 服务器根据这个域名返回一个合适的证书。

目前,大多数操作系统和浏览器都已经很好地支持SNI扩展,OpenSSL 0.9.8也已经内置这一功能。

上述过程中,当客户端使用HTTPDNS解析域名时,请求URL中的host会被替换成HTTPDNS解析出来的IP,导致服务器获取到的域名为解析后的IP,无法找到匹配的证书,只能返回默认的证书或者不返回,所以会出现SSL/TLS握手不成功的错误。

说明

比如当你需要通过HTTPS访问CDN资源时,CDN的站点往往服务了很多的域名,所以需要通过SNI指定具体的域名证书进行通信。

HTTPS场景(非SNI)解决方案

针对“domain不匹配”问题,可以采用如下方案解决:hook证书校验过程中第2步,将IP直接替换成原来的域名,再执行证书验证。

重要

基于该方案发起网络请求,若报出SSL校验错误,比如Android系统报错System.err: javax.net.ssl.SSLHandshakeException: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.,请检查应用场景是否为SNI(单IP多HTTPS域名)。

此示例针对HttpURLConnection接口。

try {
    val url = "https://140.205.XX.XX/?sprefer=sypc00"
    val connection = URL(url).openConnection() as HttpURLConnection
    connection.setRequestProperty("Host", "m.taobao.com")
    if (connection is HttpsURLConnection) {
        connection.hostnameVerifier = HostnameVerifier { _, session ->
            /*
            * 关于这个接口的说明,官方有文档描述:
            * This is an extended verification option that implementers can provide.
            * It is to be used during a handshake if the URL's hostname does not match the
            * peer's identification hostname.
            *
            * 使用HTTPDNS后URL里设置的hostname不是远程的主机名(如:m.taobao.com),与证书颁发的域不匹配,
            * Android HttpsURLConnection提供了回调接口让用户来处理这种定制化场景。
            * 在确认HTTPDNS返回的源站IP与Session携带的IP信息一致后,您可以在回调方法中将待验证域名替换为原来的真实域名进行验证。
            *
            */
            HttpsURLConnection.getDefaultHostnameVerifier().verify("m.taobao.com", session)
        }
    }
    connection.connect()
} catch (e: java.lang.Exception) {
    e.printStackTrace()
}
try {
    String url = "https://140.205.XX.XX/?sprefer=sypc00";
    HttpURLConnection connection = (HttpURLConnection) new URL(url).openConnection();

    connection.setRequestProperty("Host", "m.taobao.com");

    if (connection instanceof HttpsURLConnection) {
        connection.setHostnameVerifier(new HostnameVerifier() {
    
           /*
            * 关于这个接口的说明,官方有文档描述:
            * This is an extended verification option that implementers can provide.
            * It is to be used during a handshake if the URL's hostname does not match the
            * peer's identification hostname.
            *
            * 使用HTTPDNS后URL里设置的hostname不是远程的主机名(如:m.taobao.com),与证书颁发的域不匹配,
            * Android HttpsURLConnection提供了回调接口让用户来处理这种定制化场景。
            * 在确认HTTPDNS返回的源站IP与Session携带的IP信息一致后,您可以在回调方法中将待验证域名替换为原来的真实域名进行验证。
            *
            */
            @Override
            public boolean verify(String hostname, SSLSession session) {
                return HttpsURLConnection.getDefaultHostnameVerifier().verify("m.taobao.com", session);
            }
        });
    }

    connection.connect();
} catch (Exception e) {
    e.printStackTrace();
}

HTTPS场景(SNI)解决方案

HTTPDNS Android Demo中针对HttpsURLConnection接口,提供了在SNI业务场景下使用HTTPDNS的示例代码。

定制SSLSocketFactory,在createSocket时替换为HTTPDNS的IP,并进行SNI/HostNameVerify配置。

class TlsSniSocketFactory constructor(conn: HttpsURLConnection): SSLSocketFactory() {
    private val mConn: HttpsURLConnection

    init {
        mConn = conn
    }

    override fun createSocket(plainSocket: Socket?, host: String?, port: Int, autoClose: Boolean): Socket {
        var peerHost = mConn.getRequestProperty("Host")
        if (peerHost == null) {
            peerHost = host
        }

        val address = plainSocket!!.inetAddress
        if (autoClose) {
            // we don't need the plainSocket
            plainSocket.close()
        }

        // create and connect SSL socket, but don't do hostname/certificate verification yet
        val sslSocketFactory =
            SSLCertificateSocketFactory.getDefault(0) as SSLCertificateSocketFactory
        val ssl = sslSocketFactory.createSocket(address, R.attr.port) as SSLSocket

        // enable TLSv1.1/1.2 if available
        ssl.enabledProtocols = ssl.supportedProtocols

        // set up SNI before the handshake
        if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.JELLY_BEAN_MR1) {
            // setting sni hostname
            sslSocketFactory.setHostname(ssl, peerHost)
        } else {
            // No documented SNI support on Android <4.2, trying with reflection
            try {
                val setHostnameMethod = ssl.javaClass.getMethod(
                    "setHostname",
                    String::class.java
                )
                setHostnameMethod.invoke(ssl, peerHost)
            } catch (e: Exception) {
            }
        }

        // verify hostname and certificate
        val session = ssl.session

        if (!HttpsURLConnection.getDefaultHostnameVerifier()
                .verify(peerHost, session)
        ) throw SSLPeerUnverifiedException(
            "Cannot verify hostname: $peerHost"
        )

        return ssl
    }
}
public class TlsSniSocketFactory extends SSLSocketFactory {
    
    private HttpsURLConnection mConn;
    public TlsSniSocketFactory(HttpsURLConnection conn) {
        mConn = conn;
    }

    @Override
    public Socket createSocket(Socket plainSocket, String host, int port, boolean autoClose) throws IOException {
        String peerHost = mConn.getRequestProperty("Host");
        if (peerHost == null)
            peerHost = host;

        InetAddress address = plainSocket.getInetAddress();
        if (autoClose) {
            // we don't need the plainSocket
            plainSocket.close();
        }
        // create and connect SSL socket, but don't do hostname/certificate verification yet
        SSLCertificateSocketFactory sslSocketFactory = (SSLCertificateSocketFactory) SSLCertificateSocketFactory.getDefault(0);
        SSLSocket ssl = (SSLSocket) sslSocketFactory.createSocket(address, port);

        // enable TLSv1.1/1.2 if available
        ssl.setEnabledProtocols(ssl.getSupportedProtocols());

        // set up SNI before the handshake
        if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.JELLY_BEAN_MR1) {
            // setting sni hostname
            sslSocketFactory.setHostname(ssl, peerHost);
        } else {
            // No documented SNI support on Android <4.2, trying with reflection
            try {
                java.lang.reflect.Method setHostnameMethod = ssl.getClass().getMethod("setHostname", String.class);
                setHostnameMethod.invoke(ssl, peerHost);
            } catch (Exception e) {

            }
        }

        // verify hostname and certificate
        SSLSession session = ssl.getSession();

        if (!HttpsURLConnection.getDefaultHostnameVerifier().verify(peerHost, session))
            throw new SSLPeerUnverifiedException("Cannot verify hostname: " + peerHost);

        return ssl;
    }
}

对于需要设置SNI的站点,通常需要重定向请求,示例中也给出了重定向请求的处理方法。

fun recursiveRequest(path: String) {
    var conn: HttpURLConnection? = null
    try {
        val url = URL(path)
        conn = url.openConnection() as HttpURLConnection
        // 同步接口获取IP
        val httpdnsResult = HttpDns.getService(context, accountID)
            .getHttpDnsResultForHostSync(url.host, RequestIpType.both)
        var ip: String? = null
        if (httpdnsResult.ips != null && httpdnsResult.ips.isNotEmpty()) {
            ip = httpdnsResult.ips[0]
        } else if (httpdnsResult.ipv6s != null && httpdnsResult.ipv6s.isNotEmpty()) {
            ip = httpdnsResult.ipv6s[0]
        }
        if (!TextUtils.isEmpty(ip)) {
            // 通过HTTPDNS获取IP成功,进行URL替换和HOST头设置
            val newUrl = path.replaceFirst(url.host.toRegex(), ip!!)
            conn = URL(newUrl).openConnection() as HttpURLConnection
            conn.connectTimeout = 30000
            conn.readTimeout = 30000
            conn.instanceFollowRedirects = false

            // 设置HTTP请求头Host域
            conn.setRequestProperty("Host", url.host)
            if (conn is HttpsURLConnection) {
                val httpsURLConnection = conn

                // https场景,证书校验
                httpsURLConnection.hostnameVerifier =
                    HostnameVerifier { _, session ->
                        var host = httpsURLConnection.getRequestProperty("Host")
                        if (null == host) {
                            host = httpsURLConnection.url.host
                        }
                        HttpsURLConnection.getDefaultHostnameVerifier().verify(host, session)
                    }

                // sni场景,创建SSLScocket
                httpsURLConnection.sslSocketFactory = TlsSniSocketFactory(httpsURLConnection)
            }
        }
        val code = conn.responseCode // Network block
        if (code in 300..399) {
            var location = conn.getHeaderField("Location")
            if (location == null) {
                location = conn.getHeaderField("location")
            }
            if (location != null) {
                if (!(location.startsWith("http://") || location
                        .startsWith("https://"))
                ) {
                    //某些时候会省略host,只返回后面的path,所以需要补全url
                    val originalUrl = URL(path)
                    location = (originalUrl.protocol + "://"
                            + originalUrl.host + location)
                }
                recursiveRequest(location)
            }
        } else {
            // redirect finish.
            val dis = DataInputStream(conn.inputStream)
            var len: Int
            val buff = ByteArray(4096)
            val response = StringBuilder()
            while (dis.read(buff).also { len = it } != -1) {
                response.append(String(buff, 0, len))
            }
            Log.d(TAG, "Response: $response")
        }
    } catch (e: MalformedURLException) {
        Log.w(TAG, "recursiveRequest MalformedURLException")
    } catch (e: IOException) {
        Log.w(TAG, "recursiveRequest IOException")
    } catch (e: java.lang.Exception) {
        Log.w(TAG, "unknow exception")
    } finally {
        conn?.disconnect()
    }
}
public void recursiveRequest(String path) {
    HttpURLConnection conn = null;
    try {
        URL url = new URL(path);
        conn = (HttpURLConnection) url.openConnection();
        // 同步接口获取IP
        HTTPDNSResult httpdnsResult = HttpDns.getService(context, accountID).getHttpDnsResultForHostSync(url.getHost(), RequestIpType.both);

        String ip = null;
        if (httpdnsResult.getIps() != null && httpdnsResult.getIps().length > 0) {
            ip = httpdnsResult.getIps()[0];
        } else if (httpdnsResult.getIpv6s() != null && httpdnsResult.getIpv6s().length > 0) {
            ip = httpdnsResult.getIpv6s()[0];
        }

        if (!TextUtils.isEmpty(ip)) {
            // 通过HTTPDNS获取IP成功,进行URL替换和HOST头设置
            String newUrl = path.replaceFirst(url.getHost(), ip);
            conn = (HttpURLConnection) new URL(newUrl).openConnection();

            conn.setConnectTimeout(30000);
            conn.setReadTimeout(30000);
            conn.setInstanceFollowRedirects(false);

            // 设置HTTP请求头Host域
            conn.setRequestProperty("Host", url.getHost());

            if (conn instanceof HttpsURLConnection) {
                final HttpsURLConnection httpsURLConnection = (HttpsURLConnection) conn;

                // https场景,证书校验
                httpsURLConnection.setHostnameVerifier(new HostnameVerifier() {
                    @Override
                    public boolean verify(String hostname, SSLSession session) {
                        String host = httpsURLConnection.getRequestProperty("Host");
                        if (null == host) {
                            host = httpsURLConnection.getURL().getHost();
                        }
                        return HttpsURLConnection.getDefaultHostnameVerifier().verify(host, session);
                    }
                });

                // sni场景,创建SSLScocket
                httpsURLConnection.setSSLSocketFactory(new TlsSniSocketFactory(httpsURLConnection));
            }
        }

        int code = conn.getResponseCode();// Network block
        if (code >= 300 && code < 400) {
            String location = conn.getHeaderField("Location");
            if (location == null) {
                location = conn.getHeaderField("location");
            }

            if (location != null) {
                if (!(location.startsWith("http://") || location
                        .startsWith("https://"))) {
                    //某些时候会省略host,只返回后面的path,所以需要补全url
                    URL originalUrl = new URL(path);
                    location = originalUrl.getProtocol() + "://"
                            + originalUrl.getHost() + location;
                }

                recursiveRequest(location);
            }
        } else {
            // redirect finish.
            DataInputStream dis = new DataInputStream(conn.getInputStream());
            int len;
            byte[] buff = new byte[4096];
            StringBuilder response = new StringBuilder();
            while ((len = dis.read(buff)) != -1) {
                response.append(new String(buff, 0, len));
            }
            Log.d(TAG, "Response: " + response.toString());
        }
    } catch (MalformedURLException e) {
        Log.w(TAG, "recursiveRequest MalformedURLException");
    } catch (IOException e) {
        Log.w(TAG, "recursiveRequest IOException");
    } catch (Exception e) {
        Log.w(TAG, "unknow exception");
    } finally {
        if (conn != null) {
            conn.disconnect();
        }
    }
}