Android端HTTPDNS+Webview最佳实践

重要

  • 本文档为Android WebView场景下接入HTTPDNS的参考方案,提供的相关代码也为参考代码,非线上生产环境正式代码,建议您仔细阅读本文档,进行合理评估后再进行接入。

  • 由于Android生态碎片化严重,各厂商也进行了不同程度的定制,建议您灰度接入,并监控线上异常,有问题欢迎您随时通过技术支持向我们反馈,方便我们及时优化。

  • 当前最佳实践文档只针对结合使用时,如何使用HTTPDNS解析出的IP,关于HTTPDNS本身的解析服务,请先查看Android SDK接入

背景说明

阿里云HTTPDNS是避免DNS劫持的一种有效手段,在许多特殊场景如Android端HTTPS(含SNI)业务场景:IP直连方案Android端HTTPDNS+OkHttp接入指南等都有最佳实践,但在webview场景下却一直没完美的解决方案。

但这并不代表在WebView场景下我们完全无法使用HTTPDNS,事实上很多场景依然可以通过HTTPDNS进行IP直连,本文旨在给出AndroidHTTPDNS+WebView最佳实践供用户参考。

代码示例

HTTPDNS+WebView最佳实践完整代码请参考WebView+HTTPDNS Android Demo

拦截接口说明

void setWebViewClient (WebViewClient client);

WebView提供了setWebViewClient接口对网络请求进行拦截,通过重载WebViewClient中的shouldInterceptRequest方法,我们可以拦截到所有的网络请求:

public class WebViewClient {
    // API < 21
    public WebResourceResponse shouldInterceptRequest(WebView view, String url) {
        ...
    }

   // API >= 21
    public WebResourceResponse shouldInterceptRequest(WebView view, WebResourceRequest request) {
        ...
    }
  ......

}

shouldInterceptRequest有两个版本:

  • API < 21时,shouldInterceptRequest方法的版本为:

    public WebResourceResponse shouldInterceptRequest(WebView view, String url)

    此时仅能获取到请求URL,请求方法、头部信息以及body等均无法获取,强行拦截该请求可能无法能到正确响应。所以当API < 21时,不对请求进行拦截:

    public WebResourceResponse shouldInterceptRequest(WebView view, String url) {
      return super.shouldInterceptRequest(view, url);
    }
  • API >= 21时,shouldInterceptRequest提供了新版:

    public WebResourceResponse shouldInterceptRequest(WebView view, WebResourceRequest request)

    其中WebResourceRequest结构为:

    public interface WebResourceRequest {
        Uri getUrl(); // 请求URL
        boolean isForMainFrame(); // 是否由主MainFrame发出的请求
        boolean isRedirect();  //是否重定向请求
        boolean hasGesture(); // 是否是由某种行为(如点击)触发
        String getMethod(); // 请求方法
        Map<String, String> getRequestHeaders(); // 头部信息
    }

可以看到,在API >= 21时,在拦截请求时,可以获取到如下信息:

  • 请求URL

  • 请求方法:POST, GET…

  • 请求头

实践使用

WebView场景下的请求拦截逻辑如下所示:

WebView场景下的请求拦截逻辑

  1. 仅拦截GET请求

  2. 设置头部信息

  3. HTTPS请求证书校验

  4. SNI场景

  5. 重定向

  6. MIME&Encoding

仅拦截GET请求

由于WebResourceRequest并没有提供请求body信息,所以只能拦截GET请求,代码示例如下:

override fun shouldInterceptRequest(view: WebView?, request: WebResourceRequest?): WebResourceResponse? {
    val scheme = request!!.url.scheme!!.trim()
    val method = request.method
    val headerFields = request.requestHeaders
    val url = request.url.toString()
    // 无法拦截body,拦截方案只能正常处理不带body的请求;
    if ((scheme.equals("http", ignoreCase = true) || scheme.equals("https", ignoreCase = true))
        && method.equals("get", ignoreCase = true)
    ) {
        //TODO 在 设置头部信息 部分介绍
    } else {
        return super.shouldInterceptRequest(view, request)
    }
}
public WebResourceResponse shouldInterceptRequest(WebView view, WebResourceRequest request) {
    String scheme = request.getUrl().getScheme().trim();
    String method = request.getMethod();
    Map<String, String> headerFields = request.getRequestHeaders();
    String url = request.getUrl().toString();
    // 无法拦截body,拦截方案只能正常处理不带body的请求;
    if ((scheme.equalsIgnoreCase("http") || scheme.equalsIgnoreCase("https"))
            && method.equalsIgnoreCase("get")) {
        //TODO 在 设置头部信息 部分介绍
    } else {
        return super.shouldInterceptRequest(view, request);
    }
}

设置头部信息

这里把请求资源的方法抽象出来。

fun recursiveRequest(
    path: String,
    headers: Map<String?, String?>?,
    reffer: String?
): URLConnection? {
    val conn: HttpURLConnection
    var url: URL? = null
    try {
        url = URL(path)
        // 同步接口获取IP
        val httpdnsResult = HttpDns.getService(context, accountID)
            .getHttpDnsResultForHostSync(url.host, RequestIpType.both)
        var ip: String? = null
        if (httpdnsResult.ips != null && httpdnsResult.ips.isNotEmpty()) {
            ip = httpdnsResult.ips[0]
        } else if (httpdnsResult.ipv6s != null && httpdnsResult.ipv6s.isNotEmpty()) {
            ip = httpdnsResult.ipv6s[0]
        }
        if (!TextUtils.isEmpty(ip)) {
            // 通过HTTPDNS获取IP成功,进行URL替换和HOST头设置
            val newUrl = path.replaceFirst(url.host.toRegex(), ip!!)
            conn = URL(newUrl).openConnection() as HttpURLConnection
            conn.connectTimeout = 30000
            conn.readTimeout = 30000
            conn.instanceFollowRedirects = false
            // 添加原有头部信息
            if (headers != null) {
                headers.forEach{ entry ->
                    conn.setRequestProperty(entry.key, entry.value)
                }
            }
            // 设置HTTP请求头Host域
            conn.setRequestProperty("Host", url.host)

            //TODO 在 HTTPS请求证书校验 部分介绍
        } else {
            return null
        }

        //TODO 在 重定向 部分介绍
    } catch (e: MalformedURLException) {
        Log.w(TAG, "recursiveRequest MalformedURLException")
    } catch (e: IOException) {
        Log.w(TAG, "recursiveRequest IOException")
    } catch (e: Exception) {
        Log.w(TAG, "unknow exception")
    }
    return null
}
public URLConnection recursiveRequest(String path, Map<String, String> headers, String reffer) {
    HttpURLConnection conn;
    URL url = null;
    try {
        url = new URL(path);
        // 同步接口获取IP
        HTTPDNSResult httpdnsResult = HttpDns.getService(context, accountID).getHttpDnsResultForHostSync(url.getHost(), RequestIpType.both);

        String ip = null;
        if (httpdnsResult.getIps() != null && httpdnsResult.getIps().length > 0) {
            ip = httpdnsResult.getIps()[0];
        } else if (httpdnsResult.getIpv6s() != null && httpdnsResult.getIpv6s().length > 0) {
            ip = httpdnsResult.getIpv6s()[0];
        }

        if (!TextUtils.isEmpty(ip)) {
            // 通过HTTPDNS获取IP成功,进行URL替换和HOST头设置
            String newUrl = path.replaceFirst(url.getHost(), ip);
            conn = (HttpURLConnection) new URL(newUrl).openConnection();

            conn.setConnectTimeout(30000);
            conn.setReadTimeout(30000);
            conn.setInstanceFollowRedirects(false);
            // 添加原有头部信息
            if (headers != null) {
                for (Map.Entry<String, String> field : headers.entrySet()) {
                    conn.setRequestProperty(field.getKey(), field.getValue());
                }
            }
            // 设置HTTP请求头Host域
            conn.setRequestProperty("Host", url.getHost());

            //TODO 在 HTTPS请求证书校验 部分介绍
        } else {
            return null;
        }        
        
        //TODO 在 重定向 部分介绍
    } catch (MalformedURLException e) {
        Log.w(TAG, "recursiveRequest MalformedURLException");
    } catch (IOException e) {
        Log.w(TAG, "recursiveRequest IOException");
    } catch (Exception e) {
        Log.w(TAG, "unknow exception");
    }
    return null;
}
override fun shouldInterceptRequest(
    view: WebView?,
    request: WebResourceRequest?
): WebResourceResponse? {
    val scheme = request!!.url.scheme!!.trim()
    val method = request.method
    val headerFields = request.requestHeaders
    val url = request.url.toString()
    // 无法拦截body,拦截方案只能正常处理不带body的请求;
    if ((scheme.equals("http", ignoreCase = true) || scheme.equals("https", ignoreCase = true))
        && method.equals("get", ignoreCase = true)
    ) {
        try {
            val connection = recursiveRequest(url, headerFields, null)
                ?: return super.shouldInterceptRequest(view, request)

            //TODO 在 MIME&Encoding 部分介绍
        } catch (e: MalformedURLException) {
            e.printStackTrace()
        } catch (e: IOException) {
            e.printStackTrace()
        }
    }
    return super.shouldInterceptRequest(view, request)
}
public WebResourceResponse shouldInterceptRequest(WebView view, WebResourceRequest request) {
    String scheme = request.getUrl().getScheme().trim();
    String method = request.getMethod();
    Map<String, String> headerFields = request.getRequestHeaders();
    String url = request.getUrl().toString();
    // 无法拦截body,拦截方案只能正常处理不带body的请求;
    if ((scheme.equalsIgnoreCase("http") || scheme.equalsIgnoreCase("https"))
            && method.equalsIgnoreCase("get")) {
        try {
            URLConnection connection = recursiveRequest(url, headerFields, null);

            if (connection == null) {
                return super.shouldInterceptRequest(view, request);
            }

            //TODO 在 MIME&Encoding 部分介绍
        } catch (MalformedURLException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }
    } 
    return super.shouldInterceptRequest(view, request);
}

HTTPS请求证书校验

如果拦截到的请求是HTTPS请求,需要进行证书校验:

fun recursiveRequest(
    path: String,
    headers: Map<String?, String?>?,
    reffer: String?
): URLConnection? {
    val conn: HttpURLConnection
    var url: URL? = null
    try {
        url = URL(path)
        // 同步接口获取IP
        val httpdnsResult = HttpDns.getService(context, accountID)
            .getHttpDnsResultForHostSync(url.host, RequestIpType.both)
        var ip: String? = null
        if (httpdnsResult.ips != null && httpdnsResult.ips.size > 0) {
            ip = httpdnsResult.ips[0]
        } else if (httpdnsResult.ipv6s != null && httpdnsResult.ipv6s.size > 0) {
            ip = httpdnsResult.ipv6s[0]
        }
        if (!TextUtils.isEmpty(ip)) {
            // 通过HTTPDNS获取IP成功,进行URL替换和HOST头设置
            val newUrl = path.replaceFirst(url.host.toRegex(), ip!!)
            conn = URL(newUrl).openConnection() as HttpURLConnection
            conn.connectTimeout = 30000
            conn.readTimeout = 30000
            conn.instanceFollowRedirects = false
            // 添加原有头部信息
            headers?.forEach{ entry ->
                conn.setRequestProperty(entry.key, entry.value)
            }
            // 设置HTTP请求头Host域
            conn.setRequestProperty("Host", url.host)
            if (conn is HttpsURLConnection) {
                // https场景,证书校验
                conn.hostnameVerifier =
                    HostnameVerifier { _, session ->
                        var host = conn.getRequestProperty("Host")
                        if (null == host) {
                            host = conn.url.host
                        }
                        HttpsURLConnection.getDefaultHostnameVerifier().verify(host, session)
                    }

                //TODO 在 SNI场景 部分介绍
            }

            //TODO 在 重定向 部分介绍
        } else {
            return null
        }
    } catch (e: MalformedURLException) {
        Log.w(TAG, "recursiveRequest MalformedURLException")
    } catch (e: IOException) {
        Log.w(TAG, "recursiveRequest IOException")
    } catch (e: Exception) {
        Log.w(TAG, "unknow exception")
    }
    return null
}
public URLConnection recursiveRequest(String path, Map<String, String> headers, String reffer) {
    HttpURLConnection conn;
    URL url = null;
    try {
        url = new URL(path);
        // 同步接口获取IP
        HTTPDNSResult httpdnsResult = HttpDns.getService(context, accountID).getHttpDnsResultForHostSync(url.getHost(), RequestIpType.both);

        String ip = null;
        if (httpdnsResult.getIps() != null && httpdnsResult.getIps().length > 0) {
            ip = httpdnsResult.getIps()[0];
        } else if (httpdnsResult.getIpv6s() != null && httpdnsResult.getIpv6s().length > 0) {
            ip = httpdnsResult.getIpv6s()[0];
        }

        if (!TextUtils.isEmpty(ip)) {
            // 通过HTTPDNS获取IP成功,进行URL替换和HOST头设置
            String newUrl = path.replaceFirst(url.getHost(), ip);
            conn = (HttpURLConnection) new URL(newUrl).openConnection();

            conn.setConnectTimeout(30000);
            conn.setReadTimeout(30000);
            conn.setInstanceFollowRedirects(false);
            // 添加原有头部信息
            if (headers != null) {
                for (Map.Entry<String, String> field : headers.entrySet()) {
                    conn.setRequestProperty(field.getKey(), field.getValue());
                }
            }
            // 设置HTTP请求头Host域
            conn.setRequestProperty("Host", url.getHost());

            if (conn instanceof HttpsURLConnection) {
                HttpsURLConnection httpsURLConnection = (HttpsURLConnection)conn;
                // https场景,证书校验
                httpsURLConnection.setHostnameVerifier(new HostnameVerifier() {
                    @Override
                    public boolean verify(String hostname, SSLSession session) {
                        String host = httpsURLConnection.getRequestProperty("Host");
                        if (null == host) {
                            host = httpsURLConnection.getURL().getHost();
                        }
                        return HttpsURLConnection.getDefaultHostnameVerifier().verify(host, session);
                    }
                });

                //TODO 在 SNI场景 部分介绍
            }

            //TODO 在 重定向 部分介绍
        } else {
            return null;
        }        

        
    } catch (MalformedURLException e) {
        Log.w(TAG, "recursiveRequest MalformedURLException");
    } catch (IOException e) {
        Log.w(TAG, "recursiveRequest IOException");
    } catch (Exception e) {
        Log.w(TAG, "unknow exception");
    }
    return null;
}

SNI场景

如果请求涉及到SNI场景,需要自定义SSLSocket,对SNI场景不熟悉的用户可以参考SNI

自定义SSLSocket

class TlsSniSocketFactory constructor(conn: HttpsURLConnection): SSLSocketFactory() {
    private val mConn: HttpsURLConnection

    init {
        mConn = conn
    }

    override fun createSocket(plainSocket: Socket?, host: String?, port: Int, autoClose: Boolean): Socket {
        var peerHost = mConn.getRequestProperty("Host")
        if (peerHost == null) {
            peerHost = host
        }

        val address = plainSocket!!.inetAddress
        if (autoClose) {
            // we don't need the plainSocket
            plainSocket.close()
        }

        // create and connect SSL socket, but don't do hostname/certificate verification yet
        val sslSocketFactory =
            SSLCertificateSocketFactory.getDefault(0) as SSLCertificateSocketFactory
        val ssl = sslSocketFactory.createSocket(address, R.attr.port) as SSLSocket

        // enable TLSv1.1/1.2 if available
        ssl.enabledProtocols = ssl.supportedProtocols

        // set up SNI before the handshake
        if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.JELLY_BEAN_MR1) {
            // setting sni hostname
            sslSocketFactory.setHostname(ssl, peerHost)
        } else {
            // No documented SNI support on Android <4.2, trying with reflection
            try {
                val setHostnameMethod = ssl.javaClass.getMethod(
                    "setHostname",
                    String::class.java
                )
                setHostnameMethod.invoke(ssl, peerHost)
            } catch (e: Exception) {
            }
        }

        // verify hostname and certificate
        val session = ssl.session

        if (!HttpsURLConnection.getDefaultHostnameVerifier()
                .verify(peerHost, session)
        ) throw SSLPeerUnverifiedException(
            "Cannot verify hostname: $peerHost"
        )

        return ssl
    }
}
public class TlsSniSocketFactory extends SSLSocketFactory {
    
    private HttpsURLConnection mConn;
    public TlsSniSocketFactory(HttpsURLConnection conn) {
        mConn = conn;
    }

    @Override
    public Socket createSocket(Socket plainSocket, String host, int port, boolean autoClose) throws IOException {
        String peerHost = mConn.getRequestProperty("Host");
        if (peerHost == null)
            peerHost = host;

        InetAddress address = plainSocket.getInetAddress();
        if (autoClose) {
            // we don't need the plainSocket
            plainSocket.close();
        }
        // create and connect SSL socket, but don't do hostname/certificate verification yet
        SSLCertificateSocketFactory sslSocketFactory = (SSLCertificateSocketFactory) SSLCertificateSocketFactory.getDefault(0);
        SSLSocket ssl = (SSLSocket) sslSocketFactory.createSocket(address, port);

        // enable TLSv1.1/1.2 if available
        ssl.setEnabledProtocols(ssl.getSupportedProtocols());

        // set up SNI before the handshake
        if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.JELLY_BEAN_MR1) {
            // setting sni hostname
            sslSocketFactory.setHostname(ssl, peerHost);
        } else {
            // No documented SNI support on Android <4.2, trying with reflection
            try {
                java.lang.reflect.Method setHostnameMethod = ssl.getClass().getMethod("setHostname", String.class);
                setHostnameMethod.invoke(ssl, peerHost);
            } catch (Exception e) {

            }
        }

        // verify hostname and certificate
        SSLSession session = ssl.getSession();

        if (!HttpsURLConnection.getDefaultHostnameVerifier().verify(peerHost, session))
            throw new SSLPeerUnverifiedException("Cannot verify hostname: " + peerHost);

        return ssl;
    }
}

设置自定义SSLSocket

fun recursiveRequest(
    path: String,
    headers: Map<String?, String?>?,
    reffer: String?
): URLConnection? {
    val conn: HttpURLConnection
    var url: URL? = null
    try {
        url = URL(path)
        // 同步接口获取IP
        val httpdnsResult = HttpDns.getService(context, accountID)
            .getHttpDnsResultForHostSync(url.host, RequestIpType.both)
        var ip: String? = null
        if (httpdnsResult.ips != null && httpdnsResult.ips.isNotEmpty()) {
            ip = httpdnsResult.ips[0]
        } else if (httpdnsResult.ipv6s != null && httpdnsResult.ipv6s.isNotEmpty()) {
            ip = httpdnsResult.ipv6s[0]
        }
        if (!TextUtils.isEmpty(ip)) {
            // 通过HTTPDNS获取IP成功,进行URL替换和HOST头设置
            val newUrl: String = path.replaceFirst(url.host, ip)
            conn = URL(newUrl).openConnection() as HttpURLConnection
            conn.connectTimeout = 30000
            conn.readTimeout = 30000
            conn.instanceFollowRedirects = false
            // 添加原有头部信息
            headers?.forEach { entry ->
                conn.setRequestProperty(entry.key, entry.value)
            }
            // 设置HTTP请求头Host域
            conn.setRequestProperty("Host", url.host)
            if (conn is HttpsURLConnection) {
                // https场景,证书校验
                conn.hostnameVerifier = HostnameVerifier { _, session ->
                    var host = conn.getRequestProperty("Host")
                    if (null == host) {
                        host = conn.url.host
                    }
                    HttpsURLConnection.getDefaultHostnameVerifier().verify(host, session)
                }

                // sni场景,创建SSLScocket
                conn.sslSocketFactory = TlsSniSocketFactory(conn)
            }

            //TODO 在 重定向 部分介绍
        } else {
            return null
        }
    } catch (e: MalformedURLException) {
        Log.w(TAG, "recursiveRequest MalformedURLException")
    } catch (e: IOException) {
        Log.w(TAG, "recursiveRequest IOException")
    } catch (e: Exception) {
        Log.w(TAG, "unknow exception")
    }
    return null
}
public URLConnection recursiveRequest(String path, Map<String, String> headers, String reffer) {
    HttpURLConnection conn;
    URL url = null;
    try {
        url = new URL(path);
        // 同步接口获取IP
        HTTPDNSResult httpdnsResult = HttpDns.getService(context, accountID).getHttpDnsResultForHostSync(url.getHost(), RequestIpType.both);

        String ip = null;
        if (httpdnsResult.getIps() != null && httpdnsResult.getIps().length > 0) {
            ip = httpdnsResult.getIps()[0];
        } else if (httpdnsResult.getIpv6s() != null && httpdnsResult.getIpv6s().length > 0) {
            ip = httpdnsResult.getIpv6s()[0];
        }

        if (!TextUtils.isEmpty(ip)) {
            // 通过HTTPDNS获取IP成功,进行URL替换和HOST头设置
            String newUrl = path.replaceFirst(url.getHost(), ip);
            conn = (HttpURLConnection) new URL(newUrl).openConnection();

            conn.setConnectTimeout(30000);
            conn.setReadTimeout(30000);
            conn.setInstanceFollowRedirects(false);
            // 添加原有头部信息
            if (headers != null) {
                for (Map.Entry<String, String> field : headers.entrySet()) {
                    conn.setRequestProperty(field.getKey(), field.getValue());
                }
            }
            // 设置HTTP请求头Host域
            conn.setRequestProperty("Host", url.getHost());

            if (conn instanceof HttpsURLConnection) {
                HttpsURLConnection httpsURLConnection = (HttpsURLConnection)conn;
                // https场景,证书校验
                httpsURLConnection.setHostnameVerifier(new HostnameVerifier() {
                    @Override
                    public boolean verify(String hostname, SSLSession session) {
                        String host = httpsURLConnection.getRequestProperty("Host");
                        if (null == host) {
                            host = httpsURLConnection.getURL().getHost();
                        }
                        return HttpsURLConnection.getDefaultHostnameVerifier().verify(host, session);
                    }
                });

                // sni场景,创建SSLScocket
                httpsURLConnection.setSSLSocketFactory(new TlsSniSocketFactory(httpsURLConnection));
            }

            //TODO 在 重定向 部分介绍
        } else {
            return null;
        }        
    } catch (MalformedURLException e) {
        Log.w(TAG, "recursiveRequest MalformedURLException");
    } catch (IOException e) {
        Log.w(TAG, "recursiveRequest IOException");
    } catch (Exception e) {
        Log.w(TAG, "unknow exception");
    }
    return null;
}

重定向

如果服务端返回重定向,此时需要判断原有请求中是否含有cookie:

  • 如果原有请求报头含有cookie,因为cookie是以域名为粒度进行存储的,重定向后cookie会改变,且无法获取到新请求URL下的cookie,所以放弃拦截。

  • 如果不含cookie,重新发起二次请求。

fun recursiveRequest(
    path: String,
    headers: Map<String, String?>?,
    reffer: String?
): URLConnection? {
    val conn: HttpURLConnection
    var url: URL? = null
    try {
        url = URL(path)
        // 同步接口获取IP
        val httpdnsResult = HttpDns.getService(context, accountID)
            .getHttpDnsResultForHostSync(url.host, RequestIpType.both)
        var ip: String? = null
        if (httpdnsResult.ips != null && httpdnsResult.ips.isNotEmpty()) {
            ip = httpdnsResult.ips[0]
        } else if (httpdnsResult.ipv6s != null && httpdnsResult.ipv6s.isNotEmpty()) {
            ip = httpdnsResult.ipv6s[0]
        }
        if (!TextUtils.isEmpty(ip)) {
            // 通过HTTPDNS获取IP成功,进行URL替换和HOST头设置
            val newUrl = path.replaceFirst(url.host.toRegex(), ip!!)
            conn = URL(newUrl).openConnection() as HttpURLConnection
            conn.connectTimeout = 30000
            conn.readTimeout = 30000
            conn.instanceFollowRedirects = false
            // 添加原有头部信息
            headers?.forEach { entry ->
                conn.setRequestProperty(entry.key, entry.value)
            }
            // 设置HTTP请求头Host域
            conn.setRequestProperty("Host", url.host)
            if (conn is HttpsURLConnection) {
                // https场景,证书校验
                conn.hostnameVerifier =
                    HostnameVerifier { _, session ->
                        var host = conn.getRequestProperty("Host")
                        if (null == host) {
                            host = conn.url.host
                        }
                        HttpsURLConnection.getDefaultHostnameVerifier().verify(host, session)
                    }

                // sni场景,创建SSLScocket
                conn.sslSocketFactory = TlsSniSocketFactory(conn)
            }
        } else {
            return null
        }
        val code = conn.responseCode // Network block
        return if (code in 300..399) {
            // 原有报头中含有cookie,放弃拦截
            var containCookie = false
            if (headers != null) {
                for (item in headers.keys) {
                    if (item.contains("Cookie")) {
                        containCookie = true
                        break
                    }
                }
            }
            
            if (containCookie) {
                return null
            }
            var location = conn.getHeaderField("Location")
            if (location == null) {
                location = conn.getHeaderField("location")
            }
            if (location != null) {
                if (!(location.startsWith("http://") || location
                        .startsWith("https://"))
                ) {
                    //某些时候会省略host,只返回后面的path,所以需要补全url
                    val originalUrl = URL(path)
                    location = (originalUrl.protocol + "://"
                            + originalUrl.host + location)
                }
                recursiveRequest(location, headers, path)
            } else {
                // 无法获取location信息,让浏览器获取
                null
            }
        } else {
            // redirect finish.
            conn
        }
    } catch (e: MalformedURLException) {
        Log.w(TAG, "recursiveRequest MalformedURLException")
    } catch (e: IOException) {
        Log.w(TAG, "recursiveRequest IOException")
    } catch (e: Exception) {
        Log.w(TAG, "unknow exception")
    }
    return null
}
public URLConnection recursiveRequest(String path, Map<String, String> headers, String reffer) {
    HttpURLConnection conn;
    URL url = null;
    try {
        url = new URL(path);
        // 同步接口获取IP
        HTTPDNSResult httpdnsResult = HttpDns.getService(context, accountID).getHttpDnsResultForHostSync(url.getHost(), RequestIpType.both);

        String ip = null;
        if (httpdnsResult.getIps() != null && httpdnsResult.getIps().length > 0) {
            ip = httpdnsResult.getIps()[0];
        } else if (httpdnsResult.getIpv6s() != null && httpdnsResult.getIpv6s().length > 0) {
            ip = httpdnsResult.getIpv6s()[0];
        }

        if (!TextUtils.isEmpty(ip)) {
            // 通过HTTPDNS获取IP成功,进行URL替换和HOST头设置
            String newUrl = path.replaceFirst(url.getHost(), ip);
            conn = (HttpURLConnection) new URL(newUrl).openConnection();

            conn.setConnectTimeout(30000);
            conn.setReadTimeout(30000);
            conn.setInstanceFollowRedirects(false);
            // 添加原有头部信息
            if (headers != null) {
                for (Map.Entry<String, String> field : headers.entrySet()) {
                    conn.setRequestProperty(field.getKey(), field.getValue());
                }
            }
            // 设置HTTP请求头Host域
            conn.setRequestProperty("Host", url.getHost());

            if (conn instanceof HttpsURLConnection) {
                final HttpsURLConnection httpsURLConnection = (HttpsURLConnection) conn;
    
                // https场景,证书校验
                httpsURLConnection.setHostnameVerifier(new HostnameVerifier() {
                    @Override
                    public boolean verify(String hostname, SSLSession session) {
                        String host = httpsURLConnection.getRequestProperty("Host");
                        if (null == host) {
                            host = httpsURLConnection.getURL().getHost();
                        }
                        return HttpsURLConnection.getDefaultHostnameVerifier().verify(host, session);
                    }
                });

                // sni场景,创建SSLScocket
                httpsURLConnection.setSSLSocketFactory(new TlsSniSocketFactory(httpsURLConnection));
            }
        } else {
            return null;
        }        

        int code = conn.getResponseCode();// Network block
        if (code >= 300 && code < 400) {
            // 原有报头中含有cookie,放弃拦截
            boolean containCookie = false;
            if (headers != null) {
              for (Map.Entry<String, String> headerField : headers.entrySet()) {
                  if (headerField.getKey().contains("Cookie")) {
                      containCookie = true;
                      break;
                  }
              }
            }

            if (containCookie) {
                return null;
            }

            String location = conn.getHeaderField("Location");
            if (location == null) {
                location = conn.getHeaderField("location");
            }

            if (location != null) {
                if (!(location.startsWith("http://") || location
                        .startsWith("https://"))) {
                    //某些时候会省略host,只返回后面的path,所以需要补全url
                    URL originalUrl = new URL(path);
                    location = originalUrl.getProtocol() + "://"
                            + originalUrl.getHost() + location;
                }

                return recursiveRequest(location, headers, path);
            } else {
                // 无法获取location信息,让浏览器获取
                return null;
            }
        } else {
            // redirect finish.
            return conn;
        }
    } catch (MalformedURLException e) {
        Log.w(TAG, "recursiveRequest MalformedURLException");
    } catch (IOException e) {
        Log.w(TAG, "recursiveRequest IOException");
    } catch (Exception e) {
        Log.w(TAG, "unknow exception");
    }
    return null;
}

MIME&Encoding

如果拦截网络请求,需要返回一个WebResourceResponse

public WebResourceResponse(String mimeType, String encoding, InputStream data) ;

创建WebResourceResponse对象需要提供:

  • 请求的MIME类型

  • 请求的编码

  • 请求的输入流

其中请求输入流可以通过URLConnection.getInputStream()获取到,而MIME类型和encoding可以通过请求的ContentType获取到,即通过URLConnection.getContentType(),如:

text/html;charset=utf-8

但并不是所有的请求都能得到完整的contentType信息,此时可以参考如下策略:

override fun shouldInterceptRequest(
    view: WebView?,
    request: WebResourceRequest?
): WebResourceResponse? {
    val scheme = request!!.url.scheme!!.trim()
    val method = request.method
    val headerFields = request.requestHeaders
    val url = request.url.toString()
    // 无法拦截body,拦截方案只能正常处理不带body的请求;
    if ((scheme.equals("http", ignoreCase = true) || scheme.equals("https", ignoreCase = true))
        && method.equals("get", ignoreCase = true)
    ) {
        try {
            val connection = recursiveRequest(url, headerFields, null)
                ?: return super.shouldInterceptRequest(view, request)

            // 注*:对于POST请求的Body数据,WebResourceRequest接口中并没有提供,这里无法处理
            val contentType = connection.contentType
            var mime: String? = null
            val charset: String? = null
            if (contentType != null) {
                mime = contentType.split(";".toRegex()).dropLastWhile { it.isEmpty() }
                    .toTypedArray()[0]
                val fields = contentType.split(";".toRegex()).dropLastWhile { it.isEmpty() }
                    .toTypedArray()
                if (fields.size > 1) {
                    var charset = fields[1]
                    if (charset.contains("=")) {
                        charset = charset.substring(charset.indexOf("=") + 1)
                    }
                }
            }
            val httpURLConnection = connection as HttpURLConnection
            val statusCode = httpURLConnection.responseCode
            val response = httpURLConnection.responseMessage
            val headers = httpURLConnection.headerFields
            val headerKeySet: Set<String> = headers.keys

            // 无mime类型的请求不拦截
            return if (TextUtils.isEmpty(mime)) {
                super.shouldInterceptRequest(view, request)
            } else {
                // 二进制资源无需编码信息
                if (!TextUtils.isEmpty(charset) || (mime!!.startsWith("image")
                            || mime.startsWith("audio")
                            || mime.startsWith("video"))
                ) {
                    val resourceResponse =
                        WebResourceResponse(mime, charset, httpURLConnection.inputStream)
                    resourceResponse.setStatusCodeAndReasonPhrase(statusCode, response)
                    val responseHeader: MutableMap<String, String> = HashMap()
                    for (key in headerKeySet) {
                        // HttpUrlConnection可能包含key为null的报头,指向该http请求状态码
                        responseHeader[key] = httpURLConnection.getHeaderField(key)
                    }
                    resourceResponse.responseHeaders = responseHeader
                    resourceResponse
                } else {
                    super.shouldInterceptRequest(view, request)
                }
            }
        } catch (e: MalformedURLException) {
            e.printStackTrace()
        } catch (e: IOException) {
            e.printStackTrace()
        }
    } 
    
    return super.shouldInterceptRequest(view, request)
}
public WebResourceResponse shouldInterceptRequest(WebView view, WebResourceRequest request) {
    String scheme = request.getUrl().getScheme().trim();
    String method = request.getMethod();
    Map<String, String> headerFields = request.getRequestHeaders();
    String url = request.getUrl().toString();
    // 无法拦截body,拦截方案只能正常处理不带body的请求;
    if ((scheme.equalsIgnoreCase("http") || scheme.equalsIgnoreCase("https"))
            && method.equalsIgnoreCase("get")) {
        try {
            URLConnection connection = recursiveRequest(url, headerFields, null);

            if (connection == null) {
                return super.shouldInterceptRequest(view, request);
            }

            // 注*:对于POST请求的Body数据,WebResourceRequest接口中并没有提供,这里无法处理
            String contentType = connection.getContentType();
            String mime = null;
            String charset = null;
            if (contentType != null) {
                mime = contentType.split(";")[0];

                String[] fields = contentType.split(";");
                if (fields.length > 1) {
                    String charset = fields[1];
                    if (charset.contains("=")) {
                        charset = charset.substring(charset.indexOf("=") + 1);
                    }
                }
            }
            
            HttpURLConnection httpURLConnection = (HttpURLConnection) connection;
            int statusCode = httpURLConnection.getResponseCode();
            String response = httpURLConnection.getResponseMessage();
            Map<String, List<String>> headers = httpURLConnection.getHeaderFields();
            Set<String> headerKeySet = headers.keySet();

            // 无mime类型的请求不拦截
            if (TextUtils.isEmpty(mime)) {
                return super.shouldInterceptRequest(view, request);
            } else {
                // 二进制资源无需编码信息
                if (!TextUtils.isEmpty(charset) || (mime.startsWith("image")
                        || mime.startsWith("audio")
                        || mime.startsWith("video"))) {
                    WebResourceResponse resourceResponse = new WebResourceResponse(mime, charset, httpURLConnection.getInputStream());
                    resourceResponse.setStatusCodeAndReasonPhrase(statusCode, response);
                    Map<String, String> responseHeader = new HashMap<String, String>();
                    for (String key : headerKeySet) {
                        // HttpUrlConnection可能包含key为null的报头,指向该http请求状态码
                        responseHeader.put(key, httpURLConnection.getHeaderField(key));
                    }
                    resourceResponse.setResponseHeaders(responseHeader);
                    return resourceResponse;
                } else {
                    return super.shouldInterceptRequest(view, request);
                }
            }
        } catch (MalformedURLException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }
    } 
    return super.shouldInterceptRequest(view, request);
}

总结

场景

总结

不可用场景

  • API Level < 21的设备

  • POST请求

  • 无法获取到MIME类型的请求

  • 无法获取到编码的非二进制文件请求

可用场景

前提条件:

  • API Level >= 21

  • GET请求

  • 可以获取到MIME类型以及编码信息请求或是可以获取到MIME类型的二进制文件请求

可用场景:

  • 普通HTTP请求

  • HTTPS请求

  • SNI请求

  • HTTP报头中不含cookie的重定向请求