并发过高导致DNS解析失败

报错信息

Logview中的报错信息包含name or service not known。

问题描述

UDF或Spark任务在运行过程中，产生了大量的并发请求以访问对端服务的域名，导致DNS解析失败。

解决方案

建议您在任务的初始化阶段将域名解析为IP地址，并在执行阶段使用解析得到的IP地址进行访问。示例如下：

# -*- coding:UTF-8 -*-
from odps.udf import annotate

@annotate("string->string")
class test_udf(object):
    __ip_address = ''

    def evaluate(self, inputPath):
        import requests
        output = 'false'
        retries = 3
        print(self.__ip_address)
        url = f"http://{self.__ip_address}/{inputPath}"
        print(url)
        for i in range(retries):
            try:
                response = requests.get(url)
                if response.status_code == 200:
                    output = 'true'
                else:
                    raise
            except Exception as e:
                if i < retries:
                    print('connect retry: ' + str(i + 1))
                    print('error: ' + e.message)
                    continue
                else:
                    raise
            break

        return output

    def __init__(self):
        import socket
        retries = 3
        for i in range(retries):
            try:
                self.__ip_address = socket.gethostbyname("xxx-vpc.cn-shanghai.aliyuncs.com")
                print(self.__ip_address)
            except socket.gaierror as e:
                print('Failed to resolv domain' + e.strerror)
                if i < retries:
                    print('resolv domain retry: ' + str(i + 1))
                    continue
                else:
                    raise
            break