阿里云CDN为您提供刷新预热自动化脚本,帮助您对文件或目录快速进行刷新和预热。本文为您介绍了该脚本的功能简介、使用说明,并以Windows系统示例为您说明。
功能简介
刷新预热自动化脚本可以帮助您分批进行刷新或预热任务,替代手动分批提交的繁琐操作。
当您指定刷新或预热URL列表文件后,脚本按照指定的并发刷新或预热数量对URL列表文件进行切割,分批进行刷新或预热。任务运行后会自动进行判断,等待上一个任务完成,脚本自动进行下一个刷新或预热任务的操作。
如果您有以下情况,建议您使用刷新预热自动化脚本:
- 无开发人员,需手动提交刷新预热任务,运维成本高。
- 刷新或预热URL过多,分批提交导致刷新或预热效率低。
- 需要人工或程序判断刷新预热任务是否正常进行,费时费力。
使用说明
在使用刷新预热自动化脚本之前,您需要在本机安装如下环境:
- Python CDN SDK模块包:目前使用版本为v20180510,建议您使用:pip install aliyun-python-sdk-cdn命令安装。
- Python阿里云核心包:目前使用版本为 2.6.0,建议您使用:pip install aliyun-python-sdk-core命令安装。
- Windows/Linux系统请安装Python 2.75及以上版本,不支持安装Python 3及以上版本。
- 将如下代码保存为Refresh.py脚本。
#!/usr/bin/env python #coding=utf-8 # __author__ = 'hanli.zyb' '''Check Package''' try: import os, sys, getopt, time, json from aliyunsdkcore.client import AcsClient from aliyunsdkcore.acs_exception.exceptions import ClientException from aliyunsdkcore.acs_exception.exceptions import ServerException from aliyunsdkcdn.request.v20180510.RefreshObjectCachesRequest import RefreshObjectCachesRequest from aliyunsdkcdn.request.v20180510.PushObjectCacheRequest import PushObjectCacheRequest from aliyunsdkcdn.request.v20180510.DescribeRefreshTasksRequest import DescribeRefreshTasksRequest from aliyunsdkcdn.request.v20180510.DescribeRefreshQuotaRequest import DescribeRefreshQuotaRequest except: sys.exit("[Error] Please pip install aliyun-python-sdk-cdn and aliyun-python-sdk-core ,please install now......") class Refresh(object): '''init func''' def __init__(self): self.lists = [] self.param = {} ''' 描述:调度的主函数 resP:检测入参结果,如果类型不是 bool 说明有报错 ''' def main(self,argv): if len(argv) < 1 : sys.exit("\nusage: " + sys.argv[0] + " -h ") try: opts,args = getopt.getopt(argv,"hi:k:n:r:t:a:o:") except Exception as e : sys.exit("\nusage: " + sys.argv[0] + " -h ") for opt,arg in opts: if opt == '-h': self.helps() sys.exit() elif opt == '-i': self.param['-i'] = arg elif opt == '-k': self.param['-k'] = arg elif opt == '-r': self.param['-r'] = arg elif opt == '-t': self.param['-t'] = arg elif opt == '-a': self.param['-a'] = arg elif opt == '-o': self.param['-o'] = arg elif opt == '-n': self.param['-n'] = arg else: sys.exit("\nusage: " + sys.argv[0] + " -h ") resP = self.doCheck(self.param) if not isinstance(resP,bool): sys.exit(resP) try: client = AcsClient(self.param['-i'], self.param['-k'], 'cn-hangzhou') except NameError: sys.exit("[Error]: SDK module not detected") for g in self.doProd(self.param): self.lists = [] self.doRefresh(''.join(g),self.param['-t'],client) ''' 描述:检测入参数 ''' def doCheck(self,param): try: for key1 in ('-i','-k','-r','-t'): if not key1 in param.keys(): return "[Error]: {0} Must be by parameter".format(key1) try: if not param.has_key('-n'): self.param['-n'] = 50 if not (abs(int(param['-n'])) <= 100 and abs(int(param['-n'])) > 0): return "[Error]: 0 < -n <= 100" else: self.param['-n'] = int(param['-n']) except ValueError as e: return "[Error]: -n Must be int Type ,{0}".format(str(e)) if not param['-t'] in ("push","clear"): return "[Error]: taskType Error" if param.has_key('-a') and param.has_key('-o'): return "[Error]: -a and -o cannot exist at same time" if param.has_key('-a'): if not param['-a'] in ("domestic","overseas"): return "[Error]: Area value Error" if param['-t'] == 'clear': return "[Error]: -t must be push and 'clear' -o use together" if param.has_key('-o'): if not param['-o'] in ("File","Directory"): return "[Error]: ObjectType value Error" if param['-t'] == 'push': return "[Error]: -t must be clear and 'push' -a use together" except KeyError as e: return "[Error]: Parameter {0} error".format(str(e)) return True ''' 描述:生成器切分文件,对每行文件进行处理 '\n' gop:每次读取 URL 数量 ''' def doProd(self,params): gop = params['-n'] mins = 1 maxs = gop with open(params['-r'], "r") as f: for line in f.readlines(): if mins != maxs: line = line.strip("\n") + "\n" else: line = line.strip("\n") self.lists.append(line) if mins >= maxs: yield self.lists mins = maxs maxs = gop + maxs -1 else: mins += 1 if len(self.lists) > 0: yield self.lists ''' 描述:刷新/预热任务 ''' def doRefresh(self,lists,types,client): try: if types == 'clear': taskID = 'RefreshTaskId' request = RefreshObjectCachesRequest() if self.param.has_key('-o'): request.set_ObjectType(self.param['-o']) elif types == 'push': taskID = 'PushTaskId' request = PushObjectCacheRequest() if self.param.has_key('-a'): request.set_Area(self.param['-a']) taskreq = DescribeRefreshTasksRequest() request.set_accept_format('json') request.set_ObjectPath(lists) response = json.loads(client.do_action_with_exception(request)) print(response) while True: count = 0 taskreq.set_accept_format('json') taskreq.set_TaskId(int(response[taskID])) taskresp = json.loads(client.do_action_with_exception(taskreq)) print("[" + response[taskID] + "]" + "is doing... ...") for t in taskresp['Tasks']['CDNTask']: if t['Status'] != 'Complete': count += 1 if count == 0: break else: continue time.sleep(5) except Exception as e: sys.exit("[Error]" + str(e)) ''' 描述:帮助信息 ''' def helps(self): print("\nscript options explain: \ \n\t -i <AccessKey> 访问阿里云凭证,访问控制台上可以获得; \ \n\t -k <AccessKeySecret> 访问阿里云密钥,访问控制台上可以获得; \ \n\t -r <filename> 文件名称,每行一条 URL,有特殊字符先做 URLencode,以 http/https 开头; \ \n\t -t <taskType> 任务类型 clear 刷新,push 预热; \ \n\t -n [int,[..100]] 可选项,每次操作文件数量,做多 100 条; \ \n\t -a [String,<domestic|overseas> 可选项,预热范围,不传是默认是全球;\ \n\t domestic 仅中国大陆; \ \n\t overseas 全球(不包含中国大陆); \ \n\t -o [String,<File|Directory>] 可选项,刷新的类型; \ \n\t File 文件刷新(默认值); \ \n\t Directory 目录刷新") #TODO 入口 if __name__ == '__main__': fun = Refresh() fun.main(sys.argv[1:])
安装成功后,在CMD窗口使用脚本帮助命令:python $scripte -h。帮助参数如下所示。
script options explain:
-i <AccessKey> //访问阿里云凭证,访问控制台获得;
-k <AccessKeySecret> //访问阿里云密匙,访问控制台获得;
-r <filename> //文件名称,每行一条URL,有特殊字符先做URLencode,以http/https开头;
-t <taskType> //任务类型:clear刷新,push预热;
-n [nums,[..100]] //可选项,每次操作文件数量,最多100条;
-a [String,<domestic|overseas> //可选项,预热范围,不传是默认是全球;
domestic //仅中国大陆;
overseas //全球(不包含中国大陆);
-o [String,<File|Directory>] 可选项,刷新的类型;
File //文件刷新(默认值);
Directory //目录刷新;
脚本示例
在CMD窗口,使用命令:python Refresh.py -i yourAccessKey -k yourAccessKeySecret -r filename -t clear。运行示例如下所示。
python Refresh.py -i yourAccessKey -k yourAccessKeySecret -r /root/test/test.lst -t clear
{u'RefreshTaskId': u'8211135448', u'RequestId': u'093602B9-6192-4137-8CCA-0D8C85729DF6'}
[8211135448]is doing... ...
[8211135448]is doing... ...
[8211135448]is doing... ...
[8211135448]is doing... ...
[8211135448]is doing... ...
...
{u'RefreshTaskId': u'8211136701', u'RequestId': u'C1896DDD-003B-41F9-B481-93BC509ED5A3'}
[8211136701]is doing... ...
[8211136701]is doing... ...
...
在文档使用中是否遇到以下问题
更多建议
匿名提交