全部产品
云市场

PAI视频分类

更新时间:2020-05-11 14:26:12

PAI平台提供视频分类相关算法,可以基于用户的短视频数据生成视频分类模型,支持千万级别差大规模的视频样本训练,生成的模型可以部署到PAI-EAS成为Restful API服务供调用。视频分类组件包含:数据格式转换和模型训练两部分。服务调用方式可使用MaxCompute console工具或者Dataworks SQL节点,详见:https://help.aliyun.com/document_detail/154185.html

实验提供一份demo数据,数据说明见下文,另外实验会需要一个与训练的模型,以及一个配置文件和一个数据地址文件。

下载地址:https://help.aliyun.com/document_detail/155513.htm

数据说明

原始数据可以是avi、mp4等常见格式的视频,本案例提供两份数据做分类模型训练,分别是eyemakeup和lipsmakeup,具体视频截图如下图所示:

数据格式转换

数据格式转换模块可以将原始的视频文件转换为tfrecord格式,这个格式的文件可以加快模型训练的速度。数据转换的命令如下:

  1. pai -name easy_vision_ext
  2. -project algo_public
  3. -Dbuckets='oss://{bucket_name}.{oss_host}/{path}/'
  4. -Darn='acs:ram::*******:role/aliyunodpspaidefaultrole'
  5. -DossHost='{oss_host}'
  6. -Dcmd convert
  7. -Dconvert_config='{bucket_name}.{oss_host}/{path}/{config_file}'
  8. -Dlabel_file='{bucket_name}.{oss_host}/{path}/{config_file}/{label_file}'
  9. -Doutput_tfrecord='{bucket_name}.{oss_host}/{path}/'

例如:

  1. pai -name easy_vision_ext
  2. -project algo_public
  3. -Dbuckets='oss://demo-yuze.oss-cn-beijing-internal.aliyuncs.com/vip/'
  4. -Darn='acs:ram::******:role/aliyunodpspaidefaultrole'
  5. -DossHost='oss-cn-beijing-internal.aliyuncs.com'
  6. -Dcmd convert
  7. -Dconvert_config='oss://demo-yuze.oss-cn-beijing-internal.aliyuncs.com/vip/ucf101_qince.config'
  8. -Dlabel_file='oss://demo-yuze.oss-cn-beijing-internal.aliyuncs.com/vip/vip.csv'
  9. -Doutput_tfrecord='oss://demo-yuze.oss-cn-beijing-internal.aliyuncs.com/vip/'
  • Dbuckets:OSS地址的根目录
  • Darn:访问OSS的授权,可以在https://help.aliyun.com/document_detail/154186.html 的IO相关参数说明中找到获取方法
  • DossHost:OSS的host地址
  • Dconvert_config:配置文件,本文开篇提供了下载地址,在配置文件中需要标记类别的种类,如下
  1. class_map {
  2. label_name: "ApplyEyeMakeup"
  3. }
  4. class_map {
  5. label_name: "ApplyLipstick"
  6. }
  7. model_type: VIDEO_CLASSIFICATION
  8. converter_class: "QinceConverter"
  9. write_thread_num: 8
  10. part_record_num: 64
  11. test_ratio: 0.0
  • Dlabel_file:训练视频的所在OSS地址,需要将视频上传到OSS并在文件中注明路径,例如,需要替换成自己的oss路径:

    1. 数据ID,原始数据,融合答案
    2. 1,"{""tfspath"": ""oss://demo-yuze/data/eye/public_v_ApplyEyeMakeup_g01_c01.avi""}","{""option"": ""ApplyEyeMakeup""}"
    3. 2,"{""tfspath"": ""oss://demo-yuze/data/eye/public_v_ApplyEyeMakeup_g02_c03.avi""}","{""option"": ""ApplyEyeMakeup""}"
    4. 3,"{""tfspath"": ""oss://demo-yuze/data/eye/public_v_ApplyEyeMakeup_g02_c04.avi""}","{""option"": ""ApplyEyeMakeup""}"
    5. 4,"{""tfspath"": ""oss://demo-yuze/data/eye/public_v_ApplyEyeMakeup_g03_c01.avi""}","{""option"": ""ApplyEyeMakeup""}"
    6. 5,"{""tfspath"": ""oss://demo-yuze/data/eye/public_v_ApplyEyeMakeup_g04_c01.avi""}","{""option"": ""ApplyEyeMakeup""}"
    7. 6,"{""tfspath"": ""oss://demo-yuze/data/lips/public_v_ApplyLipstick_g04_c02.avi""}","{""option"": ""ApplyEyeMakeup""}"
    8. 7,"{""tfspath"": ""oss://demo-yuze/data/lips/public_v_ApplyLipstick_g05_c01.avi""}","{""option"": ""ApplyLipstick""}"
    9. 8,"{""tfspath"": ""oss://demo-yuze/data/lips/public_v_ApplyLipstick_g07_c04.avi""}","{""option"": ""ApplyLipstick""}"
    10. 9,"{""tfspath"": ""oss://demo-yuze/data/lips/public_v_ApplyLipstick_g01_c02.avi""}","{""option"": ""ApplyLipstick""}"
  • Doutput_tfrecord:tfrecord的输出路径

视频分类模型训练

基于数据转换生成的数据,训练分类模型。命令参数:

  1. pai -name ev_video_classification_ext
  2. -project algo_public
  3. -Dbackbone='resnet_3d_50'
  4. -Dnum_epochs=50
  5. -Ddecay_epochs=5
  6. -Dsave_checkpoints_epochs=1
  7. -Dmodel_dir='{bucket_name}.{oss_host}/{output_model_path}/'
  8. -Duse_pretrained_model=true
  9. -Dpretrained_model='{bucket_name}.{oss_host}/{model_path}/resent_3d_50_model.ckpt'
  10. -Dtrain_data='{bucket_name}.{oss_host}/{path}/data_train_0_0.tfrecord'
  11. -Dtest_data='{bucket_name}.{oss_host}/{path}/data_train_0_0.tfrecord'
  12. -Dlabel_map_path='{bucket_name}.{oss_host}/{path}/data_label_map.pbtxt'
  13. -Dnum_test_example=10
  14. -Dtrain_batch_size=2
  15. -Dtest_batch_size=2
  16. -Dbuckets='{bucket_name}.{oss_host}/{path}'
  17. -Darn='acs:ram::*********:role/aliyunodpspaidefaultrole'
  18. -DossHost='{oss_host}'
  19. -Dinitial_learning_rate=0.0001
  20. -Dstaircase=false
  21. -DgpuRequired=100
  22. -Dnum_classes=2

示例:

  1. pai -name ev_video_classification_ext
  2. -project algo_public
  3. -Dbackbone='resnet_3d_50'
  4. -Dnum_epochs=50
  5. -Ddecay_epochs=5
  6. -Dsave_checkpoints_epochs=1
  7. -Dmodel_dir='oss://demo-yuze.oss-cn-beijing-internal.aliyuncs.com/model/'
  8. -Duse_pretrained_model=true
  9. -Dpretrained_model='oss://demo-yuze.oss-cn-beijing-internal.aliyuncs.com/model/resent_3d_50_model.ckpt'
  10. -Dtrain_data='oss://demo-yuze.oss-cn-beijing-internal.aliyuncs.com/vip/data_train_0_0.tfrecord'
  11. -Dtest_data='oss://demo-yuze.oss-cn-beijing-internal.aliyuncs.com/vip/data_train_0_0.tfrecord'
  12. -Dlabel_map_path='oss://demo-yuze.oss-cn-beijing-internal.aliyuncs.com/vip/data_label_map.pbtxt'
  13. -Dnum_test_example=10
  14. -Dtrain_batch_size=2
  15. -Dtest_batch_size=2
  16. -Dbuckets='oss://demo-yuze.oss-cn-beijing-internal.aliyuncs.com/vip/'
  17. -Darn='acs:ram::********:role/aliyunodpspaidefaultrole'
  18. -DossHost='oss-cn-beijing-internal.aliyuncs.com'
  19. -Dinitial_learning_rate=0.0001
  20. -Dstaircase=false
  21. -DgpuRequired=100
  22. -Dnum_classes=2
  • Dbackbone:选用的网络类型
  • Dmodel_dir:输出的模型地址
  • Dpretrained_model:上传的预训练模型地址,预训练模型上文给出了下载地址
  • Dtrain_data:数据转换生成的tfrecord文件
  • Dtest_data:数据转换生成的tfrecord文件
  • Dlabel_map_path:数据转换生成的.pbtxt文件
  • Dnum_test_example:测试的样本数
  • Dtrain_batch_size:每次参与训练的样本数
  • Dbuckets:根目录
  • Darn:见上文
  • Dnum_classes:分类个数

最终生成的模型会是Tensorflow的savemodel格式模型,可以在Dmodel_dir找到,该模型可以部署到PAI-EAS成为RestfulAPI服务,详见PAI-EAS文档:https://help.aliyun.com/document_detail/110985.html