全部产品
云市场

Tensorflow服务请求构造

更新时间:2019-05-27 14:58:02

TensorFlow服务请求构造

输入数据说明

PAI-EAS内置了tensorflow processor,用户可一键将tensorflow模型部署成服务,出于性能考虑输入输出为protobuf,用户需按需求以protobuf的形式构建输入。下面我们以一个实际的例子进讲解如何对基于通用processor的tensorflow服务构造调用数据并进行服务请求。

调用案例

我们在cn-shanghai部署了一个public的测试Demo,服务名字为mnist_saved_model_example,访问token为空,在上海vpc环境中,用户可通过
http://pai-eas-vpc.cn-shanghai.aliyuncs.com/api/predict/mnist_saved_model_example
这个地址对服务进行访问,具体请求方式可按照以下步骤操作。

1. 获取模型信息

若不清楚模型的相关信息,可通过GET请求获取,返回结果中包括signature_name以及输入输出的name,type和shape,如下:

  1. $curl http://pai-eas-vpc.cn-shanghai.aliyuncs.com/api/predict/mnist_saved_model_example | python -mjson.tool
  2. {
  3. "inputs": [
  4. {
  5. "name": "images",
  6. "shape": [
  7. -1,
  8. 784
  9. ],
  10. "type": "DT_FLOAT"
  11. }
  12. ],
  13. "outputs": [
  14. {
  15. "name": "scores",
  16. "shape": [
  17. -1,
  18. 10
  19. ],
  20. "type": "DT_FLOAT"
  21. }
  22. ],
  23. "signature_name": "predict_images"
  24. }

该模型是一个对mnist进行分类的模型,输入是的类型是DT_FLOAT,输入shape为[-1,784],其中第一维为batch_size,若单个请求只包含一张图片,则batch_size为1,第二维为784维的向量,我们在train这个测试模型的时候将输入展成了一维,因此,单张图片输入也需要变换成28*28 = 784的一维向量;在构建输入时不论shape是什么,我们都要求将输入展开成一维向量进行传入,在该例中,若输入单张图片,则输入是1*784的一维向量。若在训练模型时输入的shape为[-1, 28, 28],则在构建输入时同样需要将输入构建成1*28*28的一维向量,只不过shape需要指定成[-1, 28, 28],若request中指定的shape与模型的shape不一致,则预测请求会报错。

2. 客户端使用Python进行预测

我们以python为例介绍如何对tensorflow服务进行调用,首先需要生成输入的protobuf格式的request文件,我们为python预先生成了pb包,可直接通过下面的命令进行安装:

  1. $ pip install http://eas-data.oss-cn-shanghai.aliyuncs.com/sdk/pai_tf_predict_proto-1.0-py2.py3-none-any.whl

示例代码如下:

  1. #!/usr/bin/env python
  2. # -*- coding: UTF-8 -*-
  3. import json
  4. from urlparse import urlparse
  5. from com.aliyun.api.gateway.sdk import client
  6. from com.aliyun.api.gateway.sdk.http import request
  7. from com.aliyun.api.gateway.sdk.common import constant
  8. from pai_tf_predict_proto import tf_predict_pb2
  9. import cv2
  10. import numpy as np
  11. with open('2.jpg', 'rb') as infile:
  12. buf = infile.read()
  13. # 使用numpy将字节流转换成array
  14. x = np.fromstring(buf, dtype='uint8')
  15. # 将读取到的array进行图片解码获得28 × 28的矩阵
  16. img = cv2.imdecode(x, cv2.IMREAD_UNCHANGED)
  17. # 由于预测服务API需要长度为784的一维向量将矩阵reshape成784
  18. img = np.reshape(img, 784)
  19. def predict(url, app_key, app_secret, request_data):
  20. cli = client.DefaultClient(app_key=app_key, app_secret=app_secret)
  21. body = request_data
  22. url_ele = urlparse(url)
  23. host = 'http://' + url_ele.hostname
  24. path = url_ele.path
  25. req_post = request.Request(host=host, protocol=constant.HTTP, url=path, method="POST", time_out=6000)
  26. req_post.set_body(body)
  27. req_post.set_content_type(constant.CONTENT_TYPE_STREAM)
  28. stat,header, content = cli.execute(req_post)
  29. return stat, dict(header) if header is not None else {}, content
  30. def demo():
  31. # 输入模型信息,点击模型名字就可以获取到了
  32. app_key = 'YOUR_APP_KEY'
  33. app_secret = 'YOUR_APP_SECRET'
  34. url = 'YOUR_APP_URL'
  35. # 构造服务
  36. request = tf_predict_pb2.PredictRequest()
  37. request.signature_name = 'predict_images'
  38. request.inputs['images'].dtype = tf_predict_pb2.DT_FLOAT # images 参数类型
  39. request.inputs['images'].array_shape.dim.extend([1, 784]) # images参数的形状
  40. request.inputs['images'].float_val.extend(img) # 数据
  41. request.inputs['keep_prob'].dtype = tf_predict_pb2.DT_FLOAT # keep_prob 参数的类型
  42. request.inputs['keep_prob'].float_val.extend([0.75]) # 默认填写一个
  43. # 将pb序列化成string进行传输
  44. request_data = request.SerializeToString()
  45. stat, header, content = predict(url, app_key, app_secret, request_data)
  46. if stat != 200:
  47. print 'Http status code: ', stat
  48. print 'Error msg in header: ', header['x-ca-error-message'] if 'x-ca-error-message' in header else ''
  49. print 'Error msg in body: ', content
  50. else:
  51. response = tf_predict_pb2.PredictResponse()
  52. response.ParseFromString(content)
  53. print(response)
  54. if __name__ == '__main__':
  55. demo()

上述示例中所需的mnist输入图片下载地址:http://eas-data.oss-cn-shanghai.aliyuncs.com/data/test_images.zip

上述示例输出为:

  1. outputs {
  2. key: "scores"
  3. value {
  4. dtype: DT_FLOAT
  5. array_shape {
  6. dim: 1
  7. dim: 10
  8. }
  9. float_val: 0.0
  10. float_val: 0.0
  11. float_val: 1.0
  12. float_val: 0.0
  13. float_val: 0.0
  14. float_val: 0.0
  15. float_val: 0.0
  16. float_val: 0.0
  17. float_val: 0.0
  18. float_val: 0.0
  19. }
  20. }

outputs分别10个类别所对应的score,由上可看出,输入的图片为2.jpg,除value[2]之外均为0,则最终预测结果为2,预测正确。

3. 其它语言版本使用方法

除python外其它语言的客户端,用户需要根据proto文件自行生成预测的request代码文件,创建tf.proto,文件内容为:

  1. syntax = "proto3";
  2. option cc_enable_arenas = true;
  3. option java_package = "com.aliyun.openservices.eas.predict.proto";
  4. option java_outer_classname = "PredictProtos";
  5. enum ArrayDataType {
  6. // Not a legal value for DataType. Used to indicate a DataType field
  7. // has not been set.
  8. DT_INVALID = 0;
  9. // Data types that all computation devices are expected to be
  10. // capable to support.
  11. DT_FLOAT = 1;
  12. DT_DOUBLE = 2;
  13. DT_INT32 = 3;
  14. DT_UINT8 = 4;
  15. DT_INT16 = 5;
  16. DT_INT8 = 6;
  17. DT_STRING = 7;
  18. DT_COMPLEX64 = 8; // Single-precision complex
  19. DT_INT64 = 9;
  20. DT_BOOL = 10;
  21. DT_QINT8 = 11; // Quantized int8
  22. DT_QUINT8 = 12; // Quantized uint8
  23. DT_QINT32 = 13; // Quantized int32
  24. DT_BFLOAT16 = 14; // Float32 truncated to 16 bits. Only for cast ops.
  25. DT_QINT16 = 15; // Quantized int16
  26. DT_QUINT16 = 16; // Quantized uint16
  27. DT_UINT16 = 17;
  28. DT_COMPLEX128 = 18; // Double-precision complex
  29. DT_HALF = 19;
  30. DT_RESOURCE = 20;
  31. DT_VARIANT = 21; // Arbitrary C++ data types
  32. }
  33. // Dimensions of an array
  34. message ArrayShape {
  35. repeated int64 dim = 1 [packed = true];
  36. }
  37. // Protocol buffer representing an array
  38. message ArrayProto {
  39. // Data Type.
  40. ArrayDataType dtype = 1;
  41. // Shape of the array.
  42. ArrayShape array_shape = 2;
  43. // DT_FLOAT.
  44. repeated float float_val = 3 [packed = true];
  45. // DT_DOUBLE.
  46. repeated double double_val = 4 [packed = true];
  47. // DT_INT32, DT_INT16, DT_INT8, DT_UINT8.
  48. repeated int32 int_val = 5 [packed = true];
  49. // DT_STRING.
  50. repeated bytes string_val = 6;
  51. // DT_INT64.
  52. repeated int64 int64_val = 7 [packed = true];
  53. // DT_BOOL.
  54. repeated bool bool_val = 8 [packed = true];
  55. }
  56. // PredictRequest specifies which TensorFlow model to run, as well as
  57. // how inputs are mapped to tensors and how outputs are filtered before
  58. // returning to user.
  59. message PredictRequest {
  60. // A named signature to evaluate. If unspecified, the default signature
  61. // will be used
  62. string signature_name = 1;
  63. // Input tensors.
  64. // Names of input tensor are alias names. The mapping from aliases to real
  65. // input tensor names is expected to be stored as named generic signature
  66. // under the key "inputs" in the model export.
  67. // Each alias listed in a generic signature named "inputs" should be provided
  68. // exactly once in order to run the prediction.
  69. map<string, ArrayProto> inputs = 2;
  70. // Output filter.
  71. // Names specified are alias names. The mapping from aliases to real output
  72. // tensor names is expected to be stored as named generic signature under
  73. // the key "outputs" in the model export.
  74. // Only tensors specified here will be run/fetched and returned, with the
  75. // exception that when none is specified, all tensors specified in the
  76. // named signature will be run/fetched and returned.
  77. repeated string output_filter = 3;
  78. }
  79. // Response for PredictRequest on successful run.
  80. message PredictResponse {
  81. // Output tensors.
  82. map<string, ArrayProto> outputs = 1;
  83. }

其中protobuf的详细用法参考:https://developers.google.com/protocol-buffers/ ,PredictRequest定义TF服务的输入格式,PredictResponse定义服务的输出格式。

安装protoc

执行下面的脚本可安装protoc

  1. #/bin/bash
  2. PROTOC_ZIP=protoc-3.3.0-linux-x86_64.zip
  3. curl -OL https://github.com/google/protobuf/releases/download/v3.3.0/$PROTOC_ZIP
  4. unzip -o $PROTOC_ZIP -d ./ bin/protoc
  5. rm -f $PROTOC_ZIP

生成Request代码文件

  • Java版本:
  1. $ bin/protoc --java_out=./ tf.proto

执行完成后在当前目录下生成com/aliyun/openservices/eas/predict/proto/PredictProtos.java,在项目中导入该文件即可。

  • Python版本
  1. $ bin/protoc --python_out=./ tf.proto

执行完成后在当前目前下生成tf_pb2.py,直接import即可。

  • C++版本
  1. $ bin/protoc --cpp_out=./ tf.proto

执行完成后在当前目录下生成tf.pb.cc和tf.pb.h,在code中include tf.pb.h并将tf.pb.cc加入compile列表即可。