Construct a request for a TensorFlow service

EAS has deployed a public test service in a VPC environment in the China (Shanghai) region. The service name is mnist_saved_model_example, and the access token is empty. You can access the service at http://pai-eas-vpc.cn-shanghai.aliyuncs.com/api/predict/mnist_saved_model_example. The following steps describe how to access this service:

1. Obtain the model information.

   Send a GET request to obtain the model information, which includes signature_name, name, type, and shape. The following code shows the response.

   $curl http://pai-eas-vpc.cn-shanghai.aliyuncs.com/api/predict/mnist_saved_model_example | python -mjson.tool
   {
       "inputs": [
           {
               "name": "images",
               "shape": [
                   -1,
                   784
               ],
               "type": "DT_FLOAT"
           }
       ],
       "outputs": [
           {
               "name": "scores",
               "shape": [
                   -1,
                   10
               ],
               "type": "DT_FLOAT"
           }
       ],
       "signature_name": "predict_images"
   }

   This is a classification model that uses the Mixed National Institute of Standards and Technology (MNIST) dataset (Download the MNIST dataset). The input data type is DT_FLOAT. For example, the shape is [-1,784]. The first dimension represents the batch_size. If a single request contains only one image, the batch_size is 1. The second dimension represents a 784-dimensional vector. Because the input for this model was flattened into a one-dimensional vector during training, a single image input must also be transformed into a 28 × 28 = 784 one-dimensional vector. You must flatten the input when you construct it, regardless of the shape value. In this example, if you input a single image, the input is a 1 × 784 one-dimensional vector. If the input shape during model training was [-1, 28, 28], you must construct the input with a shape of 1 × 28 × 28. If the shape specified in the service request does not match the model's shape, the prediction request returns an error.

2. Install protobuf and call the service. The following example shows how to call a TensorFlow service using Python 2.

   EAS provides a pre-generated protobuf package for Python. You can run the following command to install it.

   $ pip install http://eas-data.oss-cn-shanghai.aliyuncs.com/sdk/pai_tf_predict_proto-1.0-py2.py3-none-any.whl

   The following sample code shows how to call the service to make a prediction using Python 2.

   #!/usr/bin/env python
   # -*- coding: UTF-8 -*-
   import json
   from urlparse import urlparse
   from com.aliyun.api.gateway.sdk import client
   from com.aliyun.api.gateway.sdk.http import request
   from com.aliyun.api.gateway.sdk.common import constant
   from pai_tf_predict_proto import tf_predict_pb2
   import cv2
   import numpy as np
   with open('2.jpg', 'rb') as infile:
       buf = infile.read()
       # Use NumPy to convert the byte stream to an array.
       x = np.fromstring(buf, dtype='uint8')
       # Decode the read array to get a 28 × 28 matrix.
       img = cv2.imdecode(x, cv2.IMREAD_UNCHANGED)
       # The prediction service API requires a one-dimensional vector of length 784, so reshape the matrix to 784.
       img = np.reshape(img, 784)
   def predict(url, app_key, app_secret, request_data):
       cli = client.DefaultClient(app_key=app_key, app_secret=app_secret)
       body = request_data
       url_ele = urlparse(url)
       host = 'http://' + url_ele.hostname
       path = url_ele.path
       req_post = request.Request(host=host, protocol=constant.HTTP, url=path, method="POST", time_out=6000)
       req_post.set_body(body)
       req_post.set_content_type(constant.CONTENT_TYPE_STREAM)
       stat,header, content = cli.execute(req_post)
       return stat, dict(header) if header is not None else {}, content
   def demo():
       # Enter the model information. Click the model name to obtain it.
       app_key = 'YOUR_APP_KEY'
       app_secret = 'YOUR_APP_SECRET'
       url = 'YOUR_APP_URL'
       # Construct the service.
       request = tf_predict_pb2.PredictRequest()
       request.signature_name = 'predict_images'
       request.inputs['images'].dtype = tf_predict_pb2.DT_FLOAT  # The type of the images parameter.
       request.inputs['images'].array_shape.dim.extend([1, 784])  # The shape of the images parameter.
       request.inputs['images'].float_val.extend(img)  # Data.
       request.inputs['keep_prob'].dtype = tf_predict_pb2.DT_FLOAT  # The type of the keep_prob parameter.
       request.inputs['keep_prob'].float_val.extend([0.75])  # Enter a default value.
       # Serialize the protobuf to a string for transmission.
       request_data = request.SerializeToString()
       stat, header, content = predict(url, app_key, app_secret, request_data)
       if stat != 200:
           print 'Http status code: ', stat
           print 'Error msg in header: ', header['x-ca-error-message'] if 'x-ca-error-message' in header else ''
           print 'Error msg in body: ', content
       else:
           response = tf_predict_pb2.PredictResponse()
           response.ParseFromString(content)
           print(response)
   if __name__ == '__main__':
       demo()

   The following output is returned.

   outputs {
     key: "scores"
     value {
       dtype: DT_FLOAT
       array_shape {
         dim: 1
         dim: 10
       }
       float_val: 0.0
       float_val: 0.0
       float_val: 1.0
       float_val: 0.0
       float_val: 0.0
       float_val: 0.0
       float_val: 0.0
       float_val: 0.0
       float_val: 0.0
       float_val: 0.0
     }
   }

   The outputs field contains the scores for the 10 categories. When the input image is 2.jpg, all values are 0 except for value[2]. Therefore, the final prediction result is 2, which is correct.

To call the service from clients in languages other than Python, you must generate the request code from the .proto file. The following steps describe how to do this:

1. Create a `.proto` file. For example, create a file named tf.proto with the following content.

   syntax = "proto3";
   option cc_enable_arenas = true;
   option java_package = "com.aliyun.openservices.eas.predict.proto";
   option java_outer_classname = "PredictProtos";
   enum ArrayDataType {
     // Not a legal value for DataType. Used to indicate a DataType field
     // has not been set.
     DT_INVALID = 0;
     // Data types that all computation devices are expected to be
     // capable to support.
     DT_FLOAT = 1;
     DT_DOUBLE = 2;
     DT_INT32 = 3;
     DT_UINT8 = 4;
     DT_INT16 = 5;
     DT_INT8 = 6;
     DT_STRING = 7;
     DT_COMPLEX64 = 8;  // Single-precision complex.
     DT_INT64 = 9;
     DT_BOOL = 10;
     DT_QINT8 = 11;     // Quantized int8.
     DT_QUINT8 = 12;    // Quantized uint8.
     DT_QINT32 = 13;    // Quantized int32.
     DT_BFLOAT16 = 14;  // Float32 truncated to 16 bits.  Only for cast ops.
     DT_QINT16 = 15;    // Quantized int16.
     DT_QUINT16 = 16;   // Quantized uint16.
     DT_UINT16 = 17;
     DT_COMPLEX128 = 18;  // Double-precision complex.
     DT_HALF = 19;
     DT_RESOURCE = 20;
     DT_VARIANT = 21;  // Arbitrary C++ data types.
   }
   // Dimensions of an array.
   message ArrayShape {
     repeated int64 dim = 1 [packed = true];
   }
   // Protocol buffer representing an array.
   message ArrayProto {
     // Data Type.
     ArrayDataType dtype = 1;
     // Shape of the array.
     ArrayShape array_shape = 2;
     // DT_FLOAT.
     repeated float float_val = 3 [packed = true];
     // DT_DOUBLE.
     repeated double double_val = 4 [packed = true];
     // DT_INT32, DT_INT16, DT_INT8, DT_UINT8.
     repeated int32 int_val = 5 [packed = true];
     // DT_STRING.
     repeated bytes string_val = 6;
     // DT_INT64.
     repeated int64 int64_val = 7 [packed = true];
     // DT_BOOL.
     repeated bool bool_val = 8 [packed = true];
   }
   // PredictRequest specifies which TensorFlow model to run, along with
   // how inputs are mapped to tensors and how outputs are filtered before
   // returning to user.
   message PredictRequest {
     // A named signature to evaluate. If unspecified, the default signature
     // will be used.
     string signature_name = 1;
     // Input tensors.
     // Names of input tensor are alias names. The mapping from aliases to real
     // input tensor names is expected to be stored as named generic signature
     // under the key "inputs" in the model export.
     // Each alias listed in a generic signature named "inputs" should be provided
     // exactly once to run the prediction.
     map<string, ArrayProto> inputs = 2;
     // Output filter.
     // Names specified are alias names. The mapping from aliases to real output
     // tensor names is expected to be stored as named generic signature under
     // the key "outputs" in the model export.
     // Only tensors specified here will be run/fetched and returned, with the
     // exception that when none is specified, all tensors specified in the
     // named signature will be run/fetched and returned.
     repeated string output_filter = 3;
   }
   // Response for PredictRequest on successful run.
   message PredictResponse {
     // Output tensors.
     map<string, ArrayProto> outputs = 1;
   }

   In the file, PredictRequest defines the input format for the TensorFlow service, and PredictResponse defines the output format of the service. For more information, see the protobuf documentation.

2. Install protoc.

   #/bin/bash
   PROTOC_ZIP=protoc-3.3.0-linux-x86_64.zip
   curl -OL https://github.com/google/protobuf/releases/download/v3.3.0/$PROTOC_ZIP
   unzip -o $PROTOC_ZIP -d ./ bin/protoc
   rm -f $PROTOC_ZIP

3. Generate the request code file.

   • Java

     $ bin/protoc --java_out=./ tf.proto

     After you run the command, the com/aliyun/openservices/eas/predict/proto/PredictProtos.java file is generated in the current directory. You can then import this file into your project.

   • Python

     $ bin/protoc --python_out=./ tf.proto

     After you run the command, the tf_pb2.py file is generated in the current directory. Use the import command to import the file.

   • C++

     $ bin/protoc --cpp_out=./ tf.proto

     After you run the command, the tf.pb.cc and tf.pb.h files are generated in the current directory. In your code, include tf.pb.h and add tf.pb.cc to your compile list.