Elastic Algorithm Service (EAS) includes a built-in Caffe processor that lets you deploy Caffe models as online services based on a universal processor. Both request input and response output must use Protocol Buffers format.
This page walks through calling a deployed Caffe service using Python, Java, and C++.
Jump to your language: Python · Java or C++
How it works
Read the Caffe model file to determine the input layer name and input shape.
Install the Protocol Buffers client package for your language.
Build a
PredictRequestobject, flatten the input data into a one-dimensional array, and serialize it.Send an HTTP POST request to the service endpoint and parse the
PredictResponse.
Prerequisites
Before you begin, ensure that you have:
A Caffe model deployed as a service on EAS
The service endpoint URL and, if required, an access token
Python installed (for the Python path), or a Java/C++ build environment
Step 1: Read the model file
Open the Caffe model file to identify the input layer and its shape. A typical CaffeNet model file looks like this:
name: "CaffeNet"
layer {
name: "data"
type: "Input"
top: "data"
input_param {
shape {
dim: 10
dim: 3
dim: 227
dim: 227
}
}
}
....
layer {
name: "prob"
type: "Softmax"
bottom: "fc8"
top: "prob"
}The layer with type: "Input" defines the model input — typically the first layer. The last layer defines the output. In this example:
Input layer name:
dataInput shape:
[10, 3, 227, 227]— the first dimension isbatch_sizeSingle-image vector: 1 × 3 × 227 × 227 (flatten to one dimension regardless of shape)
The input shape in your request must match the model's declared input shape exactly. A mismatch causes the request to fail.
Call with Python
EAS provides a Protocol Buffers package for Python that includes the PredictRequest and PredictResponse message types.
Install the package
pip install http://eas-data.oss-cn-shanghai.aliyuncs.com/pai_caffe_predict_proto-1.0-py2.py3-none-any.whlThis installs pai_caffe_predict_proto, which provides caffe_predict_pb2 — the Python bindings for building and parsing Caffe prediction messages.
Send a prediction request
The following example calls the public test service caffenet_serving_example, which is deployed in the China (Shanghai) region and accessible to all users in Virtual Private Clouds (VPCs) in that region without an access token.
#! /usr/bin/env python
# -*- coding: UTF-8 -*-
import requests
from pai_caffe_predict_proto import caffe_predict_pb2
# Build the request
request = caffe_predict_pb2.PredictRequest()
request.input_name.extend(['data'])
array_proto = caffe_predict_pb2.ArrayProto()
array_proto.shape.dim.extend([1, 3, 227, 227]) # batch_size=1, 3 channels, 227x227
array_proto.data.extend([1.0] * 3 * 227 * 227) # Flatten to a 1D array
request.input_data.extend([array_proto])
# Serialize to Protocol Buffers format
data = request.SerializeToString()
# Send the request — must be called from a VPC in the China (Shanghai) region
url = 'http://pai-eas-vpc.cn-shanghai.aliyuncs.com/api/predict/caffenet_serving_example'
s = requests.Session()
resp = s.post(url, data=data)
# Parse the response
if resp.status_code != 200:
print(resp.content)
else:
response = caffe_predict_pb2.PredictResponse()
response.ParseFromString(resp.content)
print(response)The endpoint http://pai-eas-vpc.cn-shanghai.aliyuncs.com/api/predict/caffenet_serving_example is only reachable from a VPC in the China (Shanghai) region. Replace this URL with your own service endpoint when integrating your model.Call with Java or C++
If your client is not Python, generate the request code from the .proto definition file and then build PredictRequest in your language.
Step 1: Create the .proto file
Create a file named caffe.proto with the following content:
syntax = "proto2";
package caffe.eas;
option java_package = "com.aliyun.openservices.eas.predict.proto";
option java_outer_classname = "CaffePredictProtos";
message ArrayShape {
repeated int64 dim = 1 [packed = true];
}
message ArrayProto {
optional ArrayShape shape = 1;
repeated float data = 2 [packed = true];
}
message PredictRequest {
repeated string input_name = 1;
repeated ArrayProto input_data = 2;
repeated string output_filter = 3;
}
message PredictResponse {
repeated string output_name = 1;
repeated ArrayProto output_data = 2;
}PredictRequest defines the input format; PredictResponse defines the output format. For more about Protocol Buffers syntax, see Protocol Buffers.
Step 2: Install protoc
#!/bin/bash
PROTOC_ZIP=protoc-3.3.0-linux-x86_64.zip
curl -OL https://github.com/google/protobuf/releases/download/v3.3.0/$PROTOC_ZIP
unzip -o $PROTOC_ZIP -d ./ bin/protoc
rm -f $PROTOC_ZIPStep 3: Generate language-specific code
Run the appropriate command for your language.
Java
bin/protoc --java_out=./ caffe.protoThis generates com/aliyun/openservices/eas/predict/proto/CaffePredictProtos.java. Import the file into your project.
Python
bin/protoc --python_out=./ caffe.protoThis generates caffe_pb2.py. Use import to add it to your project.
C++
bin/protoc --cpp_out=./ caffe.protoThis generates caffe.pb.cc and caffe.pb.h. Add #include "caffe.pb.h" to your code and add caffe.pb.cc to your compile list.