SDK for Go
The SDK for Go wraps multiple input and output formats, including TensorFlow, PyTorch, and strings, and supports asynchronous invocation of the queue service. This topic describes each API and provides complete code examples.
For information about the use cases and invocation principles of the SDK, see Service invocation SDKs.
Prerequisites
The Go package management tool automatically downloads the SDK code during compilation. You do not need to install the SDK in advance. To customize the invocation logic, first download the SDK for Go code and then modify it.
To import the SDK, use the following code:
import (
"github.com/pai-eas/eas-golang-sdk/eas"
)Quick start
Select the Request class that corresponds to your model's input data format. The following example shows a minimal, end-to-end invocation using a string request.
package main
import (
"fmt"
"github.com/pai-eas/eas-golang-sdk/eas"
)
func main() {
client := eas.NewPredictClient("182848887922****.cn-shanghai.pai-eas.aliyuncs.com", "my_service")
client.SetToken("YOUR_SERVICE_TOKEN")
client.Init()
resp, err := client.StringPredict("[{}]")
if err != nil {
fmt.Printf("failed to predict: %v\n", err.Error())
} else {
fmt.Printf("%v\n", resp)
}
}API reference
The SDK for Go provides the following API classes, which are divided into three groups by purpose:
Group | Description |
Main client | PredictClient: Configures service information (Endpoint / ServiceName / Token), sends requests, and receives responses. |
Input and output |
|
queue service |
|
PredictClient
The main client class used to configure service information, send requests, and receive prediction results.
Method | Description |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| Initializes the PredictClient object. After you set all parameters, call |
|
|
|
|
|
|
|
|
TFRequest
Builds input data for TensorFlow models.
Method | Description |
|
|
|
|
|
|
TFResponse
Parses output data from TensorFlow models.
Method | Description |
|
|
|
|
TorchRequest
Builds input data for PyTorch models.
Method | Description |
| The constructor of the TorchRequest class. |
|
|
|
|
TorchResponse
Parses output data from PyTorch models.
Method | Description |
|
|
|
|
QueueClient
Interacts with the EAS queue service to produce, consume, and manage data.
Method | Description |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
types.Watcher
Reads pushed data from the subscription channel of the queue service.
Method | Description |
|
|
| Closes a watcher object and its backend data connection. Note A client can have only one active watcher object at a time. You must close the current watcher object before creating a new one. |
Examples
Synchronous inference by format
Choose the code sample based on your service's input and output types.
String
If you deploy a service with a custom processor, you typically invoke it using strings, for example, when calling a PMML model service. The following program shows a complete example.
package main
import (
"fmt"
"github.com/pai-eas/eas-golang-sdk/eas"
)
func main() {
client := eas.NewPredictClient("182848887922****.cn-shanghai.pai-eas.aliyuncs.com", "scorecard_pmml_example")
client.SetToken("YWFlMDYyZDNmNTc3M2I3MzMwYmY0MmYwM2Y2MTYxMTY4NzBkNzdj****")
client.Init()
req := "[{\"fea1\": 1, \"fea2\": 2}]"
for i := 0; i < 100; i++ {
resp, err := client.StringPredict(req)
if err != nil {
fmt.Printf("failed to predict: %v\n", err.Error())
} else {
fmt.Printf("%v\n", resp)
}
}
}TensorFlow
For TensorFlow models, use TFRequest and TFResponse as the input and output data formats, respectively. The following program shows a complete example.
package main
import (
"fmt"
"github.com/pai-eas/eas-golang-sdk/eas"
)
func main() {
client := eas.NewPredictClient("182848887922****.cn-shanghai.pai-eas.aliyuncs.com", "mnist_saved_model_example")
client.SetToken("YTg2ZjE0ZjM4ZmE3OTc0NzYxZDMyNmYzMTJjZTQ1YmU0N2FjMTAy****")
client.Init()
tfreq := eas.TFRequest{}
tfreq.SetSignatureName("predict_images")
tfreq.AddFeedFloat32("images", []int64{1, 784}, make([]float32, 784))
for i := 0; i < 100; i++ {
resp, err := client.TFPredict(tfreq)
if err != nil {
fmt.Printf("failed to predict: %v", err)
} else {
fmt.Printf("%v\n", resp)
}
}
}PyTorch
For PyTorch models, use TorchRequest and TorchResponse as the input and output data formats, respectively. The following program shows a complete example.
package main
import (
"fmt"
"github.com/pai-eas/eas-golang-sdk/eas"
)
func main() {
client := eas.NewPredictClient("182848887922****.cn-shanghai.pai-eas.aliyuncs.com", "pytorch_resnet_example")
client.SetTimeout(500)
client.SetToken("ZjdjZDg1NWVlMWI2NTU5YzJiMmY5ZmE5OTBmYzZkMjI0YjlmYWVl****")
client.Init()
req := eas.TorchRequest{}
req.AddFeedFloat32(0, []int64{1, 3, 224, 224}, make([]float32, 150528))
req.AddFetch(0)
for i := 0; i < 10; i++ {
resp, err := client.TorchPredict(req)
if err != nil {
fmt.Printf("failed to predict: %v", err)
} else {
fmt.Println(resp.GetTensorShape(0), resp.GetFloatVal(0))
}
}
}VPC direct connection
With a VPC direct connection, you can access only services that are deployed in an EAS dedicated resource group, and you must first connect that resource group to the vSwitch you specify. For information about how to purchase an EAS dedicated resource group and connect the network, see Use EAS resource groups and Configure EAS to access public or internal resources. Compared with a standard invocation, this method requires only one additional line of code, client.SetEndpointType(eas.EndpointTypeDirect), which makes it well suited to high-traffic, high-concurrency services. The following code shows an example:
package main
import (
"fmt"
"github.com/pai-eas/eas-golang-sdk/eas"
)
func main() {
// Format of a VPC direct connection endpoint: {uid}.vpc.{region-id}.pai-eas.aliyuncs.com. You can find the endpoint on the Invocation Information tab of the service details page in the EAS console.
client := eas.NewPredictClient("182848887922****.vpc.cn-shanghai.pai-eas.aliyuncs.com", "scorecard_pmml_example")
client.SetToken("YWFlMDYyZDNmNTc3M2I3MzMwYmY0MmYwM2Y2MTYxMTY4NzBkNzdj****")
client.SetEndpointType(eas.EndpointTypeDirect)
client.Init()
req := "[{\"fea1\": 1, \"fea2\": 2}]"
for i := 0; i < 100; i++ {
resp, err := client.StringPredict(req)
if err != nil {
fmt.Printf("failed to predict: %v\n", err.Error())
} else {
fmt.Printf("%v\n", resp)
}
}
}Client connection parameters
You can set the connection parameters of the request client by using the http.Transport property. The following example demonstrates how to configure these settings:
package main
import (
"fmt"
"github.com/pai-eas/eas-golang-sdk/eas"
"net/http"
"time"
)
func main() {
// Format of a VPC direct connection endpoint: {uid}.vpc.{region-id}.pai-eas.aliyuncs.com. You can find the endpoint on the Invocation Information tab of the service details page in the EAS console.
client := eas.NewPredictClient("182848887922****.vpc.cn-shanghai.pai-eas.aliyuncs.com", "network_test")
client.SetToken("MDAwZDQ3NjE3OThhOTI4ODFmMjJiYzE0MDk1NWRkOGI1MmVhMGI0****")
client.SetEndpointType(eas.EndpointTypeDirect)
client.SetHttpTransport(&http.Transport{
MaxConnsPerHost: 300,
TLSHandshakeTimeout: 100 * time.Millisecond,
ResponseHeaderTimeout: 200 * time.Millisecond,
ExpectContinueTimeout: 200 * time.Millisecond,
})
}queue service
You can use QueueClient to send data to a queue service, query data, query the status of the queue service, and subscribe to data pushes from the queue service. In this example, one goroutine sends data to the queue service, while another uses a watcher to subscribe to and receive that data.
When you deploy an asynchronous inference service in EAS, an input queue and an output queue are automatically generated. The addresses are typically in the following formats:
Input queue: <domain>/api/predict/<service_name>
Output queue: <domain>/api/predict/<service_name>/sink
Use <service_name> or <service_name>/sink to build the QueueClient based on your requirements.
const (
QueueEndpoint = "182848887922****.cn-shanghai.pai-eas.aliyuncs.com"
// For example, if the EAS service name is test_qservice, the input queue name is test_qservice, and the output queue name is test_qservice/sink.
QueueName = "test_qservice"
QueueToken = "YmE3NDkyMzdiMzNmMGM3ZmE4ZmNjZDk0M2NiMDA3OTZmNzc1MTUx****"
)
queue, err := NewQueueClient(QueueEndpoint, QueueName, QueueToken)
// truncate all messages in the queue
attrs, err := queue.Attributes()
if index, ok := attrs["stream.lastEntry"]; ok {
idx, _ := strconv.ParseUint(index, 10, 64)
queue.Truncate(context.Background(), idx+1)
}
ctx, cancel := context.WithCancel(context.Background())
// create a goroutine to send messages to the queue
go func() {
i := 0
for {
select {
case <-time.NewTicker(time.Microsecond * 1).C:
_, _, err := queue.Put(context.Background(), []byte(strconv.Itoa(i)), types.Tags{})
if err != nil {
fmt.Printf("Error occured, retry to handle it: %v\n", err)
}
i += 1
case <-ctx.Done():
break
}
}
}()
// create a watcher to watch the messages from the queue
watcher, err := queue.Watch(context.Background(), 0, 5, false, false)
if err != nil {
fmt.Printf("Failed to create a watcher to watch the queue: %v\n", err)
return
}
// read messages from the queue and commit manually
for i := 0; i < 100; i++ {
df := <-watcher.FrameChan()
err := queue.Commit(context.Background(), df.Index.Uint64())
if err != nil {
fmt.Printf("Failed to commit index: %v(%v)\n", df.Index, err)
}
}
// everything is done, close the watcher
watcher.Close()
cancel()Troubleshooting
For the symptoms, causes, and troubleshooting directions of SDK for Go invocation exceptions, including common issues with authentication, routing, connections, and the server, see the "Troubleshoot invocation exceptions" section in Service invocation SDKs.
For a complete list of service status codes, error messages, and recommended actions, see Appendix: Service status codes and common errors.