Deploy a model using the EAS Python SDK

Background information

Prerequisites

• An OSS bucket is required to store the model files and configuration files. For more information, see Create a bucket.

• Prepare an environment to run the Python SDK using one of the following methods.

  • (Recommended) Use the Notebook environment of a DSW instance. When you create the DSW instance, set the runtime image to `tensorflow:2.3-cpu-py36-ubuntu18.04` and the instance type to `ecs.c6.large`. For more information, see Create and manage DSW instances.

  • Use a local Python environment. We recommend that you use Python 3.6 or later and JupyterLab as the development environment. You must also download the EASCMD client and complete identity authentication. For more information, see Download and authenticate the client.

Step 1: Prepare the model

1. Install the Python SDK. You can use the SDK to call EAS APIs to deploy the model service and make predictions.

   1. Go to the Notebook page.

      • If you are using a DSW instance, click Open in the Actions column of the target instance to open the DSW instance. On the Notebook tab, in the Quick Start section, click Python 3 under Notebook to open the Notebook editing page. For more information, see Create and manage DSW instances.

      • If you use a local Python environment, open the JupyterLab editing page after you install Python and JupyterLab.

   2. In the Notebook, run the following code to install the Python SDK. The installation takes about 15 minutes.

      ! pip install tensorflow tensorflow_datasets
      ! pip install opencv-python 
      ! pip install eas-prediction alibabacloud_eas20210701==1.1.2 --upgrade

      Note

      You can ignore any ERROR and WARNING messages in the output.

   3. Run the following command to check whether the installation is successful.

      pip list

      If `tensorflow`, `tensorflow_datasets`, `opencv-python`, and `eas-prediction` are included in the output, the Python packages were installed successfully.

2. Train a model and generate a model file.

   In the Notebook, run the following code to train a TensorFlow model based on a basic TensorFlow example. The trained model files are saved to the `eas_demo_output3` folder in the current directory. The training takes about 20 minutes.

   import tensorflow as tf
   import tensorflow_datasets as tfds
   
   (ds_train, ds_test), ds_info = tfds.load(
       'mnist',
       split=['train', 'test'],
       data_dir='./cached_datasets',
       shuffle_files=True,
       as_supervised=True,
       with_info=True,
   )
   
   def normalize_img(image, label):
     """Normalizes images: `uint8` -> `float32`."""
     return tf.cast(image, tf.float32) / 255., label
   
   ds_train = ds_train.map(
       normalize_img)
   ds_train = ds_train.cache()
   ds_train = ds_train.shuffle(ds_info.splits['train'].num_examples)
   ds_train = ds_train.batch(128)
   ds_train = ds_train.prefetch(10)
   
   ds_test = ds_test.map(normalize_img)
   ds_test = ds_test.batch(128)
   ds_test = ds_test.cache()
   ds_test = ds_test.prefetch(10)
   model = tf.keras.models.Sequential([
     tf.keras.layers.Flatten(input_shape=(28, 28)),
     tf.keras.layers.Dense(128, activation='relu'),
     tf.keras.layers.Dense(10)
   ])
   model.compile(
       optimizer=tf.keras.optimizers.Adam(0.001),
       loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
       metrics=[tf.keras.metrics.SparseCategoricalAccuracy()],
   )
   
   model.fit(
       ds_train,
       epochs=6,
       validation_data=ds_test,
   )
   model.save('./eas_demo_output3')

   The model files generated in the eas_demo_output3 folder are shown in the following figure.

Step 2: Deploy the model to EAS

1. In the Notebook, run the following code to create an EAS client object. This object is used to call EASCMD client commands to create the model service.

   from alibabacloud_eas20210701.client import Client as eas20210701Client
   from alibabacloud_tea_openapi import models as open_api_models
   from alibabacloud_eas20210701 import models as eas_20210701_models
   from alibabacloud_tea_util import models as util_models
   from alibabacloud_tea_util.client import Client as UtilClient
   from alibabacloud_eas20210701.models import (ListServicesRequest, CreateServiceRequest)
   
   access_key_id = "<AccessKey>"
   access_key_secret = "<AccessKeySecret>"
   
   import os.path
   config_path="/mnt/data/pai.config"
   if os.path.isfile(config_path):
       with open(config_path) as f:
           access_key_id = f.readline().strip('\n')
           access_key_secret = f.readline().strip('\n')
   
   config = open_api_models.Config(
               access_key_id=access_key_id,
               access_key_secret=access_key_secret
           )
   
   # The endpoint to access.
   region = "cn-shanghai"
   config.endpoint = f'pai-eas.{region}.aliyuncs.com'
   eas_client = eas20210701Client(config)

   The key parameters are described as follows.

   Parameter	Description
   region	The ID of the region where you want to deploy the service. For more information, see Regions and zones. In this example, the value is cn-shanghai. Note The region must be the same as the region of the OSS Bucket. Otherwise, the EAS service will fail to read the model file during creation.
   access_key_id	Replace these with your AccessKey ID and AccessKey secret. You can use an Alibaba Cloud account or a Resource Access Management (RAM) user to create the EAS client object. For more information about how to obtain an AccessKey, see Obtain an AccessKey pair. This topic uses an Alibaba Cloud account as an example. If you use a RAM user, grant the RAM user permissions to perform operations on EAS. For more information, see Cloud service dependencies and authorization: EAS.
   access_key_secret

2. Prepare the model files.

   Upload all files from the `eas_demo_output3` folder to your OSS bucket. Use the folder structure shown in the following figure. For more information about how to upload files, see Upload objects.

   

3. In the Notebook, you can run the following code to create an EAS service.

   import json
   resource_config = { "instance": 1,
                      	 "memory": 7000,
                        "cpu": 4}
   model_path = "oss://examplebucket/dsw/eas_demo_output3/"                     
   service_config = {"name": "service_from_dsw",
                    "model_path": model_path,
                    "processor": "tensorflow_cpu_2.4",
                    "metadata": resource_config}
   print(json.dumps(service_config))
   
   service1 = eas_client.create_service(CreateServiceRequest(body=json.dumps(service_config))).body
   print(service1)

   The key parameters are described as follows. It takes about 5 minutes to create the service. Once created, the service appears on the PAI-EAS Online Model Services page in the console.

   Parameter	Description
   model_path	Replace this with the path to your model files in the OSS bucket.
   service_config.name	The custom name of the model service. The name must meet the following requirements: It can contain only digits, lowercase letters, and underscores (_). It must start with a letter. The service name must be unique within the same region. In this example, the value is service_from_dsw.

4. In the Notebook, run the following code to check the EAS service status.

   Note

   The cluster_id must match the region configured in the previous step.

   service2 = eas_client.describe_service(cluster_id='cn-shanghai', service_name=service1.service_name).body
   print(service2.status)

   If the output is `Running`, the service was created successfully.

Step 3: Simulate prediction

1. Create a test sample.

   In the Notebook, run the following code to randomly select a test sample of a handwritten digit from the MNIST dataset and display its grayscale image. Each time you run the code, a different digit image is generated. You can then check the digit in the image to evaluate the accuracy of the prediction. You can also create your own test data to simulate predictions.

   # import tensorflow.compat.v2 as tf
   import tensorflow_datasets as tfds
   import matplotlib.pyplot as plt
   import numpy as np
   
   # Construct a tf.data.Dataset
   ds = tfds.load('mnist', split='train', data_dir='./cached_datasets',  shuffle_files=False)
   
   # Build your input pipeline
   ds = ds.shuffle(1024).take(3)
   
   target = []
   for example in ds.take(1):
       image, label = example['image'], example['label']
       print(label)
       target = np.reshape(image, 784)
       plt.imshow(tf.squeeze(image))
       plt.show()

   The following figure shows the system output. Your actual output may vary.

2. Use `eas_prediction` to call the deployed service for online prediction.

   1. In the Notebook, run the following code to query the details of the model service, including the AccessToken and Endpoint. This information is used to call the service for testing.

      service3 = eas_client.describe_service(cluster_id='cn-shanghai', service_name='service_from_dsw').body
      print(service3)

      The key parameters are described as follows.

      Parameter	Description
      cluster_id	The name of the region where the service is located. In this example, the value is cn-shanghai.
      service_name	Replace this with the name of the model service deployed in the previous step. In this example, the value is service_from_dsw.

      The following figure shows the system output. Your actual output may vary.

      

   2. In the Notebook, run the following code to construct a `PredictClient` object to call the service.

      Use the grayscale image from the previous step as input data. Use the `access_token` and `Endpoint` retrieved in the preceding step to call the service and output the prediction result.

      from eas_prediction import PredictClient, TFRequest
      import urllib
      client = PredictClient(urllib.parse.urlsplit(service3.internet_endpoint).hostname,
                             service3.service_name)
      client.set_token(service3.access_token)
      client.init()
      
      req = TFRequest('serving_default') # The signature_name parameter is serving_default.
      req.add_feed('flatten_input', [1, 28, 28], TFRequest.DT_FLOAT, target)
      
      resp = client.predict(req)
      print((resp.response.outputs).keys)

      The following figure shows the prediction result. Your actual prediction result may vary.

      Description of the prediction result:

      The `float_val` values correspond to the digits 0 to 9, from top to bottom. The row with the highest `float_val` indicates the predicted digit. For example, in the figure above, the `float_val` in the first row is the highest. This means that the predicted digit is 0, which matches the digit in the test sample image.