Machine learning edge inference-Link IoT Edge(IoT Edge)-阿里云帮助中心

IoT Edge enables you to deploy machine learning (ML) models from the cloud to the edge as edge applications. You can then perform ML inference at the edge. This feature is ideal for services that run on the edge, require real-time processing, and handle large volumes of data, such as computer vision.

Prerequisites

Install and start the IoT Edge software package on a Raspberry Pi 4B or Raspberry Pi 3B/B+. For more information, see Set up a Raspberry Pi environment.
Connect a camera to your Raspberry Pi. For more information, see the Raspberry Pi camera setup guide.

Background information

You can train your inference models on Alibaba Cloud Platform for AI or another platform. Host the trained models and related code on Alibaba Cloud services, such as Function Compute (FC), Object Storage Service (OSS), or in a container image. Then, in an IoT Edge instance, deploy the models to a gateway as an edge application. After the gateway performs inference using the local model, it uploads the results to IoT Platform.

Link IoT Edge ML Inference工作原理

This topic demonstrates how to use machine learning inference on IoT Edge by deploying a deep learning object detection model based on TensorFlow Lite to a Raspberry Pi 4B.

Step 1: Configure the Raspberry Pi and install the edge inference runtime

Use a Secure Shell (SSH) tool to connect to the Raspberry Pi terminal and run the commands in the following steps.

Open the Raspberry Pi configuration tool and enable the camera.
1. Run the following command to open the Raspberry Pi configuration tool.
```
sudo raspi-config
```
2. Follow the prompts to select Interfacing Options and click Select.
3. Select Camera and click Select to enable the camera.
4. Follow the prompts to click Finish and restart the Raspberry Pi.
5. Run the following command in the Raspberry Pi shell terminal to verify that the camera is working correctly using the built-in raspistill tool.
  The command outputs camera information to the terminal and saves an image file named cam.jpg to the current directory.
```
raspistill -v -o cam.jpg
```
Install the TensorFlow Lite interpreter.
1. Run the following command in the Raspberry Pi shell to download the installation package for the TensorFlow Lite interpreter.
```
curl -O https://iotedge-web.oss-cn-shanghai.aliyuncs.com/public/LeMLInterpreter/ARMv7hf/linkedge_ml_tflite_raspi4_cp3x_armv7hf_installer.tar.gz
```
2. Unzip and install the package.
```
tar xzvf linkedge_ml_tflite_raspi4_cp37_armv7hf_installer.tar.gz 
cd linkedge_ml_tflite_raspi4_cp37_armv7hf_installer/
./le_ml_installer.sh
```
  If the system displays the following output, the TensorFlow Lite interpreter is installed.

Step 2: Publish the detector device driver in the cloud

Download the detector device driver package object_detector_driver.zip.
Log on to the IoT Edge console.
In the navigation pane on the left, click Driver Management.

Add a custom driver. For more information, see Publish in the cloud.

Configure the parameters as described in the following table.

Table 1. Driver information parameters
Parameter	Description
Driver Name	Enter a custom name for the driver, such as obj_detector_driver.
Protocol Type	Select Custom.
Language	Select Python 3.5.
Is the driver built-in?	Select No.
Driver File	Click Upload File and upload the object_detector_driver.zip file that you downloaded.
Driver Version	Set the value to v1.0.0.
Compatible Edge Version	Select 2.4.0 and later.
Driver Version Description	Enter a description for the driver and version. This parameter can be left empty.

You do not need to configure the other parameters.

Step 3: Assign the detector device driver to the edge instance

In the navigation pane on the left, click Edge Instances. Find the edge instance that you created in the "Prerequisites" section and click View in the Actions column.
On the Instance Details page, on the Devices and Drivers tab, click the + icon to the right of All Drivers.
In the Assign Driver panel, select Custom Driver, find the obj_detector_driver driver, and click Assign in the Actions column. Then, click OK.
Click Assign Sub-device. In the Assign Sub-device panel that appears, click Add Sub-device to add a sub-device for the obj_detector_driver driver to the edge instance.

In the Add Device dialog box, click Create Product to create a detector product.

In the Create Product dialog box, set the parameters and click OK.

Table 2. Parameter descriptions
Parameter	Description
Product Name	Set the value to detector.
Category	Select a category to define a Thing Specification Language (TSL) model for the product. Select Custom Category.
Access gateway protocol	Select Custom.

In the Add Device dialog box, the product that you created is automatically selected for the Product parameter. Click Go to Configure next to the Product parameter to add custom features. For more information, see Add a TSL model for a single product.
Configure the two properties as shown in the following figures.
- Object category property:
- Detection score property:
Return to the Add Device dialog box on the Instance Details page in the IoT Edge console, and add a device for the detector product.
Assign the new tflite_detector device to the edge instance.

Step 4: Create the edge inference function

Download the edge inference function code package object_detector_app.zip.
Log on to the Function Compute console.
If you have not activated this service, read the terms and select I have read and agree to the terms. Then, click Activate Now.
(Optional) In the navigation pane on the left, click Services & Functions. On the Services & Functions page, in the Service List section, click Create Service to create a service.
The Service Name parameter is required. Set this parameter to EdgeFC. You can configure the other parameters as needed or use their default settings.
Note
- If this is your first time creating a service in Function Compute, follow the configuration wizard.
- If you have already created the EdgeFC service for other scenarios or miniprogram examples, you do not need to create it again.
After the service is created, on the Services & Functions page, find the EdgeFC service and click Create Function.
On the Create Function page, click Configure and Deploy in the Event Function section.

Configure the basic management parameters for the edge inference function.

Parameter	Description
Function Type	Keep the default option.
Service	Select the EdgeFC service that you created.
Function Name	Set the value to object_detector_app.
Runtime	Set the runtime for the function and select a method to upload code. For this example, select Python 3. To the right of Upload Code, select Upload Code Package. Click Upload Code and upload the object_detector_app.zip package that you downloaded in Step 1.
Handler	Use the default value: index.handler.

You can configure the other parameters as needed or use their default settings. For more information, see Function Compute.

After you confirm the function information, click Create.

After the function is created, you are redirected to the function details page. On the Code tab, in the Code Execution Management section, select the Edit Online radio button to view the source code.

Step 5: Assign the function to the edge instance

Log on to the IoT Edge console.
In the navigation pane on the left, click Application Management.

Use the function that you created in Step 4 to create a Function Compute edge application. For more information, see Function Compute applications.

The following table describes the application information parameters.

Parameter	Description
Application Name	Enter a name for your application, such as le_object_detector.
Application Type	Select Function Compute.
Region	Select the region where your service was created.
Service	Select the EdgeFC service.
Function	Select the object_detector_app function.
Authorization	Select AliyunIOTAccessingFCRole.
Application Version	Set the application version. The version number must be unique for this application. You cannot have two identical version numbers for the same application.

In the Function Configuration section, set Enable Default Configurations to Yes. The other parameters can be left at their default settings.

In the navigation pane on the left, click Edge Instances.
Find the edge instance that you created in the "Prerequisites" section and click View.
On the Instance Details page, go to the Edge Applications tab and click Assign Application.
Assign the le_object_detector edge inference application to the edge instance and click Close.

Step 6: Deploy the edge instance

On the Instance Details page, click Deploy in the upper-right corner. In the dialog box that appears, click OK to deploy resources, such as sub-devices and functions, to the edge.
After the deployment is complete, go to the Devices & Drivers tab. The status of the tflite_detector device changes to Online.
Click View to the right of the tflite_detector device. You are redirected to the Device Details page.
On the Device Details page, on the TSL Data > Running Status tab, you can view the edge inference results.
Place a common object or stand in front of the Raspberry Pi camera. The le_object_detector Function Compute application detects the object or face and reports the results to IoT Platform.
You have now deployed a machine learning model as an edge application and performed inference on the edge.