configure the source node-IoT Platform (Deprecated)(IoT)-阿里云帮助中心

Configure the source node in the data parsing workspace to define the data source for subsequent parsing tasks.

Prerequisites

A data parsing task is created. For more information, see Create a data parsing job.

Background information

For an overview of the data parsing feature, see Usage Notes.
For instructions on using the data parsing workspace, see Data Parsing Workspace Instructions.

Procedure

Access the data parsing workbench.
In the workspace, click the default Source Node on the canvas.

In the right-side configuration panel, configure the Basic Information based on the Data Source Type.

For the IoT Instance Topic type, configure the parameters below to process data from devices using custom or Thing Specification Language communication topics.

Parameter Name	Description	References
Associated Instance	Lists all instances under the current Alibaba Cloud account.	Instance overview
Associated Product	Lists all product names under the selected instance.	Create a product
Associated Device	Select from devices under the chosen product. Only the All Devices option is available.	Create a device
Topic Type	Options: System Topic, custom topic, Thing Specification Language communication topic. See the Topic Type description table below for details.	What is a topic
Topic Name	The communication data of this topic serves as the source for the parsing task. For cloud gateway products' custom topics: Manually input the topic name in the Topic Name field, e.g., `/${productKey}/${deviceName}/user/update`. For other scenarios: Select the topic for analysis from the Topic Name list.	What is a topic

Topic Type description table:

Device	System Topic	Custom Topic	Thing Specification Language Topic
Cloud Gateway Device: MQTT	Not supported.	Supported. For more information, see Add a custom topic category.	Supports property and event types. For more information, see Device reports properties.
Cloud Gateway Device: NB-IoT	Not supported.
Cloud Gateway Device: JT/T 808	Supports device report data topic: `$JT808/${manufacturer}/${deviceModel}/${deviceId}/up`. For more information, see Device data submission.	Not supported.
Cloud Gateway Device: GB/T 32960	Supports device report data topic: `$GB23960/${VIN}/up`. For more information, see Device data submission.	Not supported.
Non-cloud Gateway Device	Supports device shadow publish topic: `/shadow/update/${YourProductKey}/${YourDeviceName}`. For more information, see Device shadow data stream.	Supported. For more information, see Add a custom topic category.

For the API Data Source type, select an API data source to process external data imported through the API.

For more information, see Configure an API data source.

Click Next, select Topic Format, and set up the format parsing.

JSON, ProtoBuf, Base64(to_JSON)

Based on the selected topic format, configure the sample data.

Note

If data has been reported in the last 7 days, click Pull online data to automatically populate the sample data.

Topic format	Configure sample
JSON	Enter sample data in the Sample Data field. The content size must not exceed 16 KB.
ProtoBuf	Click Upload .desc file to upload the `.desc` file necessary for parsing ProtoBuf format data. Refer to Appendix: Generating .desc file for the .desc file generation method. After selecting the message type, click Upload binary data file to input the sample data.
Base64(to_JSON)	Click Upload Base64 data file to upload the sample data.

After configuring the sample data, click Validate parsing:
- If parsing succeeds, the data structure appears in the Parsing Preview.
- If parsing fails, revise the sample data as prompted and try again.
Select the Pass-through option.
- No (default).
- Yes: Only parsed data passes through the topic. Custom data storage and SQL offline analysis are not supported.
Click Save.

The parsed fields, including names and data types, are listed under Data Structure below the canvas.

Note
If Pass-through is set to Yes, any nodes following the Source Node will be removed, and a Target Node will be automatically added and connected. In this case, no other nodes can be inserted between the Source Node and the Target Node.

Raw data

Select Pass-through.
- No (default).
- Yes: Only parsed data is transmitted through the topic. Custom data storage and SQL offline analysis are not supported.
Click Save.

A Custom node is automatically created and linked to the Source Node on the canvas.

You cannot insert nodes between the Source Node and the Custom node. Any nodes previously connected after the Source Node are rerouted to the Custom node.

Note
If Pass-through is set to Yes, any nodes following the Source Node will be removed, and a Target Node will be automatically added and connected after the Custom node. In this case, no nodes can be placed between the Custom node and the Target Node.

Configure the Custom node script to process raw data.

Click the Custom node on the canvas.

In the custom script panel, select the scripting language and enter your script under Edit script.

Supported Scripting Languages	Functions to Define	Example Code
JavaScript (ECMAScript 5)	executeScript()	JavaScript script example
Python 2.7	execute_script()	Python script example
PHP 7.2	executeScript()	PHP script example

In the Analog Input tab, enter the data reported by the simulated device.
Click Execute.

On success, the processed data appears in the Execution Result tab. To view the script execution log, click the Execution Log tab.

To save the source node configuration, click Save in the upper-right corner of the Data Analysis Workbench.

What to do next

After the source node is configured, set up additional processing nodes for data parsing, or configure the target node to complete the parsing task.

Appendix: Generating .desc Files

Important

Before configuring sample data in ProtoBuf format, you must upload the corresponding .desc file for data parsing.
A basic understanding of Protocol Buffers is required to use this feature.

Download and install Protocol Buffers.

Run the following command to generate the .desc file:

protoc -I=/filepath1/ --descriptor_set_out=/filepath2/proto.desc /filepath3/proto.proto

Command parameters:

Parameter	Description
`-I`	Abbreviation for `--proto_path`. Specifies the directory to search for `import` file dependencies when compiling the `.proto` file. Replace `/filepath1/` with the dependency directory. If there are no dependencies, set it to any local directory.
`--descriptor_set_out`	Specifies the output directory for the generated `.desc` file. Replace `/filepath2/proto.desc` with the desired output path and file name.
`/filepath3/proto.proto`	The path and name of the source `.proto` file. To generate a `.desc` file from multiple files, specify multiple paths separated by commas (,). Replace this with the path and name of your source `proto` file.