Configure a parser

How it works

IoT Platform lets you configure a data forwarding parser to forward messages from a data source (device messages) to a data destination (another topic, an AMQP server, or another Alibaba Cloud service). This enables communication between devices, between devices and servers, and between devices and Alibaba Cloud services.

Limits

• You can create a maximum of 1,000 parsers for each instance.

• A parser can be associated with only one data source.

• A parser can be associated with a maximum of 10 data destinations.

• A parser can be associated with only one error data destination.

• The maximum size of a parsing script is 120 KB.

• The maximum number of times a forwarding function can be executed in a loop within a parsing script is 100.

Step 1: Create a parser

1. Log on to the IoT Platform console.

2. On the Overview page, find the instance that you want to manage and click the instance ID or instance name.

3. In the navigation pane on the left, choose Message Forwarding > Data Forwarding.

4. On the Data Forwarding page, click Try New Version in the upper-right corner to go to the new version of the page.
   Note If you have already performed this operation, the console automatically redirects you to the new version page when you visit the Data Forwarding page.

5. On the Parser tab, click Create Parser.

6. Enter a Parser Name and Parser Description, and then click OK.

   A parser name supports Chinese characters, English letters, Japanese characters, numbers, underscores (_), and hyphens (-). The length must be 1 to 30 characters. A Chinese or Japanese character is counted as 2 characters.

   The console takes you to the parser details page. You can click Edit in the upper-right corner to modify the name and description.

Step 2: Associate data source and destination

1. On the parser details page, click Associate Data Source.

2. In the dialog box that appears, select an existing data source from the Data Source drop-down list, and then click OK.

   A dialog box informs you that messages from all topics in the data source will be forwarded to the parser for processing, and a parser can be associated with only one data source. To create a data source, click the Create Data Source link below the drop-down list.

3. On the parser details page, click the Data Destination tab.

   This page has two sections: Data Destination and Error Data Destination. Each section includes columns for the data destination name, ID, and type. To add a data destination, click the Associate Data Destination button.

4. Click Associate Data Destination. In the dialog box that appears, select an existing data destination from the Data Destination drop-down list, and then click OK.

   A dialog box informs you that data processed by the parser can be forwarded only to associated data destinations, which can be controlled by the script. You can associate up to 10 data destinations. If no data destination has been created, you can click the Create Data Destination link to create one.

5. In the Error Data Destination section, click Associate Data Destination to specify a location for forwarding error messages that fail after all retries.

   Important

   • You can add only one error action.

   • A regular data destination and an error data destination cannot use the same type of cloud service. For example, data cannot be forwarded to Tablestore as both a regular destination and an error destination.

   • If a message fails to be forwarded to the error data destination, no further retries are performed.

   • Error messages in this context refer to failures that occur when forwarding data to a data destination after the parsing script has run.

   If a message fails to be forwarded to a cloud service, IoT Platform retries to forward the message. If the retry fails, an error message is forwarded based on the error operation that you specify for data forwarding.

   Error message format:

   {
      "ruleName":"",
      "topic":"",
      "productKey":"",
      "deviceName":"",
      "messageId":"",
      "base64OriginalPayload":"",
      "failures":[
       {
        "actionType":"OTS",
        "actionRegion":"cn-shanghai",
        "actionResource":"table1",
        "errorMessage":""
       },
       {
        "actionType":"RDS",
        "actionRegion":"cn-shanghai",
        "actionResource":"instance1/table1",
        "errorMessage":""
       }
      ]
   }

   The following table describes the parameters of error messages.

   Parameter	Description
   ruleName	The rule name.
   topic	The source topic of the message.
   productKey	The ProductKey of the product.
   deviceName	The DeviceName of the device.
   messageId	The ID of the message that is sent from IoT Platform.
   base64OriginalPayload	The Base64-encoded raw data.
   failures	The error details. Multiple errors may occur.
   actionType	The type of the failed operation.
   actionRegion	The region where the error occurred.
   actionResource	The resource targeted by the failed operation.
   errorMessage	The error message.

Step 3: Configure script and start parser

1. On the parser details page, click the Parser Script tab.

2. In the script editor, enter the parsing script.

   For information about how to write scripts, see Script examples. For examples of data forwarding functions and scripts, see the topics in Data forwarding examples.

3. Click Debug. In the panel that appears, select a product and a device, and enter a topic and payload data to verify that the script is executable.

   • Topic: The data format of the input topic must be compatible with the script's parsing logic.

   • Payload Data: The input data must use the format required by the rules engine.

     • For custom topics, the data format is the original format reported by the device.

     • For information about the data formats of basic communication topics and Thing Specification Language (TSL) communication topics, see Data formats.

   The Result tab displays the values of variables declared in the script and the data forwarding functions that were executed.

   After you successfully debug the script, the system writes the test data to the corresponding cloud service. You can log on to the console of that service to view the forwarded data.

4. Click Publish.

5. After you complete the configuration, return to the Parser tab on the Data Forwarding page. Find your parser and click Start in the Actions column. Once started, the parser forwards data according to the parsing script.

Import previous rules

You can import rules from the previous version of the data forwarding feature. After importing, you can reconfigure the data source, data destination, and parsing script as needed. To import previous rules:

1. On the Parser tab, click Import Previous Rules.

   In the Import Previous Rules panel that appears, use the search box to filter rules by name. Select the checkboxes of the rules that you want to import, and then click OK.

2. In the rule list in the panel on the right, select the rules to import, and then click OK.

   After importing, the rules appear in the parser list and are automatically renamed using the format "connector_" + ${PreviousRuleName}.

   On the parser details page, you can modify the parser name and reconfigure the data source, data destination, and parsing script based on your business requirements.

   Important

   The value of a device tag is a string, and the attribute() function also returns a string. If you need to perform mathematical calculations on this value, you must manually convert its data type.

More operations

On the Parser tab of the Data Forwarding page, you can also perform the following operations:

• View: Click a parser's name to open its parser details page. From there, you can update its settings and query its run logs. For example, you can modify or delete the data source topic or data destination.

  Note

  You can view logs only for parsers that are in the Running state. On the parser details page, click View Log next to Parser Status to open the parser's run logs in Log Service.

• Delete: Deletes the parser.

  Important

  You cannot delete a running parser.

• Stop: Stops the parser from forwarding data.

Related documents

• For instructions on how to connect devices to IoT Platform and have them report data, see Device connection guide.

• After you connect a device to IoT Platform and it starts reporting data, you can view its message forwarding logs in the IoT Platform console. For more information, see Cloud-side run logs.

• For information about the limits on device message forwarding for an IoT Platform instance, see Limits.

API reference

FAQ

• Can I subscribe to hexadecimal data reported by devices through an AMQP server-side subscription?

• How do I configure a server to monitor messages from all of my devices?

• How does IoT Platform ensure that a single message from a device is sent to multiple servers without duplication?

• How long is device data retained in IoT Platform if it is not forwarded?

• Can I export data reported by devices?