Forward data to ApsaraMQ for RocketMQ

How it works

The data forwarding feature sends device messages to a RocketMQ SDK client for consumption. The workflow is as follows:

1. Application server: Use the RocketMQ SDK to register a consumer. The consumer receives messages that IoT Platform forwards to ApsaraMQ for RocketMQ.

2. IoT Platform: Configure data forwarding to send device messages to ApsaraMQ for RocketMQ.

   1. Data source: For information about supported topic message types, see Data formats (for products and devices that are not cloud gateways), Custom topics (for MQTT cloud gateways), Message forwarding topics (for GB/T 32960 cloud gateways), Message forwarding topics (for JT/T 808 cloud gateways), and Message forwarding topics (for SL 651 cloud gateways).

   2. Data destination: The ApsaraMQ for RocketMQ topic that receives device messages. You must first create this topic in the ApsaraMQ for RocketMQ console. For more information, see Create a topic.

   3. Resolver script: Configure the data forwarding function writeMq(destinationId, payload, tag) to forward device data to the ApsaraMQ for RocketMQ topic.

      For more information about the function, see Functions for forwarding data to data destinations.

      You can set properties and tags for device messages. Consumers can then use these properties and tags as filter conditions.

3. ApsaraMQ for RocketMQ: When a consumer registered by the RocketMQ SDK retrieves messages, ApsaraMQ for RocketMQ performs dynamic server-side filtering. It matches messages against the consumer's filter expression and delivers only the matching messages, allowing the application server to receive specific device data from IoT Platform.

Scenarios

Receiving device messages on an application server: Forward device messages to ApsaraMQ for RocketMQ for consumption.

• Scenarios with a massive number of devices that require complex or fine-grained data processing.

• Scenarios where the device message volume is greater than 1,000 QPS.

Limits

• IoT Platform instances in specific regions support data forwarding to ApsaraMQ for RocketMQ. For more information about the supported regions, see Features available in different regions.

• The region of the ApsaraMQ for RocketMQ instance must be the same as the region of the IoT Platform instance. Existing data forwarding configurations for cross-region instances are not affected and continue to work.

• You can forward data only to topics in ApsaraMQ for RocketMQ 4.x or 5.x instances. Serverless instances are not supported. For topics in ApsaraMQ for RocketMQ 5.x instances, the Message Type must be Normal Message.

• Both the new and previous versions of the data forwarding feature support forwarding data to ApsaraMQ for RocketMQ. For an example of how to use the previous version, see Forward data to ApsaraMQ for RocketMQ (previous version).

Background information

When you use ApsaraMQ for RocketMQ 5.0, the following configurations are automatically applied after you set up the data destination. These enable the IoT Platform rules engine to forward device data to ApsaraMQ for RocketMQ.

• IoT Platform occupies two IP addresses of the virtual switch (vSwitch) in which the ApsaraMQ for RocketMQ instance is located.

• A managed security group is created in the VPC where the ApsaraMQ for RocketMQ instance resides. The security group name starts with sg-nsm- by default.

Step 1: Create a data destination

1. Log on to the IoT Platform console.

2. On the Overview page, find the instance that you want to manage and click the instance ID or instance name.

3. In the left-side navigation pane, choose Message Forwarding > Data Forwarding.

4. On the Data Forwarding page, click Go to New Version in the upper-right corner.

   Note

   If you have performed this operation before, you are automatically redirected to the new version of the Data Forwarding page.

5. Click the Data destination tab, and then click Create data destination.

6. In the Create Data Destination dialog box, enter a name for the data destination, such as DataPurpose. Configure the parameters as described in the following table, and then click OK.

   

   Parameter	Description
   Select operation	Select Forward data to ApsaraMQ for RocketMQ.
   Region	Select the region of the ApsaraMQ for RocketMQ instance.
   Instance	Select an ApsaraMQ for RocketMQ instance. Click Create Instance to create one in the ApsaraMQ for RocketMQ console. ApsaraMQ for RocketMQ documentation.
   Topic	Select the ApsaraMQ for RocketMQ topic to receive data from IoT Platform. Click Create Topic to create one in the ApsaraMQ for RocketMQ console.
   Authorization	Grant IoT Platform write access to ApsaraMQ for RocketMQ. If no RAM role exists, click Create RAM Role to create a RAM role and authorization policy in the RAM console. Create a RAM role.

Step 2: Configure and start the resolver

1. Create a parser, such as DataParser. For more information, see Step 1: Create a parser.

2. On the Resolver Details page, associate the data source.

   1. In the configuration wizard, under Data Source, click Associate Data Source.

   2. In the dialog box, select DataSource from the Data Source drop-down list and click OK.

3. On the Resolver Details page, you can associate data destinations.

   1. In the configuration wizard, click Data Destination. Then, in the upper-right corner of the Data Destination list, click Associate Data Destination.

   2. In the dialog box that appears, from the Data Destination drop-down list, select DataPurpose and click OK.

   3. In the data destination list, find the Data Destination ID and record it. For example, 1000.

      You will need this Data Destination ID in the parsing script.

4. On the parser details page, click the Parser Script tab.

5. In the script editor, enter a resolver script.

   For more information about function parameters, see Function list.

   // Use the payload() function to get the message content reported by the device and convert it to the JSON format.
   var data = payload("json");
   // Directly forward the data reported by the TSL model.
   writeMq(1000, data, "debug");

6. Click Test. Follow the on-screen instructions to select a product and a device, enter the topic and payload data, and verify that the script runs as expected.

   The following figure shows sample parameters:

   The following output indicates that the script executed successfully.

   

7. Click Publish.

8. Return to the Resolver tab on the Data Forwarding page and click Start for the DataParser resolver.

9. In the ApsaraMQ for RocketMQ console, check whether the message was received.

Step 3: View operational logs for device message forwarding

1. In the IoT Platform console, click the target Enterprise instance to view the device status and message forwarding logs. For more information, see IoT Platform logs.

2. In the terminal on the application server that is subscribed to ApsaraMQ for RocketMQ resources, view the logs for message subscription and consumption.

3. In the ApsaraMQ for RocketMQ console, on the Instance Details page, view the content and trace of the messages received by the consumer. For more information, see Query message traces.

Operation example

Consume device messages using an ApsaraMQ for RocketMQ client

FAQ

• What are the advantages of ApsaraMQ for RocketMQ 5.0 instances over 4.0 instances?

• FAQ about ApsaraMQ for RocketMQ 5.x instances

• FAQ about ApsaraMQ for RocketMQ 4.x instances