Forward data to Apache Kafka

Limitations

• Serverless Kafka is not supported.

Create a data destination

1. Log on to the IoT Platform console.

2. On the Overview page, find the instance that you want to manage and click the instance ID or instance name.

3. In the navigation pane on the left, choose Message Forwarding > Data Forwarding.

4. On the Data Forwarding page, click Try New Version in the upper-right corner to open the new feature page.

   Note

   If you have already performed this operation, you will be taken directly to the new feature page the next time you access the Data Forwarding page.

5. Click the Data Destination tab. On this tab, click Create Data Destination.

6. In the Create Data Destination dialog box, enter a name for the data destination, such as DataPurpose. Configure the parameters as described in the following table, and then click OK.

   Parameter	Description
   Select an operation	Select Send data to ApsaraMQ for Kafka.
   Role	Grants IoT Platform the permissions to write data to Kafka. If you have not created the required RAM role, click Create RAM Role to go to the RAM console. Then, create a RAM role and an authorization policy. For more information, see Create a RAM role.
   Region	The region where your IoT Platform instance resides. This parameter is fixed and cannot be changed.
   Instance	Select an ApsaraMQ for Kafka instance. You can click Create Instance to go to the ApsaraMQ for Kafka console and create an instance. For more information, see Create an instance.
   Topic	Select the ApsaraMQ for Kafka topic that you want to use to receive data from IoT Platform. You can click Create Topic to go to the ApsaraMQ for Kafka console and create a topic. For more information, see Create a topic.

Configure and start the parser

1. Create a parser named DataParser. For more information, see Create a parser.

2. On the Parser Details page, associate the parser with the created data source.

   1. In the Data Source step of the wizard, click Associate Data Source.

   2. In the dialog box that appears, select DataSource from the Data Source drop-down list, and then click OK.

3. On the Parser Details page, associate the parser with the created data destination.

   1. Click Data Destination in the wizard. In the Data Destination section, click Associate Data Destination.

   2. In the dialog box that appears, select DataPurpose from the Data Destination drop-down list, and then click OK.

   3. In the Data Destination section, view and save the data destination ID. In this example, the ID is 1000.

      When you write the parser script, you must use the data destination ID.

4. On the Parser Details page, click Parser.

5. In the script editor, enter a script.

   For more information about the function parameters, see Function list.

   // Use the payload() function to get the message reported by the device and convert it to the JSON format.
   var data = payload("json");
   // Forward the TSL data directly.
   writeKafka(1000, data, "Debugging");

6. Click Debugging. Follow the on-screen instructions to select a product and device, enter a topic and payload data, and verify that the script runs correctly.

   The following figure shows sample parameters:

   The following output indicates that the script ran successfully.

   action:
       transmit to kafka[destinationId=1000], data:{"deviceType":"CustomCategory","iotId":"JCp9u***","requestId":"1626948228247","checkFailedData":{},"productKey":"a1o***","gmtCreate":1626948134445,"deviceName":"Device1","items":{"Temperature":{"time":1626948134319,"value":38},"Humidity":{"time":1626948134319,"value":25}}}
   variables:
       data : {"deviceType":"CustomCategory","iotId":"JCp9u***","requestId":"1626948228247","checkFailedData":{},"productKey":"a1o***","gmtCreate":1626948134445,"deviceName":"Device1","items":{"Temperature":{"time":1626948134319,"value":38},"Humidity":{"time":1626948134319,"value":25}}}

7. Click Publish.

8. Go to the Parser tab of the Data Forwarding page. Find the DataParser parser and click Start in the Actions column to start the parser.

9. In the Message Queue for Apache Kafka console, go to the Topic Details page for your instance to view the forwarded messages.