Use the data forwarding feature to parse, process, and store data from IoT Platform in an ApsaraDB RDS instance.
How it works
The data forwarding feature routes messages from device topics to a table in an ApsaraDB RDS database, which is a managed database service. For more information, see What is ApsaraDB RDS?.
In the preceding figure:
-
Data source: Messages from supported topic types. For more information about supported data formats, see Data formats.
-
SQL expression: Write an SQL expression to parse and process JSON data from devices. For more information about SQL syntax, see SQL expressions.
NoteBinary data is passed through without being parsed.
-
Data destination: A table in a database that belongs to an ApsaraDB RDS for MySQL or ApsaraDB RDS for SQL Server instance within a Virtual Private Cloud (VPC). This table receives device data.
Limitations
-
Region restrictions:
-
IoT Platform instances in specific regions support data forwarding to ApsaraDB RDS. For more information about supported regions, see Feature availability by region.
-
The ApsaraDB RDS instance must be in the same region as your IoT Platform instance. For example, data from an IoT Platform instance in the China (Shanghai) region can be forwarded only to a table in an ApsaraDB RDS instance in the China (Shanghai) region.
-
-
You can forward data only to ApsaraDB RDS instances in a Virtual Private Cloud (VPC).
-
ApsaraDB RDS for MySQL and ApsaraDB RDS for SQL Server instances are supported.
-
Data can be forwarded to standard and high-privilege databases.
-
Data forwarding only supports data in JSON format.
-
Both the new and legacy versions of the data forwarding feature support routing data to ApsaraDB RDS. For an example that uses the new version, see Forward data to ApsaraDB RDS (new version).
-
The legacy version of data forwarding cannot be used for cloud gateway products and devices, or for products and devices under an MQTT-based instance. To forward data from these products and devices, you must use the new version of data forwarding. For more information, see Data forwarding (new version).
-
For more limitations on forwarding device messages from an IoT Platform instance, see Limitations.
Prerequisites
-
An ApsaraDB RDS for MySQL or ApsaraDB RDS for SQL Server instance in a VPC, located in the same region as your IoT Platform instance. A database and a table must be created in the instance. For more information about how to use ApsaraDB RDS, see the ApsaraDB RDS documentation.
-
A data forwarding rule with an SQL expression to process the data. For more information, see Create a rule to process data.
Background information
After you configure the data destination, the IoT Platform rules engine automatically performs the following configurations to forward device data to ApsaraDB RDS.
-
IoT Platform consumes two IP addresses from the vSwitch where the ApsaraDB RDS instance resides. The IPv4 CIDR block of the vSwitch is added to the whitelist of the ApsaraDB RDS instance.
-
A managed security group is created in the VPC where the ApsaraDB RDS instance is located. The name of the security group starts with sg-nsm- by default.
Configure data destination
Log on to the IoT Platform console.
On the Overview page, find the instance that you want to manage and click the instance ID or instance name.
In the navigation pane on the left, choose .
Find the rule that you want to manage and click View in the Actions column. The Data Forwarding Rule page appears.
ImportantIf the new version of the data forwarding page is displayed, click Back to Old Version in the upper-right corner. Then, find the destination rule and click View.
In the Forward Data section, click Add Operation.
-
In the Add Operation dialog box, select Store Data to ApsaraDB RDS for the Operation parameter. Configure the other parameters and click OK.
Parameter
Description
Operation
Select Store Data to ApsaraDB RDS.
Region
The region where your IoT Platform instance resides. This parameter is fixed.
RDS instance
Select the destination ApsaraDB RDS instance.
Database
Enter the name of the database.
NoteFor a high-privilege database, you must enter the database name manually.
Account
Enter the username for the ApsaraDB RDS instance. This account must have read and write permissions on the database. Otherwise, the rules engine cannot write data to the ApsaraDB RDS instance.
NoteAfter the rules engine obtains the account, it only writes data that matches the rule to the database and performs no other operations.
Password
Enter the password for the database account.
Table name
Enter the name of the table that you created in the database. The rules engine writes data to this table.
Key
Enter a column name from the ApsaraDB RDS table. The rules engine stores data in this column.
Value
Enter a field from the topic's message payload defined in the SQL expression. The rules engine stores this field's value in the column specified by Key.
Note-
The data type of the Value must match the data type of the Key (column). Otherwise, the write operation will fail.
-
You can enter a variable, such as
${deviceName}.
Role
Select a RAM role that allows IoT Platform to write data to ApsaraDB RDS.
If you have not created the required RAM role, click Create RAM Role to go to the RAM console. Then, create a RAM role and an authorization policy. For more information, see Create a RAM role.
-
Return to the Data Forwarding page. Find the rule and click Start to enable the rule.
-
In the ApsaraDB RDS console, on the Data Security page, configure the whitelist. The rules engine requires a connection to ApsaraDB RDS. After you configure the data destination, the rules engine automatically adds its IP addresses to the ApsaraDB RDS whitelist. If the following IP addresses are not in the whitelist, you must add them manually.ApsaraDB RDS
-
China (Shanghai):
100.104.53.192/26,100.104.148.64/26,100.104.6.192/26,100.104.143.128/26,100.104.76.0/24,100.104.73.128/26,100.104.200.64/26,100.104.40.64/26,100.104.3.0/26,100.104.29.128/26,100.104.121.0/26,100.104.84.64/26 -
Asia Pacific (Singapore):
100.104.106.0/24 -
US (Silicon Valley):
100.104.8.0/24 -
US (Virginia):
100.104.133.64/26 -
Germany (Frankfurt):
100.104.160.192/26 -
Japan (Tokyo):
100.104.160.192/26
In the left-side navigation pane for your ApsaraDB RDS instance, click Data Security. On the Whitelist settings tab, click Add whitelist group and add the IP address CIDR blocks for the corresponding IoT Platform region to the group.
-
What to do next
In the IoT Platform console, go to . On the IoT Platform Logs tab, you can view the operational logs for Device-to-Cloud Messages and Data Forwarding. For more information, see IoT Platform logs.
You can connect to the RDS instance and log on to the RDS database to view the forwarded data. For more information, see Connect to an ApsaraDB for MySQL instance and Connect to an ApsaraDB for SQL Server instance.
References
For instructions on how to connect a device to IoT Platform to report data, see Device connection guide.
On the instance details page, you can view the available resources for message forwarding transactions per second (TPS) to ensure that device messages can be forwarded to the RDS database. For more information, see View instance information and operational data. If the resources are insufficient, you can upgrade the instance. For more information, see Upgrade an instance.
For answers to frequently asked questions about using ApsaraDB RDS databases, see FAQ about databases.