Forward data to Tablestore

更新时间:
复制 MD 格式

Device metadata, such as tags, properties, and status, demands high-frequency updates and advanced query capabilities. To manage devices, filter groups, and query status, you must store this data in a service that supports high-concurrency, low-latency updates, multi-dimensional retrieval, geospatial queries, and real-time analytics. Tablestore, with its distributed wide column model, is an ideal solution for storing and updating device metadata. This topic shows how to use the data forwarding feature of IoT Platform to route device data to a Tablestore wide column model table.

How it works

image

The data forwarding feature routes messages from a device topic to a data table within a Tablestore instance, using the Tablestore wide column model.

The following list describes the components in the preceding diagram:

  • Data source: A message from a supported topic. For more information, see Data formats.

  • SQL statement: An SQL statement used to parse and process the JSON data that devices report. For more information about SQL syntax, see SQL statements.

    Note

    The system does not parse binary data and forwards it as is.

  • Data destination: A data table in your Tablestore instance configured to receive device data.

Limitations

  • This feature is available only in regions that support forwarding data to the Tablestore wide column model. For a list of supported regions, see Feature availability by region.

  • Both the new and previous versions of data forwarding support routing data to the Tablestore wide column model. For an example that uses the new version, see Forward data to a Tablestore wide column model table.

  • The new version of data forwarding also supports routing data to the Tablestore timeseries model. For an example, see Forward data to a Tablestore timeseries model table.

  • The previous version of data forwarding does not support Cloud Gateway products and devices, or products and devices under MQTT-based instances. You must use the new version of data forwarding for these products and devices. For more information, see Data forwarding (new version).

  • For more information about the limitations on forwarding device messages from an IoT Platform instance, see Limitations.

Scenarios

This feature can be used in Internet of Things (IoT) scenarios such as Internet of Vehicles (IoV), smart homes, and industrial IoT. For more information, see Application scenarios of IoTstore.

Prerequisites

  • You have created a Tablestore instance and a data table. For more information, see the Tablestore documentation.

    Important

    If you use an Enterprise Edition instance of IoT Platform, the Tablestore instance must be in the same region as your IoT Platform instance.

  • You have created a data forwarding rule and written an SQL statement to process the data. For more information, see Configure a data forwarding rule.

    The rule in this example uses the following SQL statement:

    SELECT items.temperature.value as temperature, items.humidity.value as humidity,deviceName() as deviceName 
    FROM "/sys/a1ktuxe****/BZoyHO***/thing/event/property/post"

Configure a data destination

  1. Log on to the IoT Platform console.

  2. On the Overview page, find the instance that you want to manage and click the instance ID or instance name.

  3. In the navigation pane on the left, choose Message Forwarding > Data Forwarding.

  4. Find the rule that you want to manage and click View in the Actions column. The Data Forwarding Rule page appears.

    Important

    If the new version of the data forwarding page is displayed, click Back to Old Version in the upper-right corner. Then, find the destination rule and click View.

  5. In the Forward Data section, click Add Operation.

  6. In the Add Operation dialog box, configure the parameters based on the following table, and then click OK.

    Note

    You can forward only data in JSON format.

    Parameter

    Description

    Operation

    Select Store Data to Tablestore.

    Region

    Select the region where your Tablestore instance is located.

    Instance

    Select a Tablestore instance.

    You can click Create Instance to go to the Tablestore console and create an instance. For more information, see the Tablestore documentation.

    data table

    Select the Tablestore data table to receive the data.

    You can click Create Data Table to go to the Tablestore console and create a data table.

    primary key

    Set the primary key value by specifying a field from the SQL statement. During data forwarding, the system uses this field's value as the primary key value.

    Note
    • You can use the variable format ${}. For example, ${deviceName} indicates that the primary key value is the value of the deviceName field in the message.

    • If the primary key type is an auto-increment column, you do not need to enter a value for this column. Tablestore automatically generates a value for it. The system automatically sets the value to AUTO_INCREMENT, and it cannot be edited.

      For more information about auto-increment primary keys, see Auto-increment primary key columns.

    role

    Select a RAM role that grants IoT Platform permissions to write data to Tablestore.

    If you have not created the required RAM role, click Create RAM Role to go to the RAM console. Then, create a RAM role and an authorization policy. For more information, see Create a RAM role.

  7. Return to the Data Forwarding page. Find the rule and click Start to enable the rule.

  8. Test the rule.

    1. Log on to the IoT Platform console and go to your instance.

    2. In the left-side navigation pane, choose Maintenance > Device Simulator.

    3. Select the device for debugging, click Upstream Debug, then Properties, and then Start Device Simulator.

    4. Under the Default Module, set test data, and then click Send Command.

      For example, set humidity to 45 and temperature to 35.

What to do next

  1. In the IoT Platform console, click the destination Enterprise instance to view the device status and message forwarding logs. For more information, see IoT Platform logs.

  2. After the data is pushed, go to the Data Management tab of the data table in Tablestore that receives data. Check whether the specified data is received.

    数据管理

Example

Device metadata ingestion solution

References

  • For more information about how to connect a device to IoT Platform to report data, see Device connection guide.

  • On the instance details page, you can view the available resources for message forwarding transactions per second (TPS) to ensure that device messages can be forwarded to Tablestore. For more information, see View instance information and runtime data. If the available resources are insufficient, you can upgrade the instance. For more information, see Upgrade an instance.

  • For more information about the limits and FAQ of Tablestore, see General FAQ about Tablestore.