Comparison of data forwarding solutions

更新时间:
复制 MD 格式

In many scenarios, you need to process data that devices report to IoT Platform or use the data for business applications. You can use the server-side subscription and data forwarding features of IoT Platform to forward device data. This topic compares data forwarding solutions and their application scenarios to help you choose the most suitable solution.

Data forwarding methods

Method

Description

References

Data forwarding

Provides basic data filtering and transformation. You can filter and transform device data, and then forward the data to other Alibaba Cloud product instances.

Configure data forwarding rules

Server-side subscription

Directly obtain device messages through an AMQP client or a Simple Message Queue (formerly MNS) (SMQ) client. This method allows for fast message retrieval. It does not support message filtering or transformation. The feature is simple but easy to use and efficient.

Comparison of forwarding solutions

Comparison of forwarding methods

Method

Scenarios

Pros

Cons

Limits

Data forwarding

  • Scenarios where you need to forward device messages to other topics or other Alibaba Cloud products for storage or processing.

  • Scenarios that require high throughput.

  • Comprehensive features.

  • Supports the adjustment of forwarding rules at runtime.

  • Supports simple data filtering.

  • Supports forwarding data to other Alibaba Cloud products.

    For a comparison of data forwarding destinations, see Comparison of data forwarding destinations.

Requires you to write SQL statements and configure rules, which can be complex.

For more information, see Data forwarding limits.

Server-side subscription

  • Scenarios where you only need to receive device data.

  • Scenarios in which the server-side receives all data from subscribed devices under a specific product.

Simple, easy to use, and efficient.

Lacks filtering and data transformation capabilities.

  • Does not support fine-grained filtering or subscription. You can only receive all messages for a tenant.

  • The throughput limit for a single consumer group is 1,000 QPS.

For more information about the limits, see Server-side subscription limits.

Comparison of data forwarding destinations

Destination

Scenarios

Pros

Cons

References

Message Queue for Apache RocketMQ

Scenarios that involve many devices and require complex or fine-grained data processing.

Message Queue for Apache RocketMQ is recommended for scenarios where the message volume is greater than 1,000 QPS.

  • Stable and reliable.

  • Supports massive data volumes.

Limited support for public networks. The Platinum edition provides better performance over public networks.

Simple Message Queue (formerly MNS) (SMQ)

Scenarios that require complex or fine-grained data processing over a public network.

MNS is recommended for scenarios where the message volume is less than 1,000 QPS.

  • Uses the HTTPS protocol.

  • Good support for public networks.

Lower performance compared to Message Queue for Apache RocketMQ.

ApsaraDB RDS

Suitable for simple data storage scenarios.

Data is written directly to the database.

N/A.

Time Series Database (TSDB)

Suitable for scenarios that require business analysis and monitoring based on device data.

Data is written directly to the time series database.

N/A.

DataHub

Suitable for scenarios that require data analysis and processing.

Data is written directly to DataHub.

N/A.

Tablestore

Suitable for simple data storage scenarios.

Data is written directly to a Tablestore instance.

N/A.

Function Compute

Scenarios that require a simplified device development process and flexible data processing.

  • Highly flexible data processing.

  • Versatile features.

  • No deployment required.

Higher cost.

Usage and notes

Server-side subscription

Feature description

Notes

A server can use an AMQP SDK or an Simple Message Queue (formerly MNS) (SMQ) SDK to receive all types of messages for a subscribed product. These messages include device-reported messages, device status notifications, device lifecycle changes, Thing Specification Language (TSL) model historical data reports, over-the-air (OTA) update status notifications, sub-device discovery reports from gateways, and device topology changes.

  • Server-side subscription prioritizes real-time messages and may downgrade stacked messages. Make sure that the consumer can consume messages promptly.

  • Server-side subscription is not suitable for scenarios that require advanced filtering or fine-grained processing. For these scenarios, use the rules engine.

Forwarding data to Message Queue for Apache RocketMQ

Feature description

Limits

Notes

Use the data forwarding feature to forward messages from a specified topic in IoT Platform to a topic in Message Queue for Apache RocketMQ. Then, use the RocketMQ SDK to receive the messages.

  • We recommend using Message Queue for Apache RocketMQ to receive device messages because of its high performance.

  • Message Queue for Apache RocketMQ supports cross-tenant topic authorization, which meets the requirements for cross-tenant data forwarding in certain scenarios.

  • The public test cluster of Message Queue for Apache RocketMQ cannot be used in a production environment.

  • The non-public Endpoint of Message Queue for Apache RocketMQ cannot be used for local testing. You must run tests on an ECS instance in the same region.

  • The public test cluster of Message Queue for Apache RocketMQ allows developers to receive messages locally, but it has poor stability. Do not use it in a production environment.

  • For scenarios with a large volume of device messages (>5,000 QPS) or high stability requirements, use the Platinum Edition of Message Queue for Apache RocketMQ.

  • If data forwarding fails and subsequent retries also fail, the rules engine discards the message. In addition, message-based products may experience delays. Your business must be designed to handle message loss or delayed delivery.

Forwarding data to Simple Message Queue (formerly MNS) (SMQ)

Feature description

Limits

Notes

You can use the data forwarding feature to forward messages from a specified topic to a subject in MNS. Then, use the MNS SDK to receive the messages. MNS provides good support for public network environments. MNS is recommended if the device message volume is not particularly large (<1,000 QPS).

For more information, see the subject-related limits in MNS limits.

If data forwarding fails and subsequent retries also fail, the rules engine discards the message. In addition, message-based products may experience delays. Your business must be designed to handle message loss or delayed delivery.

Forwarding data to Function Compute

Feature description

Limits

Notes

Use data forwarding to transfer messages from a specified topic to Function Compute, where developers can further process the messages. Function Compute is deployment-free and can simplify business development.

For more information, see Function Compute limits.

  • Suitable for scenarios that require custom processing of device messages or simplified development and operations.

  • If data forwarding fails and subsequent retries also fail, the rules engine discards the message. Your business must be designed to handle message loss or delayed delivery.