Data access overview
Data access serves as the cornerstone of IoT data services, encompassing a variety of sources such as devices, cloud products, IoT twin, API, and local uploads. These sources underpin critical IoT data service functions, including data parsing, storage, and metrics.
Device data sources
Device data sources encompass custom topic data and Thing Specification Language data reported by devices. By default, device data is retained for 30 days. Configuring data sources allows for extended data backup and retention, facilitating in-depth data analysis.
Data backup options include backing up all product data at the instance level or selecting specific products for backup at the product level, depending on business requirements.
Thing Specification Language properties come with default data tables, eliminating the need for data parsing. However, data parsing can be utilized to direct reported data from events and custom topics into custom data tables.
Cloud product data sources
The IoT Platform offers cloud product data access capabilities, allowing for the synchronization of data from other cloud platforms to the IoT Platform's data service for subsequent processing and analysis.
Currently, synchronization is limited to files in OSS Buckets.
IoT twin engine data sources
IoT twin engine data sources refer to Thing Specification Language property data from twin nodes within the twin entity graph under the designated twin space. Backing up and storing IoT twin engine data sources enables comprehensive data analysis.
Backup options include all IoT twin engine data sources at the instance level or selected twin spaces at the IoT twin engine space level, tailored to business needs. For detailed procedures, see Back up IoT Twin Engine Data Sources.
API data sources
The IoT Platform provides API data source capabilities, enabling the importation of local data into the platform for further analysis via API calls.
Upload local data sources
The IoT Platform enables the uploading of local files, allowing for the importation of local file data into the platform's custom storage tables for analysis, application, or as data sources for the IoT twin engine.
Application data sources
Once data sources are configured, they can be leveraged across various functions to maximize data utilization.
Function | Device Data Sources | Cloud Product Data Sources | IoT Twin Engine Data Sources | API Data Sources | Local Upload Data Sources |
Supported | Supported | Not Available | Supported | Supported | |
Supported | Supported | Supported | Supported | Supported | |
Supported | Not Available | Not Available | Not Available | Not Available | |
Supported | Supported | Supported | Not Available | Supported |
The following outlines the specific applications for each data source.
Device data sources
Once device data backup is enabled, the data is stored across the platform's system table, time series table, and cold data snapshot table.
Function | Support Description |
Data Parsing | Processes and transforms device Thing Specification Language (properties and events) and custom topic data, outputting the necessary data tables for business requirements. Applications for the output data storage tables include:
|
Cold Data Storage |
|
Data Metrics | Manages data metrics from metric and domain perspectives for Thing Specification Language data within device data sources, enhancing configurations and business efficiency. |
Analysis Insight | Leverages data from the time series and snapshot tables as query subjects to set up analysis tasks using SQL statements. |
Cloud product data sources
Cloud product data synchronized to the IoT Platform is stored in the designated custom storage table.
Function | Support Description |
Data Parsing | Serves as the output for data parsing tasks, storing the necessary data. |
Cold Data Storage | Organizes cloud product data in a tabular layout. |
Analysis Insight |
|
IoT twin engine data sources
With IoT twin engine twin space backup enabled, the data is stored in the cold data time series and snapshot tables.
Function | Support Description |
Cold Data Storage | Displays the Thing Specification Language properties' time series and snapshot data of twin nodes within the twin entity graph, under the corresponding twin space of the IoT twin engine, in a structured table format. |
Analysis Insight | Employs data from the IoT twin engine's time series and snapshot tables as query subjects for configuring SQL statement-based analysis tasks. |
API data sources
Upon configuring API data sources, the data imported into the IoT Platform via APIs is stored in the specified custom storage table.
Function | Support Description |
Data Parsing | Processes and transforms device Thing Specification Language (properties and events) and custom topic data, outputting the necessary data tables for business requirements. Applications for the output data storage tables include:
|
Cold Data Storage |
|
Local upload data sources
Following the upload of local files, the data is stored in the designated custom storage table.
Function | Support Description |
Data Parsing | Acts as the output for data parsing tasks, storing the necessary data. |
Cold Data Storage | Organizes local file data in a tabular layout. |
Analysis Insight |
|