File upload

Updated at:

IoT Platform allows devices to upload files using the Message Queuing Telemetry Transport (MQTT) protocol. You can store the files in your own OSS bucket or the Alibaba Cloud IoT Platform space. This topic describes the topics and data formats for the file upload process, which includes requesting a file upload, uploading file shards, and canceling a file upload.

Background information

For information about how to develop the file upload feature on a device, see File upload (MQTT).

If you upload files to your own OSS bucket, you can access them in OSS after the upload is complete.

Important The OSS bucket must belong to the user who owns the device. The bucket region must be the same as the device region.

If you upload files to the Alibaba Cloud IoT Platform space, you can view, download, and delete them in the IoT Platform console. For more information, see File management.

Device requests a file upload

  • Request topic: /sys/${productKey}/${deviceName}/thing/file/upload/mqtt/init.
  • Response topic: /sys/${productKey}/${deviceName}/thing/file/upload/mqtt/init_reply.

Request data format:

{
    "id":"123456",
    "params":{
        "fileName":"abc.bin",
        "fileSize":123455,
        "conflictStrategy":"overwrite",
        "ficMode":"crc64",
        "ficValue":"0205d7***",
        "initUid":"ab1***34",
        "extraParams":{
            "ossOwnerType":"device-user",
            "serviceId":"device1.bucket",
            "fileTag":{
                "tagKey1":"tagValu1",
                "tagKey2":"tagValue2"
            }
        }
    }
}
Table 1. Request parameter description
ParameterTypeDescription
idString
The message ID. The ID must be a string-type number that is unique to the device. Valid values: 0 to 4294967295.
paramsObject
Business request parameters
fileNameStringThe name of the file that the device uploads. The name must meet the following requirements:
  • It can contain digits, letters, underscores (_), and periods (.).
  • The first character must be a digit or a letter.
  • The length cannot exceed 100 bytes.
fileSizeLongThe size of the file to upload, in bytes. A single file cannot exceed 16 MB.

A value of -1 indicates that the file size is unknown. When the file upload is complete, you must specify the isComplete parameter in the message for uploading file shards. For more information, see Device uploads file shards.

conflictStrategyStringThe policy for handling files with the same name that are uploaded by the device. This parameter is optional. The default value is overwrite.

Usage notes:

  • If a file with the same name does not exist in IoT Platform, a file upload task is created.
  • If a file with the same name already exists in IoT Platform, the file is processed based on the specified policy:
    • overwrite: Overwrite mode.

      The existing file is deleted, and then a new upload task is created.

    • append: Append mode.
      • If the upload of the existing file is incomplete, the device can continue to upload the file based on the information returned by IoT Platform.
      • If the upload of the existing file is complete, the request to create an upload task fails. The device can change the file name or send another upload request in overwrite mode.
    • reject: Reject mode.

      IoT Platform rejects the request to upload the file and returns an error code indicating that the file already exists.

    Important If the device does not complete the upload within 24 hours after the upload task is created, IoT Platform automatically deletes the task.
ficModeStringThe file integrity check mode. The only valid value is crc64. This parameter is optional.
  • If you do not specify this parameter, file integrity is not checked after the upload is complete.
  • If you specify this parameter, you must also specify ficValue. File integrity is checked based on the specified mode and value.
Important If fileSize is -1, you cannot set ficMode and ficValue.
ficValueStringThe file integrity check value. It is a 16-digit hexadecimal string.

This parameter is optional. If you specify this parameter, you must also specify ficMode.

initUidStringA custom unique ID for the file upload request task. The ID must meet the following requirements:
  • It can contain digits, letters, hyphens (-), underscores (_), and periods (.).
  • The first character must be a digit or a letter.
  • The length cannot exceed 16 characters.

This parameter is optional:

  • If you specify this parameter, it is used for idempotence when a file upload request is retried. Make sure that you use the same initUid for retries of the same upload task.

    If two file upload requests from the same device have an identical value for this parameter, the IoT Platform detects a message retry and returns the same response.

    For the same device, if the interval between retries exceeds 24 hours, the retry request is treated as a new upload task request. If the new request is successful, a new uploadId is returned.

  • If you do not specify this parameter, IoT Platform treats the request as a new file upload request.
extraParamsObjectThe configuration parameters for uploading the file to an OSS bucket. This parameter is optional.

If you do not specify this parameter, the device uploads the file to the IoT Platform storage space.

ossOwnerTypeStringThe owner type of the destination OSS bucket. Valid values:
  • iot-platform: Uploads the file to the OSS bucket of Alibaba Cloud IoT Platform.

    The uploaded file appears in the device file list in the IoT Platform console and can be managed using cloud APIs. For more information, see File management.

  • device-user: Uploads the file to the user's own OSS bucket.

    The uploaded file does not appear in the device file list in the IoT Platform console and cannot be managed using cloud APIs.

    The file is uploaded to the following path: {ossbucket}/aliyun-iot-device-file/${instanceId}/${productKey}/${serviceId}/${deviceName}/${fileName}.

    • {ossbucket}: The destination bucket that is configured in the IoT Platform console.
    • {instanceId}: The ID of the instance to which the device belongs.
    • {productKey}: The ProductKey of the product to which the device belongs.
    • {serviceId}: The business ID that is configured in the IoT Platform console.
    • {deviceName}: The device name.
    • {fileName}: The device file name.
serviceIdStringThe business ID for the file upload. You must define this ID in the IoT Platform console in advance. For more information, see Configure file uploads to a bucket.

serviceId is valid only when ossOwnerType is set to device-user.

fileTagObjectThe tags to add to the file when it is saved to the OSS bucket. You can add up to five tags. For information about the tag definition rules, see Object tagging.

A tag key cannot start with two underscores (__).

fileTag is valid only when ossOwnerType is set to device-user.

Response data format:

{
    "id": "123456",
    "code": 200,
    "message": "this is error msg when code is not 200",
    "data":{
        "fileName": "abc.bin",
        "uploadId": "03d9151e-6***",
        "offset": 123123
    }
}
Table 2. Response parameter description
ParameterTypeDescription
idString
The message ID. The ID must be a string-type number that is unique to the device. Valid values: 0 to 4294967295.
codeInteger
The result code. A value of 200 indicates success. Other values indicate failure. For more information, see Error codes received by devices.
messageStringThe error message that is returned if the request fails.
dataObjectThe data that is returned to the device.
fileNameStringThe name of the file that the device uploads.
uploadIdStringThe ID of this file upload task. You must pass this ID when you upload file shards.
offsetLongThe size of the uploaded part of the file, in bytes. This parameter is returned only when conflictStrategy is set to append and an incomplete upload of the file exists in the IoT Platform cloud.

Device uploads file shards

  • Request topic: /sys/${productKey}/${deviceName}/thing/file/upload/mqtt/send.
  • Response topic: /sys/${productKey}/${deviceName}/thing/file/upload/mqtt/send_reply.

Request data format:

  • The request has the following structure:Structure
    Structure ItemDescription
    Header LengthThe length of the byte array that corresponds to the JSON string in the request header. This field must be 2 bytes long, in big-endian format (most significant byte first).

    For example, if the length of the byte array after the header JSON string is converted using UTF-8 encoding is 87 in decimal (57 in hexadecimal), the most significant byte is 0x00 and the least significant byte is 0x57.

    Header String BytesThe byte array that corresponds to the JSON string in the request header. The encoding format is UTF-8. For more information, see the "Header JSON data format" section below.
    File Block BytesThe byte array of the current file shard. The bytes are ordered by their offset from the beginning of the file, from smallest to largest.
    CRC16/IBMThe checksum of the file shard. Only CRC16/IBM is supported. This field must be 2 bytes long, in little-endian format (least significant byte first).

    For example, if the checksum of the file shard is 0x0809, the least significant byte is 0x09 and the most significant byte is 0x08.

  • Header JSON data format:
    {
        "id":"123456",
        "params":{
            "uploadId":"03d9151e-6***",
            "offset":34344,
            "bSize":123455,
            "isComplete":true
        }
    }
Table 3. Request parameter description
ParameterTypeDescription
idString
The message ID. The ID must be a string-type number that is unique to the device. Valid values: 0 to 4294967295.
paramsObject
Business request parameters
uploadIdStringThe ID of the file upload task. This ID is returned when the device requests the file upload.
offsetLongThe total size of the uploaded file shards, in bytes.
bSizeLongThe size of the current file shard, in bytes.
  • For a shard that is not the last one, the size must be from 256 B to 131,072 B.
  • For the last shard, if the file size is known, the size must be from 1 B to 131,072 B. If the file size is unknown, the size must be from 0 B to 131,072 B.
isCompleteBooleanThis parameter is valid only when fileSize in the file upload request is -1, which indicates that the file size is unknown. This parameter specifies whether the current shard is the last shard of the file.
  • true: Yes. IoT Platform checks whether the uploaded file size exceeds 16 MB:
    • Not exceeded: If the file size is greater than 0, the file is uploaded successfully. Otherwise, an error message is returned stating that the file cannot be empty, and the file upload task is deleted.
    • If the size exceeds 16 MB, an error code is returned indicating that the file size exceeds 16 MB, and the uploaded file is deleted. For more information, see Error codes for file uploads.
  • false: No. This indicates that it is not the last file shard and more shards need to be uploaded.

Response data format:

Note If the size of a file with an unknown total size exceeds 16 MB during the upload process, IoT Platform returns the error code 78117, automatically cancels the upload task, and deletes the uploaded file.
{
    "id":"123456",
    "code":200,
    "message":"this is error msg when code is not 200",
    "data":{
        "uploadId":"03d9151e-6***",
        "offset":34344,
        "bSize":123455,
        "complete":true,
        "ficMode":"crc64",
        "ficValueClient":"0205d7***",
        "ficValueServer":"0205d7***"
    }
}
Table 4. Response parameter description
ParameterTypeDescription
idString
The message ID. The ID must be a string-type number that is unique to the device. Valid values: 0 to 4294967295.
codeInteger
The result code. A value of 200 indicates success. Other values indicate failure. For more information, see Error codes received by devices.
messageStringThe error message that is returned if the upload fails.
dataObject
The data that is returned to the device.
uploadIdString
The ID of this file upload task. You must pass this ID when you upload file shards.
offsetLong
The total size of the uploaded file shards, in bytes.
bSizeLong
The size of the current file shard, in bytes.
  • For a shard that is not the last one, the size must be from 256 B to 131,072 B.
  • For the last shard, if the file size is known, the size must be from 1 B to 131,072 B. If the file size is unknown, the size must be from 0 B to 131,072 B.
completeBooleanAfter the last shard is uploaded, the file upload is complete. This parameter is returned with a value of true.
  • If fileSize in the file upload request is greater than 0 (the file size is known), the upload is considered complete when the total size of uploaded shards equals the specified file size.
  • If fileSize in the file upload request is -1 (the file size is unknown), the upload is considered complete when isComplete in a shard upload request is set to true.
ficModeStringThe file integrity check mode. If this parameter was specified in the upload request, the returned value is crc64.
ficValueClientStringAfter the file upload is complete, this parameter returns the ficValue that was specified in the file upload request.
ficValueServerStringAfter the file upload is complete, this parameter returns the file integrity check value calculated by the IoT Platform cloud. If this value is the same as ficValueClient, the file was uploaded completely.

Device cancels a file upload

  • Request topic: /sys/${productKey}/${deviceName}/thing/file/upload/mqtt/cancel.
  • Response topic: /sys/${productKey}/${deviceName}/thing/file/upload/mqtt/cancel_reply.

Request data format:

{
    "id":"123456",
    "params":{
        "uploadId":"03d9151e-6***"
    }
}
Table 5. Request parameter description
ParameterTypeDescription
idString
The message ID. The ID must be a string-type number that is unique to the device. Valid values: 0 to 4294967295.
paramsObject
Business request parameters
uploadIdString
The ID of the file upload task. This ID is returned when the device requests the file upload.

Response data format:

{
    "id":"123456",
    "code":200,
    "message":"uploading task for upload-id xxxx does not exist.",
    "data":{
        "uploadId":"03d9151e-6***"
    }
}
Table 6. Response parameter description
ParameterTypeDescription
idString
The message ID. The ID must be a string-type number that is unique to the device. Valid values: 0 to 4294967295.
codeInteger
The result code. A value of 200 indicates success. Other values indicate failure. For more information, see Error codes received by devices.
messageString
The error message that is returned if the request fails.
dataObject
The data that is returned to the device.
uploadIdStringThe ID of the canceled file upload task.

References

For information about error codes that are returned to the device, see Error codes for file uploads.