File upload
The file upload feature of the SDK uses the HTTP/2 stream protocol to upload files to the Alibaba Cloud IoT Platform server.
- The feature supports multiple upload modes, such as creating a new file or overwriting an existing file.
- The feature lets you specify an upload length and resume an interrupted upload. You can configure the upload shard size (
part_len) based on your network bandwidth to improve bandwidth utilization.
This topic uses src/http2/http2_example_uploadfile.c as an example to demonstrate how to use the file upload feature.
1. Establish a connection with the cloud
Call IOT_HTTP2_UploadFile_Connect to establish an HTTP/2 connection. Specify the device certificate information and the server address or port number.
http2_upload_conn_info_t conn_info;
void *handle;
memset(&conn_info, 0, sizeof(http2_upload_conn_info_t));
conn_info.product_key = HTTP2_PRODUCT_KEY;
conn_info.device_name = HTTP2_DEVICE_NAME;
conn_info.device_secret = HTTP2_DEVICE_SECRET;
conn_info.url = HTTP2_ONLINE_SERVER_URL;
conn_info.port = HTTP2_ONLINE_SERVER_PORT;
handle = IOT_HTTP2_UploadFile_Connect(&conn_info, NULL);
if(handle == NULL) {
return -1;
}
The domain names and ports for each region are as follows. You must replace the asterisk (*) with the ProductKey of your device. For example, if the ProductKey is a1Ign******I, the corresponding URL and port are as follows:
#define HTTP2_ONLINE_SERVER_URL "a1I******vI.iot-as-http2.cn-shanghai.aliyuncs.com"
#define HTTP2_ONLINE_SERVER_PORT 443
*.iot-as-http2.cn-shanghai.aliyuncs.com:443 // China (Shanghai)
*.iot-as-http2.us-west-1.aliyuncs.com:443 // US (West)
*.iot-as-http2.us-east-1.aliyuncs.com:443 // US (East)
*.iot-as-http2.eu-central-1.aliyuncs.com:443 // Germany (Frankfurt)
*.iot-as-http2.ap-southeast-1.aliyuncs.com:443 // Singapore
*.iot-as-http2.ap-northeast-1.aliyuncs.com:443 // Japan (Tokyo)
2. File upload
Call IOT_HTTP2_UploadFile_Request to request a file upload. The sample program uses the UPLOAD_FILE_OPT_BIT_OVERWRITE mode to upload the file. In this mode, each upload overwrites the file on the cloud. This is an asynchronous operation. You can insert multiple upload requests into the internal queue.
http2_upload_params_t fs_params;
http2_upload_result_cb_t result_cb;
memset(&result_cb, 0, sizeof(http2_upload_result_cb_t));
result_cb.upload_completed_cb = upload_file_result;
result_cb.upload_id_received_cb = upload_id_received_handle;
memset(&fs_params, 0, sizeof(fs_params));
fs_params.file_path = argv[1]; /* The file name is passed in as a command-line argument. */
fs_params.opt_bit_map = UPLOAD_FILE_OPT_BIT_OVERWRITE;
ret = IOT_HTTP2_UploadFile_Request(handle, &fs_params, &result_cb, NULL);
if(ret < 0) {
return -1;
}
The sample program registers two callback functions. One is used to receive the upload result, and the other is used to receive the upload identifier (upload_id) from the cloud. The file upload operation is complete after the SDK calls the upload_file_result callback function. You can then proceed to the next step.
void upload_file_result(const char *file_path, int result, void *user_data)
{
upload_end++;
EXAMPLE_TRACE("=========== file_path = %s, result = %d, finish num = %d ===========", file_path, result, upload_end);
}
void upload_id_received_handle(const char *file_path, const char *upload_id, void *user_data)
{
EXAMPLE_TRACE("=========== file_path = %s, upload_id = %s ===========", file_path, upload_id);
if (upload_id != NULL) {
memcpy(g_upload_id, upload_id, strlen(upload_id));
}
}
- The device sends a request to the cloud to open a channel for the file upload.
[inf] on_frame_send_callback(143): [INFO] C ---------> S (HEADERS) stream_id [1] [inf] on_frame_send_callback(145): > :method: POST [inf] on_frame_send_callback(145): > :path: /stream/open/c/iot/sys/thing/file/upload [inf] on_frame_send_callback(145): > :scheme: https [inf] on_frame_send_callback(145): > x-auth-name: devicename [inf] on_frame_send_callback(145): > x-auth-param-client-id: *******D7vI.H2_FS01 [inf] on_frame_send_callback(145): > x-auth-param-signmethod: hmacsha1 [inf] on_frame_send_callback(145): > x-auth-param-product-key: a1IgnO***** [inf] on_frame_send_callback(145): > x-auth-param-device-name: H2_FS01 [inf] on_frame_send_callback(145): > x-auth-param-sign: 8d6b80749ed63823d********** [inf] on_frame_send_callback(145): > x-sdk-version: 301 [inf] on_frame_send_callback(145): > x-sdk-version-name: 3.0.1 [inf] on_frame_send_callback(145): > x-sdk-platform: c [inf] on_frame_send_callback(145): > content-length: 0 [inf] on_frame_send_callback(145): > x-file-name: upload1M [inf] on_frame_send_callback(145): > x-file-overwrite: 1 [inf] on_begin_headers_callback(393): [INFO] C <--------- S (HEADERS) stream_id [1] [inf] on_header_callback(363): < :status: 200 [inf] on_header_callback(363): < x-request-id: 11039195007******* [inf] on_header_callback(363): < x-next-append-position: 0 [inf] on_header_callback(363): < x-data-stream-id: DS1103919500******** [inf] on_header_callback(363): < x-file-upload-id: ULDS11039195******** [inf] on_header_callback(363): < x-response-status: 200 - The channel is opened. The device receives the file upload identifier from the cloud and invokes the user callback function.
upload_id_received_handle|037 :: =========== file_path = upload1M, upload_id = ULDS11039195008******** =========== - After the channel is opened, the device sends an HTTP/2 request to upload the file.
[inf] on_frame_send_callback(143): [INFO] C ---------> S (HEADERS) stream_id [3] [inf] on_frame_send_callback(145): > :method: POST [inf] on_frame_send_callback(145): > :path: /stream/send/c/iot/sys/thing/file/upload [inf] on_frame_send_callback(145): > :scheme: https [inf] on_frame_send_callback(145): > content-length: 1048576 [inf] on_frame_send_callback(145): > x-data-stream-id: DS1103919500******** [inf] on_frame_send_callback(145): > x-sdk-version: 301 [inf] on_frame_send_callback(145): > x-sdk-version-name: 3.0.1 [inf] on_frame_send_callback(145): > x-sdk-platform: c [inf] on_frame_send_callback(145): > x-file-upload-id: ULDS1103919500******** [dbg] http2_stream_node_search(168): stream node not exist, stream_id = 3 [inf] send_callback(63): send_callback data len 10249, session->remote_window_size=16777215! [inf] send_callback(72): send_callback data ends len = 10249! [dbg] http2_stream_node_search(168): stream node not exist, stream_id = 3 [inf] iotx_http2_client_send(563): nghttp2_session_send 0 [dbg] _http2_fs_part_send_sync(250): send len = 10240 [inf] send_callback(63): send_callback data len 10249, session->remote_window_size=16766975! [inf] send_callback(72): send_callback data ends len = 10249! [inf] iotx_http2_client_send(563): nghttp2_session_send 0 [dbg] _http2_fs_part_send_sync(250): send len = 20480 [inf] send_callback(63): send_callback data len 10249, session->remote_window_size=16756735! [inf] send_callback(72): send_callback data ends len = 10249! [inf] iotx_http2_client_send(563): nghttp2_session_send 0 [dbg] _http2_fs_part_send_sync(250): send len = 30720 [inf] send_callback(63): send_callback data len 10249, session->remote_window_size=16746495! [inf] send_callback(72): send_callback data ends len = 10249! [inf] iotx_http2_client_send(563): nghttp2_session_send 0 [dbg] _http2_fs_part_send_sync(250): send len = 40960 - The file upload is complete. The device waits for an acknowledgement from the cloud. The
x-next-append-positionin the acknowledgement indicates the size of the uploaded file.[inf] on_frame_recv_callback(196): on_frame_recv_callback, type = 8 [inf] on_frame_recv_callback(197): on_frame_recv_callback, stream_id = 3 [inf] on_frame_recv_callback(205): stream user data is not exist [inf] on_begin_headers_callback(393): [INFO] C <--------- S (HEADERS) stream_id [3] [inf] on_header_callback(363): < :status: 200 [inf] on_header_callback(363): < x-request-id: 11039195011******** [inf] on_header_callback(363): < x-next-append-position: 1048576 [inf] on_header_callback(363): < x-data-stream-id: DS110391950088******** [inf] on_header_callback(363): < x-response-status: 200 [inf] on_frame_recv_callback(196): [dbg] _http2_fs_part_send_sync(250): on_frame_recv_callback, type = 1 [inf] on_frame_recv_callback(197): on_frame_recv_callback, stream_id = 3 [inf] on_frame_recv_callback(205): send len = 1048576 [inf] _http2_fs_node_handle(350): file offset = 1048576 now - The SDK closes the file upload channel.
[inf] on_frame_send_callback(143): [INFO] C ---------> S (HEADERS) stream_id [5] [inf] on_frame_send_callback(145): > :method: POST [inf] on_frame_send_callback(145): > :path: /stream/close/c/iot/sys/thing/file/upload [inf] on_frame_send_callback(145): > :scheme: https [inf] on_frame_send_callback(145): > x-data-stream-id: DS1103919500******** [inf] on_frame_send_callback(145): > x-sdk-version: 301 [inf] on_frame_send_callback(145): > x-sdk-version-name: 3.0.1 [inf] on_frame_send_callback(145): > x-sdk-platform: c [dbg] http2_stream_node_search(168): stream node not exist, stream_id = 5 [inf] iotx_http2_client_send(563): nghttp2_session_send 0 [inf] on_begin_headers_callback(393): [INFO] C <--------- S (HEADERS) stream_id [5] [inf] on_header_callback(363): < :status: 200 [inf] on_header_callback(363): < x-request-id: 11039195021******** [inf] on_header_callback(363): < x-data-stream-id: DS1103919500********* [inf] on_header_callback(363): < x-file-crc64ecma: 694777069228********* [inf] on_header_callback(363): < x-response-status: 200 [inf] on_header_callback(363): < x-file-store-id: 101184
3. Disconnect
After all files are uploaded, call IOT_HTTP2_UploadFile_Disconnect to disconnect from the cloud.
ret = IOT_HTTP2_UploadFile_Disconnect(handle);
Feature API operations
The src/http2/http2_upload_api.h file provides all API operations and related data type definitions for HTTP/2 file upload.
The src/http2/http2_wrapper.h file provides the underlying interfaces required for HTTP/2.
Establish an HTTP/2 connection
Function prototype
void *IOT_HTTP2_UploadFile_Connect(http2_upload_conn_info_t *conn_info, http2_status_cb_t *cb);
Description
This function creates an HTTP/2 connection and registers the relevant status callback functions. This is a synchronous operation. If the connection is established, the function returns an HTTP/2 connection handle. Otherwise, it returns NULL.
Parameters
| Parameter | Data type | Direction | Description |
| conn_info | http2_upload_conn_info_t * | Input | Device connection information. |
| cb | http2_status_cb_t * | Input | A pointer to the struct of device status callback functions. |
Return values
| Value | Description |
| Not NULL | The connection is established. |
| NULL | The connection failed to be established. |
Additional parameter information
typedef struct {
char *product_key;
char *device_name;
char *device_secret;
char *url;
int port;
} http2_upload_conn_info_t;
- product_key: The ProductKey of the product.
- device_name: The name of the device.
- device_secret: The device's secret key.
- url: The address of the cloud server.
- port: The port of the cloud server.
typedef struct {
http2_disconnect_cb_t on_disconnect_cb;
http2_reconnect_cb_t on_reconnect_cb;
} http2_status_cb_t;
- on_disconnect_cb: The HTTP/2 disconnection callback function.
- on_reconnect_cb: The HTTP/2 reconnection callback function.
File upload request
Function prototype
int IOT_HTTP2_UploadFile_Request(void *http2_handle, http2_upload_params_t *params, http2_upload_result_cb_t *cb, void *user_data);
Description
This function uploads a file based on the specified parameter settings and registers the relevant result callback functions. This is an asynchronous operation. The upload result is returned using the callback function.
Parameters
| Parameter | Data type | Direction | Description |
| http2_handle | void * | Input | The handle returned after a connection is established by calling IOT_HTTP2_UploadFile_Connect. |
| params | http2_upload_params_t * | Input | A pointer to the struct of upload parameters. |
| cb | http2_upload_result_cb_t | Input | A pointer to the struct of upload result callback functions. |
| user_data | void * | Input | User data. |
Return values
| Value | Description |
| 0 | The function is called successfully. |
| < 0 | The function call failed. |
typedef struct {
const char *file_path; /* The file path. The file name must be an ASCII string and its length must be less than 2014 characters. */
uint32_t part_len; /* The maximum content length of one HTTP/2 request. The value must be in the range of 100 KB to 100 MB. */
const char *upload_id; /* A specific ID that indicates an upload session. This parameter is required only when the UPLOAD_FILE_OPT_BIT_RESUME option is set. */
uint32_t upload_len; /* The upload length. This parameter is required only when the UPLOAD_FILE_OPT_BIT_SPECIFIC_LEN option is set. */
uint32_t opt_bit_map; /* The option bit map. The supported options are UPLOAD_FILE_OPT_BIT_OVERWRITE, UPLOAD_FILE_OPT_BIT_RESUME, and UPLOAD_FILE_OPT_BIT_SPECIFIC_LEN. */
} http2_upload_params_t;
- file_path: The file path.Note The file name must be in ASCII encoding and cannot start with a digit.
- part_len: The size of the upload shard. This is the maximum length of
content_lenin an HTTP/2 request. The value must be in the range of 100 KB to 100 MB. If the value is outside this range, the default length specified inhttp2_config.his used. - upload_id: The upload identifier. This is returned after the first upload. To resume an interrupted upload, you must set the
opt_bit_mapparameter toUPLOAD_FILE_OPT_BIT_RESUMEand specify the corresponding upload identifier. - upload_len: The length of the current request. This parameter takes effect only when the
opt_bit_mapparameter is set toUPLOAD_FILE_OPT_BIT_SPECIFIC_LEN. - opt_bit_map: The bit-defined option table. You can use a bitwise OR operation to configure this option.
For example,
opt_bit_map = UPLOAD_FILE_OPT_BIT_OVERWRITE | UPLOAD_FILE_OPT_BIT_SPECIFIC_LENindicates that you want to upload a file of a specified length by overwriting the existing file.
#define UPLOAD_FILE_OPT_BIT_OVERWRITE (0x00000001)
#define UPLOAD_FILE_OPT_BIT_RESUME (0x00000002)
#define UPLOAD_FILE_OPT_BIT_SPECIFIC_LEN (0x00000004)
- UPLOAD_FILE_OPT_BIT_OVERWRITE: Uploads the file by overwriting the existing file. If the file already exists on the cloud and you do not use this option, the file upload fails. If you do not use this option, a new file is created for the upload.
- UPLOAD_FILE_OPT_BIT_RESUME: Resumes an interrupted upload. If you use this option, you must specify the
upload_idparameter. - UPLOAD_FILE_OPT_BIT_SPECIFIC_LEN: Uploads a file of a specified length. If you use this option, you must specify the upload length parameter (
upload_len). If you do not use this option, the entire file is uploaded.
typedef struct {
http2_upload_id_received_cb_t upload_id_received_cb;
http2_upload_completed_cb_t upload_completed_cb;
} http2_upload_result_cb_t;
- upload_id_received_cb: This callback function is invoked when the upload identifier is received from the cloud server.
- upload_completed_cb: This callback function is invoked when the file upload is complete. The result parameter indicates the upload result.
Disconnect an HTTP/2 connection
API Prototype
int IOT_HTTP2_UploadFile_Disconnect(void *handle);
Description
This function disconnects the HTTP/2 connection specified by the handle parameter.
Parameters
| Parameter | Data type | Direction | Description |
| http2_handle | void * | Input | The handle returned after a connection is established by calling IOT_HTTP2_UploadFile_Connect. |
Return values
| Value | Description |
| 0 | The function is called successfully. |
| < 0 | The function call failed. |
Required HAL functions
The src/http2/http2_wrapper.h file provides some of the Hardware Abstraction Layer (HAL) functions that you must adapt for HTTP/2 file upload.
| Function name | Description |
| HAL_SSL_Destroy | Destroys a TLS connection. This function is used for MQTT and HTTPS features. |
| HAL_SSL_Establish | Establishes a TLS connection. This function is used for MQTT and HTTPS features. |
| HAL_SSL_Read | Reads data from a TLS connection. This function is used for MQTT and HTTPS features. |
| HAL_SSL_Write | Writes data to a TLS connection. This function is used for MQTT and HTTPS features. |
| HAL_MutexCreate | Creates a mutex object. |
| HAL_MutexDestroy | Destroys a mutex object. |
| HAL_MutexLock | Locks a mutex. |
| HAL_MutexUnlock | Unlocks a mutex. |
| HAL_SemaphoreCreate | Creates a semaphore. |
| HAL_SemaphoreDestroy | Destroys a semaphore. |
| HAL_SemaphorePost | Posts a semaphore. |
| HAL_SemaphoreWait | Waits for a semaphore. |
| HAL_ThreadCreate | Creates a thread. |
| HAL_ThreadDelete | Destroys a thread. |
| HAL_ThreadDetach | Detaches a thread. |
| HAL_Fopen | Open the file. |
| HAL_Fread | Reads data from a file. |
| HAL_Fwrite | Writes data to a file. |
| HAL_Fseek | Sets the position of the file pointer stream. |
| HAL_Fclose | Closes a file. |
| HAL_Ftell | Gets the offset in bytes of the current file pointer position relative to the beginning of the file. |
| HAL_Printf | The print function. |
| HAL_SleepMs | The sleep function. |
| HAL_Malloc | Memory allocation. |
| HAL_Free | Releases memory. |