全部产品
云市场

文件上传

更新时间:2019-03-15 17:20:08

功能说明

SDK的文件上传功能使用HTTP2流式传输协议, 将文件上传至阿里云物联网平台服务器.

  • 支持多种上传模式, 如以创建文件的方式上传, 或以覆盖文件的方式上传
  • 支持指定上传长度, 并在下次上传时续传, 用户可在上传时根据网络带宽配置上传分配大小(part_len), 以提高带宽利用效率

本节以 src/http2/http2_example_uploadfile.c 为例讲解如何使用文件上传功能

1. 与云端建立连接

调用IOT_HTTP2_UploadFile_Connect建立HTTP2连接,用户指定设备的三元组信息和服务器地址/端口号。

  1. http2_upload_conn_info_t conn_info;
  2. void *handle;
  3. memset(&conn_info, 0, sizeof(http2_upload_conn_info_t));
  4. conn_info.product_key = HTTP2_PRODUCT_KEY;
  5. conn_info.device_name = HTTP2_DEVICE_NAME;
  6. conn_info.device_secret = HTTP2_DEVICE_SECRET;
  7. conn_info.url = HTTP2_ONLINE_SERVER_URL;
  8. conn_info.port = HTTP2_ONLINE_SERVER_PORT;
  9. handle = IOT_HTTP2_UploadFile_Connect(&conn_info, NULL);
  10. if(handle == NULL) {
  11. return -1;
  12. }

目前各个区域对应的域名和端口如下,其中*符号应使用设备的ProductKey替换,如ProductKeya1IgnOND7vI时对应的URL、PORT如下:

  1. #define HTTP2_ONLINE_SERVER_URL "a1IgnOND7vI.iot-as-http2.cn-shanghai.aliyuncs.com"
  2. #define HTTP2_ONLINE_SERVER_PORT 443
  1. *.iot-as-http2.cn-shanghai.aliyuncs.com:443 // 上海正式
  2. *.iot-as-http2.us-west-1.aliyuncs.com:443 // 美西正式
  3. *.iot-as-http2.us-east-1.aliyuncs.com:443 // 美东正式
  4. *.iot-as-http2.eu-central-1.aliyuncs.com:443 // 德国正式
  5. *.iot-as-http2.ap-southeast-1.aliyuncs.com:443 // 新加坡正式
  6. *.iot-as-http2.ap-northeast-1.aliyuncs.com:443 // 日本正式

如果用户关心网络状态,可以注册相应的回调函数,目前支持网络断开连接,和网络重连成功两个回调函数。

2. 文件上传

使用IOT_HTTP2_UploadFile_Request请求文件上传,例程以UPLOAD_FILE_OPT_BIT_OVERWRITE的方式上传,每次上传都会覆盖云端的文件。此接口为异步接口,用户可以插入多个上传请求到内部队列中。

  1. http2_upload_params_t fs_params;
  2. http2_upload_result_cb_t result_cb;
  3. memset(&result_cb, 0, sizeof(http2_upload_result_cb_t));
  4. result_cb.upload_completed_cb = upload_file_result;
  5. result_cb.upload_id_received_cb = upload_id_received_handle;
  6. memset(&fs_params, 0, sizeof(fs_params));
  7. fs_params.file_path = argv[1]; /* 文件名称以命令行参数传入 */
  8. fs_params.opt_bit_map = UPLOAD_FILE_OPT_BIT_OVERWRITE;
  9. ret = IOT_HTTP2_UploadFile_Request(handle, &fs_params, &result_cb, NULL);
  10. if(ret < 0) {
  11. return -1;
  12. }

例程中注册了2个回调函数, 分别用于接收上传的结果, 和接收云端返回的上传标示符(upload_id). 在SDK调用了upload_file_result后, 文件上传操作便结束了, 用户可进行下一步操作

  1. void upload_file_result(const char *file_path, int result, void *user_data)
  2. {
  3. upload_end++;
  4. EXAMPLE_TRACE("=========== file_path = %s, result = %d, finish num = %d ===========", file_path, result, upload_end);
  5. }
  6. void upload_id_received_handle(const char *file_path, const char *upload_id, void *user_data)
  7. {
  8. EXAMPLE_TRACE("=========== file_path = %s, upload_id = %s ===========", file_path, upload_id);
  9. if (upload_id != NULL) {
  10. memcpy(g_upload_id, upload_id, strlen(upload_id));
  11. }
  12. }

在上传过程中我们可以在log中看到HTTP2的报文交互:

  1. 设备端请求云端打开文件上传的通道:
    1. [inf] on_frame_send_callback(143): [INFO] C ---------> S (HEADERS) stream_id [1]
    2. [inf] on_frame_send_callback(145): > :method: POST
    3. [inf] on_frame_send_callback(145): > :path: /stream/open/c/iot/sys/thing/file/upload
    4. [inf] on_frame_send_callback(145): > :scheme: https
    5. [inf] on_frame_send_callback(145): > x-auth-name: devicename
    6. [inf] on_frame_send_callback(145): > x-auth-param-client-id: a1IgnOND7vI.H2_FS01
    7. [inf] on_frame_send_callback(145): > x-auth-param-signmethod: hmacsha1
    8. [inf] on_frame_send_callback(145): > x-auth-param-product-key: a1IgnOND7vI
    9. [inf] on_frame_send_callback(145): > x-auth-param-device-name: H2_FS01
    10. [inf] on_frame_send_callback(145): > x-auth-param-sign: 8d6b80749ed63823dc16b2c1e7f049bbdd00bf2b
    11. [inf] on_frame_send_callback(145): > x-sdk-version: 301
    12. [inf] on_frame_send_callback(145): > x-sdk-version-name: 3.0.1
    13. [inf] on_frame_send_callback(145): > x-sdk-platform: c
    14. [inf] on_frame_send_callback(145): > content-length: 0
    15. [inf] on_frame_send_callback(145): > x-file-name: upload1M
    16. [inf] on_frame_send_callback(145): > x-file-overwrite: 1
    17. [inf] on_begin_headers_callback(393): [INFO] C <--------- S (HEADERS) stream_id [1]
    18. [inf] on_header_callback(363): < :status: 200
    19. [inf] on_header_callback(363): < x-request-id: 1103919500797702144
    20. [inf] on_header_callback(363): < x-next-append-position: 0
    21. [inf] on_header_callback(363): < x-data-stream-id: DS1103919500889976832
    22. [inf] on_header_callback(363): < x-file-upload-id: ULDS1103919500889976832
    23. [inf] on_header_callback(363): < x-response-status: 200
  2. 通道打开成功,接收到云端返回的文件上传标示符并调用用户回调函数:
    1. upload_id_received_handle|037 :: =========== file_path = upload1M, upload_id = ULDS1103919500889976832 ===========
  3. 通道打开成功后,设备端通过HTTP2请求上传文件:
    1. [inf] on_frame_send_callback(143): [INFO] C ---------> S (HEADERS) stream_id [3]
    2. [inf] on_frame_send_callback(145): > :method: POST
    3. [inf] on_frame_send_callback(145): > :path: /stream/send/c/iot/sys/thing/file/upload
    4. [inf] on_frame_send_callback(145): > :scheme: https
    5. [inf] on_frame_send_callback(145): > content-length: 1048576
    6. [inf] on_frame_send_callback(145): > x-data-stream-id: DS1103919500889976832
    7. [inf] on_frame_send_callback(145): > x-sdk-version: 301
    8. [inf] on_frame_send_callback(145): > x-sdk-version-name: 3.0.1
    9. [inf] on_frame_send_callback(145): > x-sdk-platform: c
    10. [inf] on_frame_send_callback(145): > x-file-upload-id: ULDS1103919500889976832
    11. [dbg] http2_stream_node_search(168): stream node not exist, stream_id = 3
    12. [inf] send_callback(63): send_callback data len 10249, session->remote_window_size=16777215!
    13. [inf] send_callback(72): send_callback data ends len = 10249!
    14. [dbg] http2_stream_node_search(168): stream node not exist, stream_id = 3
    15. [inf] iotx_http2_client_send(563): nghttp2_session_send 0
    16. [dbg] _http2_fs_part_send_sync(250): send len = 10240
    17. [inf] send_callback(63): send_callback data len 10249, session->remote_window_size=16766975!
    18. [inf] send_callback(72): send_callback data ends len = 10249!
    19. [inf] iotx_http2_client_send(563): nghttp2_session_send 0
    20. [dbg] _http2_fs_part_send_sync(250): send len = 20480
    21. [inf] send_callback(63): send_callback data len 10249, session->remote_window_size=16756735!
    22. [inf] send_callback(72): send_callback data ends len = 10249!
    23. [inf] iotx_http2_client_send(563): nghttp2_session_send 0
    24. [dbg] _http2_fs_part_send_sync(250): send len = 30720
    25. [inf] send_callback(63): send_callback data len 10249, session->remote_window_size=16746495!
    26. [inf] send_callback(72): send_callback data ends len = 10249!
    27. [inf] iotx_http2_client_send(563): nghttp2_session_send 0
    28. [dbg] _http2_fs_part_send_sync(250): send len = 40960
  4. 文件上传结束,等待云端上传结构应答,应答中的x-next-append-position便是已上传文件的大小
    1. [inf] on_frame_recv_callback(196): on_frame_recv_callback, type = 8
    2. [inf] on_frame_recv_callback(197): on_frame_recv_callback, stream_id = 3
    3. [inf] on_frame_recv_callback(205): stream user data is not exist
    4. [inf] on_begin_headers_callback(393): [INFO] C <--------- S (HEADERS) stream_id [3]
    5. [inf] on_header_callback(363): < :status: 200
    6. [inf] on_header_callback(363): < x-request-id: 1103919501166800896
    7. [inf] on_header_callback(363): < x-next-append-position: 1048576
    8. [inf] on_header_callback(363): < x-data-stream-id: DS1103919500889976832
    9. [inf] on_header_callback(363): < x-response-status: 200
    10. [inf] on_frame_recv_callback(196): [dbg] _http2_fs_part_send_sync(250): on_frame_recv_callback, type = 1
    11. [inf] on_frame_recv_callback(197): on_frame_recv_callback, stream_id = 3
    12. [inf] on_frame_recv_callback(205): send len = 1048576
    13. [inf] _http2_fs_node_handle(350): file offset = 1048576 now
  5. 最后SDK会关闭文件上传通道:
    1. [inf] on_frame_send_callback(143): [INFO] C ---------> S (HEADERS) stream_id [5]
    2. [inf] on_frame_send_callback(145): > :method: POST
    3. [inf] on_frame_send_callback(145): > :path: /stream/close/c/iot/sys/thing/file/upload
    4. [inf] on_frame_send_callback(145): > :scheme: https
    5. [inf] on_frame_send_callback(145): > x-data-stream-id: DS1103919500889976832
    6. [inf] on_frame_send_callback(145): > x-sdk-version: 301
    7. [inf] on_frame_send_callback(145): > x-sdk-version-name: 3.0.1
    8. [inf] on_frame_send_callback(145): > x-sdk-platform: c
    9. [dbg] http2_stream_node_search(168): stream node not exist, stream_id = 5
    10. [inf] iotx_http2_client_send(563): nghttp2_session_send 0
    11. [inf] on_begin_headers_callback(393): [INFO] C <--------- S (HEADERS) stream_id [5]
    12. [inf] on_header_callback(363): < :status: 200
    13. [inf] on_header_callback(363): < x-request-id: 1103919502177628160
    14. [inf] on_header_callback(363): < x-data-stream-id: DS1103919500889976832
    15. [inf] on_header_callback(363): < x-file-crc64ecma: 6947770692288575170
    16. [inf] on_header_callback(363): < x-response-status: 200
    17. [inf] on_header_callback(363): < x-file-store-id: 101184

    3. 断开连接

    所有文件上传结束后使用IOT_HTTP2_UploadFile_Disconnect断开云端连接。
    1. ret = IOT_HTTP2_UploadFile_Disconnect(handle);

功能API接口

src/http2/http2_upload_api.h列出了HTTP2文件上传的所有API和相关数据类型定义

src/http2/http2_wrapper.h列出了HTTP2所需的底层接口

HTTP2建立连接

接口原型

  1. void *IOT_HTTP2_UploadFile_Connect(http2_upload_conn_info_t *conn_info, http2_status_cb_t *cb);

接口说明

创建HTTP2连接, 并注册相关状态回调函数. 此接口为同步接口, 当建连成功后会返回HTTP2连接句柄. 否则返回NULL

参数说明

参数 数据类型 方向 说明
conn_info http2_upload_conn_info_t * 输入 设备连接信息
cb http2_status_cb_t * 输入 设备状态回调函数结构体指针

返回值说明

说明
非NULL 建连成功
NULL 建连失败

参数附加说明

  1. typedef struct {
  2. char *product_key;
  3. char *device_name;
  4. char *device_secret;
  5. char *url;
  6. int port;
  7. } http2_upload_conn_info_t;
  • product_key: 三元组之一, 产品标示符
  • device_name: 三元组之一, 设备名称
  • device_secret: 三元组之一, 识别秘钥
  • url: 云端服务器地址
  • port: 云端服务器端口
  1. typedef struct {
  2. http2_disconnect_cb_t on_disconnect_cb;
  3. http2_reconnect_cb_t on_reconnect_cb;
  4. } http2_status_cb_t;
  • on_disconnect_cb: HTTP2断连回调函数
  • on_reconnect_cb: HTTP2重连回调函数

文件上传请求

接口原型

  1. int IOT_HTTP2_UploadFile_Request(void *http2_handle, http2_upload_params_t *params, http2_upload_result_cb_t *cb, void *user_data);

接口说明

按照指定参数配置上传文件, 并注册相关结果回调函数. 此接口为异步接口, 上传结果由回调函数返回

参数说明

参数 数据类型 方向 说明
http2_handle void * 输入 调用IOT_HTTP2_UploadFile_Connect建连成功后返回的句柄
params http2_upload_params_t * 输入 上传参数结构体指针
cb http2_upload_result_cb_t 输入 上传结构回调函数结构体指针
user_data void * 输入 用户数据

返回值说明

说明
0 函数调用成功
< 0 函数调用失败
  1. typedef struct {
  2. const char *file_path; /* file path, filename must be ASCII string and strlen < 2014 */
  3. uint32_t part_len; /* maximum content len of one http2 request, must be in range of 100KB ~ 100MB */
  4. const char *upload_id; /* a specific id used to indicate one upload session, only required when UPLOAD_FILE_OPT_BIT_RESUME option set */
  5. uint32_t upload_len; /* used to indicate the upload length, only required when UPLOAD_FILE_OPT_BIT_SPECIFIC_LEN option set */
  6. uint32_t opt_bit_map; /* option bit map, support UPLOAD_FILE_OPT_BIT_OVERWRITE, UPLOAD_FILE_OPT_BIT_RESUME and UPLOAD_FILE_OPT_BIT_SPECIFIC_LEN */
  7. } http2_upload_params_t;
  • file_path: 文件路径, 注意文件名必须为ASCII编码, 且不能使用数字开头
  • part_len: 文件上传分片大小, 也即HTTP2请求content_len的最大长度, 必须在100KB ~ 100MB范围内, 否则会使用http2_config.h里的默认长度
  • upload_id: 上传标示符, 由首次上传时返回. 在需要使用断点续传方式上传时需添加opt_bit_map参数UPLOAD_FILE_OPT_BIT_RESUME, 并指定对应上传标示符
  • upload_len: 指定本次请求的长度, 仅在opt_bit_map参数中有配置UPLOAD_FILE_OPT_BIT_SPECIFIC_LEN时才能起作用
  • opt_bit_map: 位定义的选项表, 可以使用按位或的方式配置此选项

    • opt_bit_map = UPLOAD_FILE_OPT_BIT_OVERWRITE | UPLOAD_FILE_OPT_BIT_SPECIFIC_LEN表示使用覆盖方式上传指定的文件长度
  1. #define UPLOAD_FILE_OPT_BIT_OVERWRITE (0x00000001)
  2. #define UPLOAD_FILE_OPT_BIT_RESUME (0x00000002)
  3. #define UPLOAD_FILE_OPT_BIT_SPECIFIC_LEN (0x00000004)
  • UPLOAD_FILE_OPT_BIT_OVERWRITE: 使用覆盖的方式上传文件. 如果云端文件已存在, 而未使用覆盖方式, 则文件上传会失败. 未使用此选项, 将使用创建文件的方式上传文件
  • UPLOAD_FILE_OPT_BIT_RESUME: 使用断点续传的方式上传文件. 使用此选项需填写上传标示符参数(upload_id)
  • UPLOAD_FILE_OPT_BIT_SPECIFIC_LEN: 使用指定长度的方式上传. 使用此选项需要填写上传长度参数(upload_len). 否则无需填写upload_len, 将上传整个文件
  1. typedef struct {
  2. http2_upload_id_received_cb_t upload_id_received_cb;
  3. http2_upload_completed_cb_t upload_completed_cb;
  4. } http2_upload_result_cb_t;
  • upload_id_received_cb: 接收到云端服务器返回的上传标示符时, 将调用此回调函数
  • upload_completed_cb: 文件上传结束时, 将调用此回调函数, result参数指示了上传结果

HTTP2断开连接

接口原型

  1. int IOT_HTTP2_UploadFile_Disconnect(void *handle);

接口说明

断开参数handle指定的HTTP2连接

参数说明

参数 数据类型 方向 说明
http2_handle void * 输入 调用IOT_HTTP2_UploadFile_Connect建连成功后返回的句柄

返回值说明

说明
0 函数调用成功
< 0 函数调用失败

需要对接的HAL接口

文件src/http2/http2_wrapper.h中包含了用户对接HTTP2文件上传需要适配的部分HAL接口

函数名 说明
HAL_SSL_Destroy 销毁一个TLS连接, 用于MQTT功能, HTTPS功能
HAL_SSL_Establish 建立一个TLS连接, 用于MQTT功能, HTTPS功能
HAL_SSL_Read 从一个TLS连接中读数据, 用于MQTT功能, HTTPS功能
HAL_SSL_Write 向一个TLS连接中写数据, 用于MQTT功能, HTTPS功能
HAL_MutexCreate 创建一个互斥量对象
HAL_MutexDestroy 销毁一个互斥量对象
HAL_MutexLock 锁住一个互斥量
HAL_MutexUnlock 解锁一个互斥量
HAL_SemaphoreCreate 创建信号量
HAL_SemaphoreDestroy 销毁信号量
HAL_SemaphorePost post信号量
HAL_SemaphoreWait 等待信号量
HAL_ThreadCreate 创建线程
HAL_ThreadDelete 销毁线程
HAL_ThreadDetach 分离线程
HAL_Fopen 打开文件
HAL_Fread 读取文件数据
HAL_Fwrite 向文件写入数据
HAL_Fseek 设置文件指针stream的位置
HAL_Fclose 关闭文件
HAL_Ftell 得到文件位置指针当前位置相对于文件首的偏移字节数
HAL_Printf 打印函数
HAL_SleepMs 睡眠函数
HAL_Malloc 内存分配
HAL_Free 内存释放