RTOS C SDK (License mode)

Prerequisites

• Hardware compatibility list

1. Access mode description

Semi-managed mode

This mode is designed for customers who require post-sales operations or device upgrades. In this mode, customers use their own cloud services to manage and authenticate devices, and a bidirectional communication channel exists between the customer's cloud service and the device.

• Server-side development:

  • Complete the server-side API integration. For more information, see the Cloud API Reference section. The Model Studio device metering and management service provides two APIs: device registration and token acquisition.

• Device-side development

  • Adapt the chip. For more information, see the Chip Platform HAL Integration section. If you use a chip or module recommended by Alibaba Cloud, you can skip this step.

  • After you integrate the SDK, complete the business logic development. For more information, see the Device-side business logic integration section. The device is registered based on the one-type-one-secret principle and is pre-provisioned with the AppId and AppSecret (License key) that are generated when the application is created.

Fully managed mode

This mode is designed for customers who sell hardware as a one-time transaction and do not require post-sales operations. These customers do not have their own cloud services for device management. Alibaba Cloud handles all device management and authentication.

• Server-side development:

  • None

• Device-side development

  • Adapt the chip. For more information, see the Chip Platform HAL Layer Integration section. If you use a chip or module recommended by Alibaba Cloud, you can skip this step.

  • After you integrate the SDK, complete the business logic development. For more information, see Device-side business logic integration. Register the device using the one-model-one-secret method and pre-provision it with the AppId and AppSecret (License key) that are generated when you create the application.

2. Server-side API development guide

Import dependency packages

Domain name: bailianmodelonchip.cn-beijing.aliyuncs.com

Region: cn-beijing

Version: Use the latest version.

• Query address: https://mvnrepository.com/artifact/com.aliyun/bailianmodelonchip20240816

<dependency>
  <groupId>com.aliyun</groupId>
  <artifactId>bailianmodelonchip20240816</artifactId>
  <version>${version}</version>
</dependency>

Example:
<dependency>
  <groupId>com.aliyun</groupId>
  <artifactId>bailianmodelonchip20240816</artifactId>
  <version>1.4.0</version>
</dependency>

Python version

pip install alibabacloud_bailianmodelonchip20240816==1.4.0alibabacloud_bailianmodelonchip20240816

2.1 Device registration interface: deviceRegister

When the device connects to the network for the first time, it must call this interface through the customer's cloud service to register with the Model Studio device metering and management service and obtain the device trituple information. Authentication is performed using the Alibaba Cloud POP gateway and requires the integration of the POP SDK.

Input parameter: DeviceRegisterRequest object

Response parameter: DeviceRegisterResponseBody object

DeviceRegisterResponseBodyData

Java example

package pop;

import com.alibaba.fastjson.JSON;
import com.aliyun.bailianmodelonchip20240816.Client;
import com.aliyun.bailianmodelonchip20240816.models.*;
import com.aliyun.teaopenapi.models.Config;

public class DeviceTest {

    public static void main(String[] args) {
        Config config = new Config();
        config.setAccessKeyId("your-ak");
        config.setAccessKeySecret("your-as");
        config.setEndpoint("bailianmodelonchip.cn-beijing.aliyuncs.com");

        try {
            Client client = new Client(config);
            deviceRegister(client);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    public static void deviceRegister(Client client) throws Exception {
        DeviceRegisterRequest request = new DeviceRegisterRequest();
        request.setNonce("Your Nonce");
        request.setRequestTime("1748312026868");
        request.setAppId("Your AppId");
        request.setSignature("Your Signature");

        DeviceRegisterResponse response = client.deviceRegister(request);
        DeviceRegisterResponseBody.DeviceRegisterResponseBodyData data = response.getBody().getData();
        
        // Pass the data to the device in JSON format. The vendor must implement this.
        System.out.println("Pass-through result: " + JSON.toJSONString(data));
    }
}

Python example

import sys
from typing import List

from Tea.exceptions import UnretryableException, TeaException
from alibabacloud_bailianmodelonchip20240816 import models as bailian_model_on_chip_20240816_models
from alibabacloud_bailianmodelonchip20240816.client import Client as BailianModelOnChip20240816Client
from alibabacloud_tea_openapi import models as open_api_models
from alibabacloud_tea_util import models as util_models


class PopDeviceRegister:
    def __init__(self):
        pass

    @staticmethod
    def create_client() -> BailianModelOnChip20240816Client:
        config = open_api_models.Config(
            access_key_id='your-ak',
            access_key_secret='your-as'
        )

        config.endpoint = f'bailianmodelonchip.cn-beijing.aliyuncs.com'
        return BailianModelOnChip20240816Client(config)

    @staticmethod
    def main(
        args: List[str],
    ) -> None:
        client = PopDeviceRegister.create_client()
        device_register_request = bailian_model_on_chip_20240816_models.DeviceRegisterRequest(
            nonce='Your Nonce',
            request_time='1750313647162',
            app_id='Your AppId',
            signature='Your Signature'
        )
        headers = {}
        try:
            # Print the API return value when you run the code.
            res = client.device_register_with_options(device_register_request, headers, util_models.RuntimeOptions())
            print('body', res.body)
            if res.body.success:
                print('data', res.body.data)
        except UnretryableException as e:
            # Network exception
            print(e)
        except TeaException as e:
            # Business exception
            print(e)
        except Exception as e:
            # Other exceptions
            print(e)

if __name__ == '__main__':
    PopDeviceRegister.main(sys.argv[1:])

2.2 Interface to get the access token for service interaction: getToken

After a device is registered, it must dynamically obtain an access token to use the multi-modal interaction capabilities. You can use this interface to retrieve the token from the Model Studio device metering and management service. Authentication is performed using the Alibaba Cloud POP gateway and requires the integration of the POP SDK.

Input parameter:

GetTokenRequest

Response parameter

GetTokenResponseBody

GetTokenResponseBodyData

Java example

package pop;

import com.alibaba.fastjson.JSON;
import com.aliyun.bailianmodelonchip20240816.Client;
import com.aliyun.bailianmodelonchip20240816.models.*;
import com.aliyun.teaopenapi.models.Config;

public class TokenTest {

    public static void main(String[] args) {
        Config config = new Config();
        config.setAccessKeyId("your-ak");
        config.setAccessKeySecret("your-as");
        config.setEndpoint("bailianmodelonchip.cn-beijing.aliyuncs.com");

        try {
            Client client = new Client(config);
            getToken(client);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    public static void getToken(Client client) throws Exception {
        GetTokenRequest request = new GetTokenRequest();
        request.setNonce("Your Nonce");
        request.setRequestTime("1748570866553");
        request.setAppId("Your AppId");
        request.setDeviceName("Your DeviceName");
        request.setTokenType("MMI");
        request.setTokenKey("Your TokenKey");
        request.setSignature("Your Signature");

        GetTokenResponse response = client.getToken(request);
        GetTokenResponseBody.GetTokenResponseBodyData data = response.getBody().getData();
        
        // Pass the data to the device in JSON format. The vendor must implement this.
        System.out.println("Execution result: " + JSON.toJSONString(data));
    }
}

Python example

import sys
from typing import List

from Tea.exceptions import UnretryableException, TeaException
from alibabacloud_bailianmodelonchip20240816 import models as bailian_model_on_chip_20240816_models
from alibabacloud_bailianmodelonchip20240816.client import Client as BailianModelOnChip20240816Client
from alibabacloud_tea_openapi import models as open_api_models
from alibabacloud_tea_util import models as util_models


class PopGetToken:
    def __init__(self):
        pass

    @staticmethod
    def create_client() -> BailianModelOnChip20240816Client:
        config = open_api_models.Config(
            access_key_id='your-ak',
            access_key_secret='your-as'
        )

        config.endpoint = f'bailianmodelonchip.cn-beijing.aliyuncs.com'
        return BailianModelOnChip20240816Client(config)

    @staticmethod
    def main(
        args: List[str],
    ) -> None:
        client = PopGetToken.create_client()
        get_token_request = bailian_model_on_chip_20240816_models.GetTokenRequest(
            nonce='Your Nonce',
            request_time='1750313823168',
            app_id='Your AppId',
            device_name='Your DeviceName',
            token_type='MMI',
            token_key='Your Token Key',
            signature='Your Signature'
        )
        headers = {}
        try:
            # Print the API return value when you run the code.
            res = client.get_token_with_options(get_token_request, headers, util_models.RuntimeOptions())
            print('body', res.body)
            if res.body.success:
                print('data', res.body.data)
        except UnretryableException as e:
            # Network exception
            print(e)
        except TeaException as e:
            # Business exception
            print(e)
        except Exception as e:
            # Other exceptions
            print(e)

if __name__ == '__main__':
    PopGetToken.main(sys.argv[1:])

2.3 Server-side error codes

3. Device-side interface guide

3.1. Get the SDK

Different chip platforms require compilation with their corresponding toolchains. Model Studio supports the chips in the following list, and you can download the SDK for them directly.

If you are using a new chip platform, contact Alibaba Cloud sales for technical support.

SDK folder structure

├── ReleaseNote.md
├── include
│   ├── c_agent
│   │   └── ...
│   ├── c_mmi_cmd
│   │   └── ...
│   ├── c_visual
│   │   └── ...
│   ├── c_utils
│   │   ├── c_utils.h
│   │   └── ...
│   ├── c_mmi.h
│   ├── lib_c_license.h
│   ├── lib_c_sdk.h
│   ├── qwen_test.h
│   └── ...
├── libs
│   ├── libc_license.a
│   ├── libc_no_license.a
│   ├── libc_mmi_cmd.a
│   ├── libhal_dummy.a
│   ├── libqwen_sdk.a
│   ├── libqwen_test.a
│   └── ...
└── third_party
    ├── cJSON
    │   ├── cJSON.h
    │   └── libcjson.a
    └── tinycrypt
        ├── include
        │   └── ...
        └── libtinycrypt.a

3.2. Vendor completes HAL layer development

The SDK abstracts a Hardware Abstraction Layer (HAL). Vendors must complete the corresponding development on their own chip platforms.

3.2.1. Memory

/**
 * util_malloc - Allocates a block of memory of a specified size.
 * @size: The size of the memory to allocate, in bytes.
 *
 * This function calls the standard library function malloc to allocate memory.
 * It may include additional error checking or memory management strategies to improve
 * program stability and performance.
 * 
 * @return Returns a pointer to the allocated memory. Returns NULL if memory allocation fails.
 */
void * util_malloc(int32_t size);

/**
 * Frees dynamically allocated memory.
 * 
 * This function is intended to release memory space previously obtained through dynamic allocation
 * to avoid memory leaks. It accepts a pointer to a dynamically allocated memory region and sets it
 * to NULL to prevent dangling pointers.
 * 
 * @param ptr A pointer to the dynamically allocated memory region. If NULL, the function does nothing.
 *            After the memory is freed, this pointer will be set to NULL.
 */
void util_free(void *ptr);

3.2.2. Random number module

/**
 * Initializes the random number generator.
 * 
 * @param seed The seed value used to initialize the random number generator.
 * 
 * @return Returns the initialization result. 0 indicates success, non-zero indicates failure.
 * 
 * This function initializes the random number generator to ensure that the subsequent
 * generated random number sequence has good randomness. The choice of seed value has a
 * significant impact on the generated random number sequence. The same seed value will

 * produce the same random number sequence.
 */
int32_t util_random_init(uint32_t seed);

/**
 * Generates a random number.
 * 
 * @return Returns the generated random number.
 * 
 * Before calling this function, ensure that the random number generator has been
 * successfully initialized with the util_random_init function. The random number
 * generated by this function is based on the seed provided during initialization.
 */
uint32_t util_random(void);

3.2.3. Storage module

/**
 * @brief Erases the storage.
 * 
 * This function is used to erase all data in the storage. Before calling this function, ensure that
 * no information in the storage is needed, as the erase operation will delete all data and is irreversible.
 * 
 * @return int32_t Returns the result of the erase operation. 0 indicates success; a non-zero value indicates an error.
 */
int32_t util_storage_erase(void);

/**
 * @brief Stores data to the storage.
 * 
 * This function stores the specified data into the storage. Before calling this function, ensure the
 * correctness and integrity of the data, as the storage operation will overwrite existing data in the storage.
 * 
 * @param data A pointer to the data to be stored. The data type is uint8_t (unsigned 8-bit integer).
 * @param size The size of the data to be stored, in bytes. The data type is uint32_t (unsigned 32-bit integer).
 * @return int32_t Returns the result of the storage operation. 0 indicates success; a non-zero value indicates an error.
 */
int32_t util_storage_storage(uint8_t *data, uint32_t size);

/**
 * @brief Loads data from the storage.
 * 
 * This function loads data of a specified size from the storage. Before calling this function, ensure that
 * the memory area pointed to by the data pointer is large enough to accommodate the data loaded from the storage.
 * 
 * @param data A pointer to the buffer used to store the data loaded from the storage. The data type is uint8_t (unsigned 8-bit integer).
 * @param size The size of the data to be loaded, in bytes. The data type is uint32_t (unsigned 32-bit integer).
 * @return int32_t Returns the result of the load operation. 0 indicates success; a non-zero value indicates an error.
 */
int32_t util_storage_load(uint8_t *data, uint32_t size);

3.2.4. Time module

/**
 * Gets the current timestamp in milliseconds.
 * 
 * This function is used to get the current timestamp, accurate to the millisecond.
 * This timestamp is typically used for calculating time differences, recording event
 * occurrence times, and other similar scenarios.
 * 
 * @return The current timestamp in milliseconds.
 */
int64_t util_now_ms(void);

/**
 * Millisecond-level sleep function.
 * 
 * This function pauses the current thread for a specified number of milliseconds.
 * It is used to control program execution pace, wait for events to occur, etc.
 * 
 * @param ms The number of milliseconds to pause.
 */
void util_msleep(uint32_t ms);

/**
 * Gets the current timestamp.
 * 
 * This function is used to get the current timestamp, which is the number of milliseconds
 * since 00:00:00 UTC on January 1, 1970. It takes no input parameters and returns an
 * int64_t value representing the current timestamp.
 * 
 * @return int64_t The current timestamp in milliseconds.
 */
int64_t util_get_timestamp(void);

/**
 * Checks if the timestamp function has been initialized.
 * 
 * This function is used to check if the timestamp-related functions have been initialized.
 * If it returns true (non-zero), the timestamp function is available. If it returns false (zero),
 * you may need to perform initialization or avoid using the timestamp function.
 * 
 * @return Returns non-zero if the timestamp function is initialized, otherwise returns zero.
 */
uint8_t util_timestamp_inited(void);

3.2.5. Mutex module

/* Mutex struct definition */
typedef struct _util_mutex_t {
    void *mutex_handle; /* Mutex handle, specific implementation depends on the platform */
} util_mutex_t;

/*****************************************************
 * Function: util_mutex_create
 * Description: Creates a mutex object.
 * Parameter: None.
 * Return: util_mutex_t * --- Returns a pointer to the mutex struct.
 ****************************************************/
util_mutex_t * util_mutex_create(void);

/*****************************************************
 * Function: util_mutex_delete
 * Description: Deletes the specified mutex object.
 * Parameter:
 *     mutex --- A pointer to the mutex struct.
 * Return: None.
 ****************************************************/
void util_mutex_delete(util_mutex_t *mutex);

/*****************************************************
 * Function: util_mutex_lock
 * Description: Locks the specified mutex with a timeout mechanism.
 * Parameter:
 *     mutex --- A pointer to the mutex struct.
 *     timeout --- The lock wait timeout in milliseconds (ms). Can be set to MUTEX_WAIT_FOREVER for an infinite wait.
 * Return: int32_t --- Returns the operation result (util_result_t).
 ****************************************************/
int32_t util_mutex_lock(util_mutex_t *mutex, int32_t timeout);

/*****************************************************
 * Function: util_mutex_unlock
 * Description: Unlocks the specified mutex.
 * Parameter:
 *     mutex --- A pointer to the mutex struct.
 * Return: int32_t --- Returns the operation result (util_result_t).
 ****************************************************/
int32_t util_mutex_unlock(util_mutex_t *mutex);

3.2.6. HAL layer porting acceptance criteria

After you implement the modules, load `libqwen_test.a` and call the `qwen_sdk_test` interface directly in the main program. This tests the modules, and you can view the test results in the output log.

Provide the output log to Alibaba Cloud for confirmation.

Example of a successful test output log:

3.3. Device-side business logic integration

# Linker flags
LDFLAGS += -L./YourLibPath \
          -Wl,--whole-archive \
          -lc_license \
          -Wl,--no-whole-archive \

3.3.1. Initialization interface

/**
 * @brief Initializes the MMI SDK.
 * 
 * @return int32_t Returns the operation result. 0 indicates success, non-zero indicates failure.
 */
int32_t c_mmi_sdk_init(void);

/**
 * @brief Configures MMI module parameters.
 *
 * This function is used to initialize the configuration parameters of the MMI module,
 * including event callback, work mode, text mode, voice settings, audio stream mode,
 * and buffer size.
 * 
 * @param config A pointer to the mmi_user_config_t struct, which contains the configuration parameters.
 * @return int32_t Returns the operation result. 0 indicates success, non-zero indicates failure.
 */
int32_t c_mmi_config(mmi_user_config_t *config);

# Linker flags
LDFLAGS += -L./YourLibPath \
          -Wl,--whole-archive \
          -lc_license \
          -Wl,--no-whole-archive \

Code example 1

#include "c_mmi.h"

mmi_user_config_t mmi_config = C_MMI_CONFIG_DEFAULT();

// Initialize the SDK
c_mmi_sdk_init();

// You must configure evt_cb, otherwise the SDK will not run correctly.
mmi_config.evt_cb = _mmi_event_callback;  // Register the event callback function, see details below.
// Configure the work mode
mmi_config.work_mode = C_MMI_MODE_PUSH2TALK;
mmi_config.text_mode = C_MMI_TEXT_MODE_BOTH;
// Configure the upstream and downstream audio data format
mmi_config.upstream_mode = C_MMI_STREAM_MODE_PCM;
mmi_config.downstream_mode = C_MMI_STREAM_MODE_MP3;
// Configure the buffer size
mmi_config.recorder_rb_size = 8 * 1024;
mmi_config.player_rb_size = 8 * 1024;

c_mmi_config(&mmi_config);
// Set the voice ID. This must be called after c_mmi_config.
c_mmi_set_voice_id("longxiaochun_v2");

3.3.2. Product information configuration interface

/**
 * @brief Checks if the device is registered.
 * 
 * @return uint8_t Device registration status.
 *         - 0: Device is not registered.
 *         - 1: Device is registered.
 */
uint8_t c_license_device_is_registered(void);

/**
 * @brief Resets the configuration.
 * 
 * This function clears all saved device configuration information.
 * After calling this function, the configuration will be restored to the default state,
 * and you will need to set the relevant parameters again.
 * 
 * @return int32_t Returns the operation result. 0 indicates success, non-zero indicates failure.
 */
int32_t c_mmi_storage_reset(void);

/**
 * @brief Saves the configuration.
 * 
 * @return int32_t Returns the operation result. 0 indicates success, non-zero indicates failure.
 */
int32_t c_mmi_storage_save(void);

/**
 * @brief Sets the AppId.
 * 
 * This function is used to set the AppId. After setting, you need to call c_mmi_storage_save to save it.
 * 
 * @param app_id_str The AppId, issued by Alibaba Cloud, in string format.
 * @return int32_t Returns the operation result. 0 indicates success, non-zero indicates failure.
 */
int32_t c_mmi_storage_set_app_id_str(char *app_id_str);

/**
 * @brief Sets the ApiKey.
 * 
 * This function is used to set the ApiKey. After setting, you need to call c_mmi_storage_save to save it.
 * 
 * @param api_key The API key obtained from the Model Studio platform.
 * @return int32_t Returns the operation result. 0 indicates success, non-zero indicates failure.
 */
int32_t c_mmi_storage_set_api_key(char *api_key);

/**
 * @brief Sets the WorkSpaceId.
 * 
 * This function is used to set the WorkSpaceId. After setting, you need to call c_mmi_storage_save to save it.
 * 
 * @param ws_id The WorkSpaceId, obtained from the Model Studio platform.
 * @return int32_t Returns the operation result. 0 indicates success, non-zero indicates failure.
 */
int32_t c_mmi_storage_set_ws_id(char *ws_id);

/**
 * @brief Sets the device name (DeviceName).
 * 
 * This function is used to set the device name. After setting, you need to call c_mmi_storage_save to save it.
 * 
 * @param dn The device name, which can be set by the user. The length must not exceed 32 characters.
 * @return int32_t Returns the operation result. 0 indicates success, non-zero indicates failure.
 */
int32_t c_mmi_storage_set_device_name(char *device_name);

/**
 * @brief Sets the AppSecret.
 * This function is used to set the AppSecret. After setting, you need to call c_mmi_storage_save to save it.
 * 
 * @param app_secret The AppSecret, issued by Alibaba Cloud, in string format.
 * @return int32_t Returns the operation result. 0 indicates success, non-zero indicates failure.
 */
int32_t c_license_set_app_secret_str(char *app_secret);

/**
 * @brief Initializes MMI.
 * 
 * This function is used to initialize MMI. It only needs to be called in pay-as-you-go mode.
 * 
 * @param workspace The WorkSpaceId, obtained from the Model Studio platform.
 * @param app_id The AppId, issued by Alibaba Cloud, in string format.
 * @param api_key The API key obtained from the Model Studio platform.
 * 
 * @return int32_t Returns the operation result. 0 indicates success, non-zero indicates failure.
 */
int32_t c_mmi_init(char *workspace, char *app_id, char *api_key);

Code example 2.1: Pay-as-you-go mode configuration

#include "c_mmi.h"

// Pre-configure the AppId
c_mmi_storage_set_app_id_str("Your AppId");
// Pre-configure the WorkspaceId
c_mmi_storage_set_ws_id("Your WorkspaceId");
// Pre-configure the ApiKey
c_mmi_storage_set_api_key("Your ApiKey");
// Pre-configure the DeviceName
c_mmi_set_device_name("Your DeviceName");
// Save the configuration information
c_mmi_storage_save();
// Initialize MMI
c_mmi_init("Your WorkspaceId", "Your AppId", "Your ApiKey");

Code example 2.2: License mode configuration

#include "c_mmi.h"
#include "lib_c_license.h"

if (c_license_device_is_registered() == 0) {
  c_mmi_storage_reset();
  // Pre-configure the AppId
  c_mmi_storage_set_app_id_str("Your AppId");
  // Pre-configure the AppSecret
  c_license_set_app_secret_str("Your AppSecret");
  // Pre-configure the ApiKey. This is required for fully managed mode, but not for semi-managed mode.
  c_mmi_storage_set_api_key("Your ApiKey");
  // Pre-configure the DeviceName
  c_mmi_set_device_name("Your DeviceName");
  // Save the configuration information
  c_mmi_storage_save();
}
// Pre-configure the ApiKey. This is required for fully managed mode, but not for semi-managed mode.
// The API key needs to be reconfigured every time the device starts up.
c_mmi_storage_set_api_key("Your ApiKey");

3.3.3. Device registration interface (This process is not required for pay-as-you-go mode)

/**
 * @brief Generates the registration string.
 * 
 * @param buffer The buffer to store the generated registration string.
 * @param buffer_size The size of the buffer.
 * @param time_ms_str The timestamp string in milliseconds.
 * @return int32_t Returns the operation result. 0 indicates success, non-zero indicates failure.
 */
int32_t c_license_gen_register_str(char *buffer, uint32_t buffer_size, char *time_ms_str);

/**
 * @brief Parses the device registration response from the server.
 * 
 * @param rsp_str A pointer to the device registration response string.
 * @return int32_t Returns the operation result. 0 indicates success, non-zero indicates failure.
 */
int32_t c_license_analyze_register_rsp(char *rsp_str);

Code example 3

#include "lib_c_license.h"

#define REQ_BUFFER_SIZE  (1024)

char req[REQ_BUFFER_SIZE];
char time_ms_str[14];

if (c_license_device_is_registered() == 0) {
    snprintf(time_ms_str, sizeof(time_ms_str), "%" PRId64, util_get_timestamp());
    // Generate the registration information string 'req' based on the timestamp.
    c_license_gen_register_str(req, sizeof(req), time_ms_str);
    // Get the device registration information. The customer must implement the HTTP interface.
    char *rsp = dummy_http_request(req);
    // Parse and complete the registration.
    int32_t err = c_license_analyze_register_rsp(rsp);
    if (err) {
        return err;
    }
    // After successful registration, call the storage interface to save the registration information.
    c_mmi_storage_save();
}

Example of a request data packet generated by the device-side SDK

{
    "appId": "<YOUR APP ID>",
    "deviceName": "<YOUR DEVICE NAME>",
    "nonce": "<YOUR NONCE>",
    "requestTime": "1753326620619",
    "sdkVersion": "0.3.2",
    "signature": "<YOUR SIGNATURE>"
}

Example of a data packet passed from the server to the device. The following data is the `data` field in the HTTP response message.

{
  "nonce": "<YOUR NONCE>",
  "responseTime": "1753326621269",
  "appId": "<YOUR APP ID>",
  "deviceName": "<YOUR DEVICE NAME>",
  "signature": "<YOUR SIGNATURE>"
}
   

3.3.4. Get interaction token interface (This process is not required for pay-as-you-go mode)

/**
 * @brief Checks if the token has expired.
 * 
 * @return uint8_t Returns 1 if the token is valid, 0 if the token has expired.
 */
uint8_t c_license_is_token_expire(void);

/**
 * @brief Generates the request data to get a token.
 *
 * This function generates a request string to get a token based on the provided parameters.
 * This is used to request an access token from the server.
 *
 * @param buffer The buffer to store the generated request string.
 * @param buffer_size The size of the buffer.
 * @param time_ms_str The timestamp string in milliseconds.
 * @param api_key The API key string. Set api_key to NULL in semi-managed mode.
 * @return int32_t Returns the operation result. 0 indicates success, non-zero indicates failure.
 */
int32_t c_license_gen_get_token_str(char *buffer, uint32_t buffer_size, char *time_ms_str, char *api_key);

/**
 * @brief Parses the login response data.
 *
 * Parses the login response string and triggers events:
 * - C_MMI_EVENT_USER_CONFIG: User configuration initialization.
 * - C_MMI_EVENT_DATA_INIT: Data initialization complete.
 *
 * @param rsp_str The string containing the login response.
 * @return int32_t Returns the operation result. 0 indicates success, non-zero indicates failure.
 */
int32_t c_license_analyze_get_token_rsp(char *rsp_str);

Code example 4

#include "lib_c_license.h"

#define REQ_BUFFER_SIZE  (1024)

char req[REQ_BUFFER_SIZE];
char time_ms_str[14];
char api_key[36] = { 0 };

snprintf(time_ms_str, sizeof(time_ms_str), "%" PRId64, util_get_timestamp());
c_mmi_storage_get_api_key(api_key);
// Generate the registration information string 'req' based on the timestamp issued by the server.
// An api_key is required for fully managed mode.
c_license_gen_get_token_str(req, sizeof(req), time_ms_str, api_key);
// For semi-managed mode, you can pass NULL.
// c_license_gen_get_token_str(req, sizeof(req), time_ms_str, NULL);
// Get the device information returned from the server. The customer must implement this interface.
char *rsp = dummy_http_request(req);
// Parse the interaction token.
int32_t err = c_license_analyze_get_token_rsp(rsp);
if (err) {
    return err;
}

Example of a request data packet generated by the device-side SDK

{
    "appId":"<YOUR APP ID>",
    "deviceName":"<YOUR DEVICE NAME>",
    "payMode":"LICENSE",
    "requestTime":"1753327457730",
    "sdkVersion":"0.3.2",
    "tokenType":"MMI",
    "signature": "<YOUR SIGNATURE>"
}

After the server adds the `tokenKey` field, it sends the request to the Alibaba Cloud POP interface.

{
    "appId":"<YOUR APP ID>",
    "deviceName":"<YOUR DEVICE NAME>",
    "payMode":"LICENSE",
    "requestTime":"1753327457730",
    "sdkVersion":"0.3.2",
    "tokenType":"MMI",
    "signature": "<YOUR SIGNATURE>",
    "tokenKey": "<YOUR TOKEN KEY>"
}

Example of a data packet sent from the server to the device

{
    "nonce": "<YOUR NONCE>",
    "responseTime": "1753327458081",
    "appId": "<YOUR APP ID>",
    "deviceName": "<YOUR DEVICE NAME>",
    "requestIp": "<YOUR IP>",
    "signature": "<YOUR SIGNATURE>"
}

3.3.5. Establish a service connection

/**
 * @brief Gets the WSS server hostname string.
 *
 * This function returns the WSS server hostname string.
 * 
 * @return char* Returns a pointer to the WSS server hostname string.
 */
char *c_mmi_get_wss_host(void);

/**
 * @brief Gets the WSS server port string.
 *
 * This function returns the WSS server port string.
 * 
 * @return char* Returns a pointer to the WSS server port string.
 */
char *c_mmi_get_wss_port(void);

/**
 * @brief Gets the WSS service API path string.
 *
 * This function returns the WSS service API path string.
 * 
 * @return char* Returns a pointer to the WSS service API path string.
 */
char *c_mmi_get_wss_api(void);

/**
 * @brief Gets the WSS request header information string.
 *
 * This function returns the WSS request header information string.
 * 
 * @return char* 
 *         Returns a pointer to the WSS request header information string.
 */
char *c_mmi_get_wss_header(void);

Code Example 5

#include "c_mmi.h"

char *wss_host = c_mmi_get_wss_host();
char *wss_port = c_mmi_get_wss_port();
char *wss_api = c_mmi_get_wss_api();
char *wss_header = c_mmi_get_wss_header();

// Establish a WSS connection. The customer must implement the dummy_wss_connect interface.
WSS_HANDLE *wss = dummy_wss_connect(wss_host, wss_port, wss_api, wss_header);

Example of an upgrade request message to establish a WebSocket connection

GET <WSS API> HTTP/1.1
Host: <WSS HOST>
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: FhVlQeR4S1N06+1/SU79XA== 
Sec-WebSocket-Version: 13
<WSS HEADER>

Example response message for establishing a WebSocket connection

HTTP/1.1 101 Switching Protocols
upgrade: websocket
connection: upgrade
sec-websocket-accept: sqchBdVDX8kKBgi90/PFl5+/4VI=
date: Thu, 24 Jul 2025 08:25:24 GMT
server: istio-envoy

3.3.7. Event callback

enum {
    C_MMI_EVENT_USER_CONFIG,        // User configuration for the SDK, such as audio buffer size, work mode, and voice, should be implemented in this event callback.
    C_MMI_EVENT_DATA_INIT,	    // This event is triggered when the SDK has finished initialization. You can start establishing the service connection in this event callback.
    C_MMI_EVENT_SPEECH_READY,	    // This event is triggered after a WSS connection is correctly established. In push and tap modes, speech start can only be called after this event.
    C_MMI_EVENT_SPEECH_PREPARE,     // This event is triggered when the SDK is ready to start a new round of conversation.
    C_MMI_EVENT_SPEECH_START,       // This event is triggered when the SDK starts audio uplink.
    C_MMI_EVENT_SPEECH_RESTART,     // This event is triggered when the SDK restarts audio uplink.
    C_MMI_EVENT_DATA_DEINIT,	    // This event is triggered after the SDK is unregistered.

    C_MMI_EVENT_ASR_START,	    // This event is triggered when ASR starts returning data.
    C_MMI_EVENT_ASR_INCOMPLETE,	    // This event returns incomplete ASR text data (full text).
    C_MMI_EVENT_ASR_COMPLETE,	    // This event returns the complete ASR text data (full text).
    C_MMI_EVENT_ASR_END,	    // This event is triggered when ASR ends.

    C_MMI_EVENT_LLM_INCOMPLETE,	    // This event returns incomplete LLM text data (full text).
    C_MMI_EVENT_LLM_COMPLETE,	    // This event returns the complete LLM text data (full text).

    C_MMI_EVENT_TTS_START,          // This event is triggered when audio downlink starts.
    C_MMI_EVENT_TTS_END,	    // This event is triggered when audio downlink is complete.

    C_MMI_EVENT_HEARTBEAT,	    // This event is triggered when the SDK receives a heartbeat reply from the cloud.
};


/**
 * @brief MMI event callback function type.
 *
 * A callback function triggered when the MMI module's state changes or an event occurs.
 * 
 * @param event The event type, which is one of the C_MMI_EVENT_xxx macros.
 * @param param The event parameter, which points to different data structures depending on the event type.
 * @return int32_t Returns 0 for successful processing, non-zero for failure.
 */
typedef int32_t(*c_mmi_event_callback)(uint32_t event, void *param);

3.3.8. Multi-modal data interaction

/**
 * @brief Writes data to the recorder buffer.
 *
 * @param data A pointer to the data to be written.
 * @param size The length of the data in bytes.
 * @return uint32_t The number of bytes actually written.
 */
uint32_t c_mmi_put_recorder_data(uint8_t *data, uint32_t size);

/**
 * @brief Reads data from the player buffer.
 *
 * @param data The buffer to store the read data.
 * @param size The available buffer size in bytes.
 * @return uint32_t The number of bytes actually read.
 */
uint32_t c_mmi_get_player_data(uint8_t *data, uint32_t size);

/**
 * @brief Gets the data to be sent via WebSocket.
 * 
 * This function is used to get data of a specified type, preparing it to be sent via WebSocket.
 * It fills the provided data buffer with data of the corresponding type based on the input type parameter.
 * 
 * @param opcode Returns the WebSocket data type, such as WS_DATA_TYPE_TEXT or WS_DATA_TYPE_BINARY.
 * @param data A pointer to a uint8_t array that will store the retrieved data.
 * @param size The maximum capacity of the data array, in bytes.
 * @return Returns the number of bytes actually filled into the data array.
 */
uint32_t c_mmi_get_send_data(uint8_t *opcode, uint8_t *data, uint32_t size);

/**
 * @brief Analyzes the received WebSocket data.
 * 
 * This function analyzes the received data packet based on the provided data type and content.
 * Its main purpose is to parse the data content for further processing or use.
 * 
 * @param opcode The WebSocket data type, such as WS_DATA_TYPE_TEXT or WS_DATA_TYPE_BINARY.
 * @param data A pointer to the received data. The content will be parsed based on the type parameter.
 * @param size The length of the data in bytes, used to determine the data's scope.
 * @return Returns the number of bytes actually parsed.
 */
uint32_t c_mmi_analyze_recv_data(uint8_t opcode, uint8_t *data, uint32_t size);

Code Example 6

int dummy_wss_task_recv(void)
{
    int opcode;
    char payload_data[8 * 1024];
    int recv_size;

    while(1){
        // Receive downstream data from the server via WebSocket.
        dummy_wss_recv(&opcode, payload_data, &recv_size);
        if(recv_size)
            // Send the received downstream data to the SDK for parsing.
            c_mmi_analyze_recv_data(opcode, payload_data, recv_size);
        else
            util_msleep(10);
    }
    return 0;
}

int dummy_wss_task_send(void)
{
    uint8_t opcode;
    uint8_t payload_data[8 * 1024];
    uint32_t size;

    while(1){
        // Get the packaged payload data from the SDK.
        size = c_mmi_get_send_data(&opcode, payload_data, sizeof(payload_data));
        if (size == 0) {
            util_msleep(10);
        } else {
            // Pass the payload data to the WebSocket send function, and package the frame header for sending.
            dummy_wss_send(opcode, data, size);
        }
    }
    return 0;
}


int _mmi_event_callback(uint32_t event, void *param)
{
    char *text;

    text = param;
    switch (event) {
        case C_MMI_EVENT_USER_CONFIG:
            // Start a new conversation.
            c_mmi_reset_dialog_id();
            break;
        case C_MMI_EVENT_DATA_INIT:
            // MMI data is ready, start the network connection.
            dummy_wss_init();
            break;
        case C_MMI_EVENT_DATA_DEINIT:
            UTIL_LOG_W("will disconnect");
            break;
        case C_MMI_EVENT_SPEECH_START:
            UTIL_LOG_D("enable recorder when send speech");
            dummy_player_stop();
            dummy_recorder_start();
            break;
        case C_MMI_EVENT_ASR_START:
            UTIL_LOG_I("event [C_MMI_EVENT_ASR_START]");
            break;
        case C_MMI_EVENT_ASR_INCOMPLETE:
            UTIL_LOG_D("ASR [%s]", text);
            break;
        case C_MMI_EVENT_ASR_COMPLETE:
            if (text) {
                UTIL_LOG_D("ASR C [%s]", text);
            } else {
                UTIL_LOG_D("ASR C [NULL]");
            }
            break;
        case C_MMI_EVENT_ASR_END:
            UTIL_LOG_D("disable record when ASR complete");
            dummy_recorder_stop();
            break;
        case C_MMI_EVENT_LLM_INCOMPLETE:
            UTIL_LOG_D("LLM [%s]", text);
            break;
        case C_MMI_EVENT_LLM_COMPLETE:
            UTIL_LOG_D("LLM C [%s]", text);
            break;
        case C_MMI_EVENT_TTS_START:
            UTIL_LOG_I("enable player when dialog start");
            dummy_player_start();
            break;
        case C_MMI_EVENT_TTS_END:
            break;
        default:
            break;
    }

    return UTIL_SUCCESS;
}