Introduction to Link Visual features for developers

Live streaming feature development

Live streaming video supports H264/H265, and audio supports G711A/AAC_LC. A hybrid method that combines RTMP cloud relay and P2P is used to reduce costs and ensure streaming stability.

To develop the live streaming feature, configure the following TSL models.

• Video playback

  The Link Visual App SDK provides a live stream player. For more information about integration, see Android Link Visual App SDK and iOS Link Visual App SDK.

• P2P support

  P2P support requires integration with both the Link Visual Device SDK and the Link Visual App SDK. During playback, the system attempts to establish a P2P connection whenever possible to reduce overhead. The backend system collects statistics on the P2P success rate. If the P2P success rate becomes abnormal, the system notifies the developer to investigate the cause. If the issue is not resolved, the service may be denied.

  The app-side P2P feature depends on the persistent connection channel of IoT Platform. The app must initialize the persistent connection channel SDK and attach the channel to an account. For more information, see Android persistent connection SDK and iOS persistent connection SDK.

  • Confirm successful P2P integration on the device

    After you integrate the device-side SDK, ensure that the Cloud Intelligence App (Developer Edition) and the IPC device are connected to the same Wi-Fi network. Enable the test information switch in the Cloud Intelligence App (Developer Edition) and go to the live streaming page. In the upper-left corner, RELAY indicates an RTMP cloud relay, LOCAL indicates a direct P2P connection (local area network), and SRFLX indicates P2P traversal (Internet).

    
  • Confirm successful P2P integration on your app

    Five seconds after the first frame appears, call the player's playback function to retrieve the connection type of the current stream. For example, RELAY indicates an RTMP cloud relay, LOCAL indicates a direct P2P connection, and SRFLX indicates P2P traversal.

• Video encryption

  To ensure video data security, the SDK supports the encryption of audio and video frames. We recommend that you enable end-to-end encryption in the app. The app can call the TSL model API (Android/iOS) to set the EncryptSwitch TSL model property to enable or disable encryption.

• Resolution switching

  The app can call a TSL model API (Android/iOS) to query the TSL model properties for the main and sub-stream definitions (StreamVideoQuality and SubStreamVideoQuality). This lets you display the current stream definition in the UI. The app can set this TSL model property to change the stream definition. After the device receives the TSL model property setting, it switches the stream definition and restarts the stream ingest. The player SDK supports automatic rotation.

• Screenshots and screen recording

  The player SDK provides interfaces for taking screenshots and recording the screen. For more information, see Android Link Visual App SDK and iOS Link Visual App SDK.

• Reduce first-frame latency (force I-frame)

  When a new player starts watching, the device must immediately encode an I-frame to reduce the first-frame latency of the live stream. For more information about how to respond to and initiate a forced I-frame, see the device and app documentation.

Cloud storage recording feature development

To use cloud storage recording, you must activate a cloud storage plan for the user. For more information, see LinkVisual video cloud storage purchase service.

• Introduction
  Cloud storage recordings can come from three sources.

  • Scheduled recording

    The camera records video to the cloud at specified times.

  • Event-triggered recording

    When the camera triggers an alarm within a specified time period, the camera uploads the recording to the cloud.

  • Manual recording

    The app can call the corresponding API to make the device push a recording of a specified duration to the cloud.

• Plan development

  There are two main methods: 1. Configure a cloud recording plan and set the time period for camera alert notifications. This method is not recommended because it can be confusing for users. 2. Use the default cloud recording plan and set the time period for camera alert notifications. The default cloud recording plan is 24/7. The following section describes how to configure a cloud recording plan. You can implement the plan based on the product's cloud recording type.

  • Configure a recording plan

    The general steps to configure a recording plan for a device are as follows:

    1. Create a recording plan to obtain a planId.
    2. Attach the planId to the device's iotId.

    For more information about operations such as configuring, updating, detaching, deleting, and querying, see Link Visual Video Service.

  • Configure an event-triggered recording plan

    The general steps to configure an event-triggered recording plan for a device are as follows:

    1. Create an event-triggered recording plan to obtain a planId.
    2. Attach the planId to the device's iotId.

    For more information about other interfaces, such as those for configuring, updating, detaching, deleting, and querying, see Link Visual Video Service.

• Obtain the playback URL for a cloud recording

  There are several ways to obtain the playback URL for a cloud recording:

  • Query the list of cloud recordings for a specific time range and obtain the video-on-demand URL for the cloud recording by filename.
  • Retrieve the event list for a specific time range and obtain the video-on-demand URL for the event-triggered cloud recording based on the associated recording filename.
• Play a cloud recording

  Cloud recordings use HLS as the streaming media playback protocol.

  • The Android Link Visual App SDK provides the ExoHlsPlayer, which is encapsulated based on ExoPlayer, for HLS stream playback.
  • The iOS SDK provides the IMSLinkVisualHLSPlayer, which is encapsulated based on AVFoundation, for HLS stream playback.
• Download a cloud recording

  You can download and play cloud recordings. For more information, see Link Visual Video Service.

• Video encryption

  Cloud recordings have AES-128 encryption enabled by default.

On-demand playback of device recordings feature development

IPC devices are typically equipped with a separate, large-capacity storage medium to continuously record video captured by the camera. This footage is not suitable for uploading to the cloud. The on-demand playback feature for device recordings lets you review this footage later.

To develop the on-demand playback feature for device recordings, configure the following TSL models.

• On-demand playback by individual file

  This feature allows on-demand playback of a single recording file on the device. The process is as follows.

  1. The app calls the QueryRecordList TSL model service to retrieve a list of device recording files for a specific time period.
  2. The device responds to the QueryRecordList TSL model service, completes the query, and returns the results to the app for display.
  3. The user selects a file from the list to play. The app calls an API operation (retrieve playback by filename) to play the recording.
  Note

  • The response from the QueryRecordList TSL model service has a size limit. When the app queries the list of device recording files, it must use a paged query. We recommend 10 records per page.
  • The device must respond to the QueryRecordList TSL model service within 3 seconds. Otherwise, the query times out and fails.
  • In the on-demand playback by individual file pattern, the length of the playback file is the actual length of the file.
  • In the on-demand playback by individual file pattern, if playback reaches the end of the file, the device explicitly sends a stream ingest end instruction to inform the app that playback is complete. However, the connection is not disconnected. This allows for another seek operation.
  • In the on-demand playback by individual file pattern, for a seek command, the app sends a relative time offset from the beginning of the file, in seconds.
• On-demand playback by time range

  Some devices store local recordings as small, individual files that are split by a fixed time interval, such as one file per minute. Retrieving a list of recordings for an entire day would be very large, and requesting playback for individual files is not user-friendly. For this storage method, on-demand playback by individual file is not suitable. Therefore, an on-demand playback by time range feature is provided. The device-side SDK merges contiguous small files and treats the content within the start and end times as a single large file for playback. It also lets you seek to any position within the start and end times during playback. The process is as follows.

  1. The app calls the QueryRecordTimeList TSL model service to retrieve the time periods that have recordings within a specific time range (usually one day).
  2. The device responds to the QueryRecordTimeList TSL model service, completes the query, and returns the results to the app for display.
  3. The app calls an API operation (request local recording playback by time) to play the recording. By default, after the connection is established, playback starts from the beginning of the first time period in the time range. To set an offset, specify a seekTime when you initiate the request.
  Note

  • The response from the QueryRecordTimeList TSL model service has a size limit. When the app queries the list of device recording files, it must use a paged query. We recommend 10 records per page.
  • The device must respond to the QueryRecordTimeList TSL model service within 3 seconds. Otherwise, the query times out and fails.
  • In the on-demand playback by time range pattern, the device enables continuous playback by default. This means that after one recording file finishes playing, the device automatically starts ingesting the stream for the next file. If the app needs to implement a non-continuous playback timeline for device recordings, we recommend that you use the on-demand playback by individual file pattern.
  • In the on-demand playback by time range pattern, if a seek command is issued for a time offset that has no video content, the device-side SDK, by default, seeks forward to the start of the next available video content and begins stream ingest from there.
  • In the on-demand playback by time range pattern, if there is no recording after the app sends a seek time, the device explicitly sends a stream ingest end instruction to inform the app that playback is complete. However, the connection is not disconnected. This allows for another seek operation.
  • In the on-demand playback by time range pattern, the length of the playback file is the duration of the time range (endTime - beginTime) specified in the playback request.
  • In the on-demand playback by time range pattern, for a seek command, the app sends a relative time offset from the beginTime, in seconds.
  
• Device stream ingest

  Similar to live streaming, the device's stream ingest rate must not be less than the original content generation rate. We recommend a rate that is slightly faster than the original generation rate, such as 1.1 times. When the player's receive cache level exceeds 75%, the player notifies the device to pause stream ingest. When the player's cache level drops below 25%, the player notifies the device to resume stream ingest. Therefore, the device must respond to pause and resume instructions. The device also needs to handle file seek operations. To respond to seek positions accurately, the Group of Pictures (GOP) of the recording file must not be set too large.

• Variable-speed playback

  Not yet supported.

• On-demand player

  Compared to the live stream player, the on-demand player adds interfaces for pause/resume, seek to a specified position, retrieve the total duration, and retrieve the current duration. For more information, see Link Visual Video Service.

Voice intercom feature development

This feature provides end-to-end real-time audio transmission between the app and the IPC device. Supported audio formats include G711a, G711u, AAC_LC, and PCM.

The intercom is initiated by the app and responded to by the device. For more information about the integration process, see Link Visual Video Service and Overview.

To develop the voice intercom feature, configure the following TSL models.

• Intercom mode

  Intercom Mode	Device-side	App	Dependencies
  Full-duplex real-time intercom	Sends and receives audio in real time	Sends and receives audio in real time	The device needs a microphone and a speaker. Both the device and the app must support echo cancellation.
  Half-duplex real-time intercom	Receives audio in real time	Sends audio in real time	The device needs a speaker.

• Link encryption

  To ensure data security, AES-128 full-frame encryption is enabled by default.

Event reporting feature development

This feature allows devices to report events, which is used for functions such as alarm detection and fault reporting. The server-side processes events at a minimum interval of 10 seconds. Events that are reported too frequently are not processed.

To develop the event reporting feature, configure the following TSL models.

Image capture feature development

To develop the image capture feature, configure the following TSL models.

• Manual image capture

  The app actively notifies the device to capture an image. The process is as follows.

  1. The app calls the Link Visual API (manual image capture) to notify the device to capture an image.
  2. The device responds to the TriggerPicCapture TSL model service call, triggers an image capture, and uploads the image to the specified URL using an HTTP POST request.
• Event-triggered image capture

  The device actively reports an alarm event, which triggers an image capture. The process is as follows.

  1. The device detects an alarm, captures an image, generates an image ID, and reports the alarm event.
  2. The server-side accepts the alarm event and triggers the device's TriggerPicCapture TSL model service.
  3. The device responds to the TriggerPicCapture TSL model service call and uploads the captured image with the corresponding image ID to the specified URL using an HTTP POST request.
• App display and management of images

  The app calls the image-related interfaces in the Link Visual API to display and manage images.

  • Query image list by time

    Retrieves the thumbnail and full-size image URLs for all images within a specified time range.

  • Query image URL by image ID

    The image ID can be obtained from the event list and is used to display images related to alarm events.

  • Batch delete images

    Deletes specified images based on a list of image IDs. A maximum of 100 images can be deleted at a time.

Other feature development

• Device PTZ control

  To develop the PTZ control feature, configure the following TSL models.

  Identifier	Feature Type	Feature Name	Description	Select in Console	Development Guide
  PTZActionControl	Service	PTZ step control	The app initiates this service, and the device responds to trigger it.	Yes	App and device developers must handle this TSL model.
  PTZCalibrate	Service	PTZ calibration	The app initiates this service, and the device responds, triggering a PTZ calibration operation.	As needed	App and device developers must handle this TSL model.

  We recommend that the app calls the PTZActionControl TSL model service at a frequency of four times per second to send control pulses to the device. The device must process the TSL model service responses promptly without accumulation. It is acceptable to ignore TSL model service calls for the same direction that cannot be processed in time.

• Device properties

  We provide a standard TSL model for the camera category. The feature definitions that you must configure are as follows.

  Identifier	Feature Type	Feature Name	Description	Select in Console	Development Guide
  DayNightMode	Property	Day/Night mode	The app sets and gets the current Day/Night mode of the device. The device actively reports this property upon startup and when its status changes.	As needed	App and device developers must handle this TSL model.
  ImageFlipState	Property	Image flip state	The app sets and gets the current image flip state of the device. The device actively reports this property upon startup and when its status changes.	As needed	App and device developers must handle this TSL model.
  MicSwitch	Property	Microphone switch	The app sets and gets the current status of the device's microphone switch. The device actively reports this property upon startup and when its status changes.	As needed	App and device developers must handle this TSL model.
  StatusLightSwitch	Property	Status light switch	The app sets and gets the current status of the device's status light switch. The device actively reports this property upon startup and when its status changes.	As needed	App and device developers must handle this TSL model.
  IpcVersion	Property	Camera version number	The app gets the current version number information of the device. The device actively reports this property upon startup.	Yes	App and device developers must handle this TSL model.

  If the standard TSL model does not meet your needs, you can extend it with custom features. For example, you can add a Reboot TSL model service to implement a device restart feature.

• Cloud storage plans

  To use cloud storage recording, you must activate a cloud storage plan for the user. For information about the procedures and APIs for vendors to call the LinkVisual video cloud storage purchase service, see LinkVisual video cloud storage purchase service.

  We also offer a free 7-day event cloud storage plan for each device. This plan is valid for three months from the date of activation. Each device can activate this plan only once. For information about app-side activation and querying, see Link Visual Video Service (Activate Cloud Storage Plan and Query Cloud Storage Plan Details).

• Message push

  This feature is used to actively push messages to users through the app, such as motion detection alarms and critical device status updates. For more information, see Mobile app push development guide.

• Network configuration

  For more information, see Configure network configuration guide.

• OTA

  For more information, see Firmware upgrade.