On-device super resolution

更新时间:
复制 MD 格式

On-device super resolution is a feature that uses a device-side model to convert low-definition videos to high-definition directly on the device. This process significantly improves video playback quality and reduces transmission costs. This topic describes how to configure the on-device super resolution feature on Android and iOS devices.

Background information

As video services evolve, users demand higher-definition and ultra-high-definition viewing experiences. Some video sources cannot provide high-definition versions because of limitations in production, storage, or distribution. This compromises the viewing experience. Traditional cloud-based ultra-high definition production solutions require high bandwidth and are not suitable for users in environments with poor network conditions.

The on-device super resolution feature provides a balance between video quality, viewing experience, and bandwidth costs. It converts low-definition videos to high-definition on the local device. This significantly improves the viewing experience without increasing bandwidth costs.

Benefits

Category

Description

Performance and power consumption

  • High efficiency: Processes a single video frame in less than 20 ms. Supports real-time super resolution for video at 30 FPS. This makes it suitable for various playback scenarios, such as live streaming and video-on-demand.

  • Powerful: Processes videos of multiple resolutions. Supports real-time processing for videos up to 540P resolution. It can upgrade a 540P video to 1080P.

  • Low power consumption: Power usage varies by device model, chip, and video resolution. The minimum power consumption ranges from 50 mA to 100 mA per hour.

Compatibility

  • Device compatibility: Compatible with most current mainstream device models.

  • Platform support: Supports Android and iOS platforms.

Feature type

  • Plugin-based and flexible: The feature is delivered as a plugin. It has no dependencies on mainstream player libraries. You can freely replace the plugin and upgrade the feature, which allows for flexible integration.

  • Small footprint: The Android algorithm library adds 408 KB. The iOS algorithm library adds 730 KB.

Prerequisites

You have obtained authorization for the Player SDK and purchased the on-device super resolution value-added service. For more information, see Manage License.

Integrate and use on Android

  1. Obtain the on-device super resolution feature library.

    To obtain the on-device super resolution feature library, submit a ticket or contact Alibaba Cloud Business Development to request access.

  2. Integrate the on-device super resolution feature library.

    1. Load the on-device super resolution feature library.

      The on-device super resolution feature library is a dynamic library. You can load the library using the following method.

      System.loadLibrary(libname);
    2. Configure the on-device super resolution feature library.

          /**
           * Set the filter configuration. Call this method before prepare. To update the filter configuration, call updateFilterConfig().
           * @param filterConfig
           */
          /****
           * Set filter config. call this before prepare. If want update filter config, call updateFilterConfig()
           * @param filterConfig
           */
          abstract public void setFilterConfig(FilterConfig filterConfig);
      
          /**
           * Update the filter configuration.
           * @param target
           * @param options
           */
          /****
           * upadate filter config.
           * @param target
           * @param options
           */
          abstract public void updateFilterConfig(String target, FilterConfig.FilterOptions options);
      
          /**
           * Enable or disable the filter.
           * @param target  If this is empty, the setting applies to all filters.
           * @param invalid  true: enable (default); false: disable
           */
          /****
           * disable/enable filter.
           * @param target  if empty , disable all filters.
           * @param invalid  true: enable(default); false: disable
           */
          abstract public void setFilterInvalid(String target, boolean invalid);
  3. Sample usage.

    // filterConfig is implemented in JSON. The JSON format is as follows:
    player.setFilterConfig(filterConfig);
    // Enable the on-device super resolution feature. The target is the same as the value of "target" in the JSON below.
    player.setFilterInvalid(target,false);
    [
      {
        "target":"sr",
        "options":{
          // No specific options are required for this feature.
        }
      }
    ]

Integrate and use on iOS

  1. Obtain the on-device super resolution feature library.

    To obtain the on-device super resolution feature library, submit a ticket or contact Alibaba Cloud Business Development to request access.

  2. Integrate the on-device super resolution feature library.

    The on-device super resolution feature library is a dynamic framework. Add the library to the frameworks and libraries in Xcode. You can add the library using the following method.

    Note

    The following sample code uses version 5.5.4.0 as an example. Select the version that meets your business needs. For more information about the release history of the Player SDK for iOS, see iOS SDK release history.

    pod 'AliPlayerSDK_iOS_NORMAL_SR_FILTER', '5.5.4.0'
  3. Sample usage.

    // Initialization
    AVPFilter* srFilter = [[AVPFilter alloc] initWithTarget:@"normal_sr"]; // To use this library, the target name must be "normal_sr". You cannot use other custom strings.
    AVPFilterOptions* srOptions = [[AVPFilterOptions alloc] init];
    [srOptions setOptions:@"path" value:@"xxx"]; // xxx is the custom relative path in the sandbox.
    [srFilter setOptions:srOptions];
    [filterConfig addFilter:srFilter];
    [self.player setFilterConfig:filterConfig];
    
    // Start to use
    [self.player setFilterInvalid:@"normal_sr" invalid:YES]; // Set invalid to YES to enable the feature, or NO to disable it.