Video sprite generation-Intelligent Media Management(IMM)-阿里云帮助中心

Introduction

Video sprite generation is a technique that combines multiple keyframes from a video into a single image, known as a sprite. By generating a sprite, you can provide a quick overview of different scenes in a video, allowing users to browse and select content more efficiently.

Use cases

Video preview and content navigation: Sprites offer a quick preview of video content, allowing users to scan keyframes and easily jump to parts that interest them.
Video playlists: You can use sprites to display thumbnails for each video in a playlist. This creates a cleaner, more visually appealing interface and improves the user experience.
Social media sharing: When you share videos on social media, a sprite can serve as a cover image to help viewers quickly understand the video's content, increasing click-through rates and views.
Live streams and event replays: For live events or recorded content, sprites help viewers quickly find highlights, making it more convenient to watch replays.

Supported audio and video formats

Category	Format
Audio	Mainstream audio formats, such as AAC, MP3, WAV, FLAC, WMA, AC3, and Opus.
Video	Mainstream video formats, such as MP4, MPEG-TS, MKV, MOV, AVI, FLV, M3U8, WebM, WMV, RM, and VOB.

Procedure

Prerequisites

An AccessKey pair is created and obtained. For more information, see Create an AccessKey pair.
Object Storage Service (OSS) is activated and a bucket is created. For more information, see Create a bucket.
IMM is activated. For more information, see Activate IMM.
A project is created in the IMM console. For more information, see Create a project.
Note
- You can call the CreateProject operation to create a project. For more information, see CreateProject.
- You can call the ListProjects operation to list information of all projects in a region.

Step 1: Upload a file

Use the OSS console to upload a media file to a bucket in the same region as your IMM project.

On the File Management page of the bucket, click Upload File. In the Upload File panel, click Scan for Files to select the media file to upload. Keep the default ACL setting of Inherit from Bucket, and then click Upload File to complete the upload. The maximum size for a file uploaded through the console is 5 GB. To upload larger files, use ossutil, the OSS SDK, or the OSS API.

Step 2: Generate video sprites

Call the CreateMediaConvertTask operation to create a video sprite generation task.

Parameter examples

The process of generating a video sprite typically includes the following steps:

Frame extraction: The service extracts specified keyframes from the video.
Image processing: The service resizes and converts the extracted frames to ensure consistency and meet output requirements.
Sprite generation: The service arranges the processed frames in a grid and merges them into a single image to form the sprite.

The following examples use the IMM project test-project and the file oss://test-bucket/video-demo/sample-video.mov to generate video sprites.

For more information about media processing features and how to use them, see Media transcoding.

Note

You can use OpenAPI Explorer to process video files and view sample SDK code.
Do not set the Target.URI and Target.Container parameters for video sprite generation.
Video sprite generation can produce multiple sprite images. The exact number depends on the number of sub-images and the row and column configuration of the sprite.
Set the sprite output URI to oss://bucket/objectPrefix-{index}.{autoext}. If you do not include the {index} variable, the output images may overwrite each other. For more information about variables, see TargetURI template.

Generate a sprite from the full video

This example generates a sprite by capturing one frame every two seconds from the entire video. Each sprite contains a 3x3 grid of sub-images, and each sub-image has a resolution of 200x150 pixels.

Transcoding information

Sprite format: jpg
Frame capture interval: 2 seconds
Sprite sub-image layout: 3x3
Sub-image resolution: 200x150
Padding between sub-images (Pad): 0
Margin between sub-images and sprite edges (Margin): 0
Sub-image scaling method: Scale and crop. For more information, see scaling method.
Output file path: oss://test-bucket/video-demo/sprites-{index}.{autoext}
Completion notification: Send an MNS message to the MNS topic test-mns-topic.

For an SDK example, see OpenAPI Explorer. The example parameters are pre-filled. You can modify them as needed before debugging.

Request parameters

{
  "ProjectName": "test-project",
  "Notification": {
    "MNS": {
      "TopicName": "test-mns-topic"
    }
  },
  "Sources": [
    {
      "URI": "oss://test-bucket/video-demo/sample-video.mov"
    }
  ],
  "Targets": [
    {
      "Image": {
        "Sprites": [
          {
            "Format": "jpg",
            "Interval": 2,
            "Margin": 0,
            "Pad": 0,
            "ScaleHeight": 150,
            "ScaleType": "crop",
            "ScaleWidth": 200,
            "TileHeight": 3,
            "TileWidth": 3,
            "URI": "oss://test-bucket/video-demo/sprites-{index}.{autoext}"
          }
        ]
      }
    }
  ]
}

Generate a sprite from a specific time range

This example generates a sprite by capturing one frame every two seconds from the 5-second mark to the 30-second mark of the video. Each sprite contains a 3x5 grid of sub-images, and the dimensions of each sub-image are scaled to 1/10 of the source video.

Transcoding information

Sprite format: jpg
Capture start time: 5 seconds
Capture duration: 25 seconds
Frame capture interval: 2 seconds
Sprite sub-image layout: 3x5
Sub-image resolution: 1/10 of the source video
Padding between sub-images (Pad): 4
Margin between sub-images and sprite edges (Margin): 5
Sub-image scaling method: Fit to size without black bars. For more information, see scaling method.
Output file path: oss://test-bucket/video-demo/sprites-{index}.{autoext}
Completion notification: Send an MNS message to the MNS topic test-mns-topic.

For an SDK example, see OpenAPI Explorer. The example parameters are pre-filled. You can modify them as needed before debugging.

Request parameters

{
  "ProjectName": "test-project",
  "Notification": {
    "MNS": {
      "TopicName": "test-mns-topic"
    }
  },
  "Sources": [
    {
      "Duration": 25,
      "StartTime": 5,
      "URI": "oss://test-bucket/video-demo/sample-video.mov"
    }
  ],
  "Targets": [
    {
      "Image": {
        "Sprites": [
          {
            "Format": "jpg",
            "Interval": 2,
            "Margin": 5,
            "Pad": 4,
            "ScaleHeight": 0.1,
            "ScaleType": "fit",
            "ScaleWidth": 0.1,
            "TileHeight": 5,
            "TileWidth": 3,
            "URI": "oss://test-bucket/video-demo/sprites-{index}.{autoext}"
          }
        ]
      }
    }
  ]
}

Generate a sprite with evenly spaced frames

This example shows how to use the DetectMediaMeta operation to get the video duration and then generate a sprite with a specific number of evenly spaced frames.

Step 1: Get the video duration

Call the DetectMediaMeta operation to get the video duration. For an SDK example, see OpenAPI Explorer. The example parameters are pre-filled. You can modify them as needed before debugging.

Request parameters

{
  "ProjectName": "test-project",
  "SourceURI": "oss://test-bucket/video-demo/sample-video.mov"
}

Response parameters

{
  "Addresses": [],
  "AudioStreams": [],
  "Bitrate": 45521134,
  "Duration": 31.64,
  "FormatLongName": "QuickTime / MOV",
  "FormatName": "mov,mp4,m4a,3gp,3g2,mj2",
  "ProduceTime": "2014-12-19T14:20:38Z",
  "RequestId": "94A9AF46-B403-51D5-988F-D9B4939DEF39",
  "Size": 180036088,
  "StreamCount": 2,
  "Subtitles": [],
  "VideoHeight": 1080,
  "VideoStreams": [
    {
      "AverageFrameRate": "25/1",
      "BitDepth": 8,
      "Bitrate": 45517646,
      "CodecLongName": "H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10",
      "CodecName": "h264",
      "CodecTag": "0x31637661",
      "CodecTagString": "avc1",
      "ColorPrimaries": "bt709",
      "ColorRange": "tv",
      "ColorSpace": "bt709",
      "ColorTransfer": "bt709",
      "Duration": 31.64,
      "FrameCount": 791,
      "FrameRate": "25/1",
      "HasBFrames": 1,
      "Height": 1080,
      "Language": "eng",
      "Level": 50,
      "PixelFormat": "yuv420p",
      "Profile": "Main",
      "TimeBase": "1/2500",
      "Width": 1920
    }
  ],
  "VideoWidth": 1920
}

Step 2: Generate the sprite with evenly spaced frames

The video duration is 31.64 seconds. To capture 9 evenly spaced sub-images, the frame capture interval is calculated as 31.64 / (9 - 1) = 3.955 seconds. The sprite has a 3x3 layout, and the dimensions of each sub-image are scaled to 1/10 of the source video.

Transcoding information

Sprite format: jpg
Frame capture interval: 3.955 seconds
Sprite sub-image layout: 3x3
Sub-image resolution: 1/10 of the source video
Padding between sub-images (Pad): 4
Margin between sub-images and sprite edges (Margin): 5
Sub-image scaling method: Fit to size without black bars. For more information, see scaling method.
Output file path: oss://test-bucket/video-demo/sprites-{index}.{autoext}
Completion notification: Send an MNS message to the MNS topic test-mns-topic.

For an SDK example, see OpenAPI Explorer. The example parameters are pre-filled. You can modify them as needed before debugging.

Request parameters

{
  "ProjectName": "test-project",
  "Notification": {
    "MNS": {
      "TopicName": "test-mns-topic"
    }
  },
  "Sources": [
    {
      "URI": "oss://test-bucket/video-demo/sample-video.mov"
    }
  ],
  "Targets": [
    {
      "Image": {
        "Sprites": [
          {
            "Format": "jpg",
            "Interval": 3.955,
            "Margin": 5,
            "Pad": 4,
            "ScaleHeight": 0.1,
            "ScaleType": "fit",
            "ScaleWidth": 0.1,
            "TileHeight": 3,
            "TileWidth": 3,
            "URI": "oss://test-bucket/video-demo/sprites-{index}.{autoext}"
          }
        ]
      }
    }
  ]
}