Build an elastic audio and video processing system with CloudFlow

Traditional self-built solution

You can purchase ECS instances to deploy your audio and video processing services and use OSS and CDN for storage and playback acceleration.

Serverless solution

Video processing workflow

If you need to accelerate the transcoding of large videos or perform complex combined operations, you can use CloudFlow to orchestrate functions and implement a powerful audio and video processing system. The following figure shows the solution architecture.

When a user uploads an MP4 video to OSS, OSS automatically triggers a CloudFlow workflow to transcode the video into one or more formats. This solution meets the following requirements:

• A video file can be simultaneously transcoded into various formats and undergo other custom processing, such as adding watermarks or updating information in a database after processing.

• When multiple files are uploaded to OSS at the same time, Function Compute automatically scales to process multiple files in parallel and transcodes the files into various formats in parallel.

• You can use NAS and video segmentation to transcode oversized videos. This process involves segmenting the video, transcoding the segments in parallel, and then merging the results. By setting a reasonable segment duration, you can improve the transcoding speed of large videos.

  Note

  Video segmentation is the process of splitting a video stream into a series of segment files at specified time intervals and generating an index file that records information about the segments.

This solution uses CloudFlow to orchestrate multiple functions into a video processing workflow. The following steps use Serverless Devs to deploy the system.

1. Go to the multimedia-process-flow template.

2. Click Deploy to navigate to the application center of Function Compute and start deploying the application.

3. On the Create Application page, configure the following parameters, and then click Create and Deploy Default Environment.

   The following table describes the key parameters. You can use the default values for other parameters.

   Parameter	Description
   Basic Information
   Deployment Type	Select Direct Deployment.
   Role Name	If you are using an Alibaba Cloud account and creating an application in the application center for the first time, click Authorize Now. This redirects you to the Role Templates page. Create the AliyunFCServerlessDevsRole role, and then click Confirm. When you grant the permissions, make sure that Function Compute is the trusted service. If you are using a RAM user, follow the on-screen instructions to copy the authorization link and have your Alibaba Cloud account owner approve it. After the authorization is complete, click Authorized. The page then indicates that the primary account has not yet created a valid AliyunFCServerlessDevsRole role. Note If the Failed to obtain the role message appears, contact the Alibaba Cloud account owner to grant the AliRAMReadOnlyAccess and AliyunFCFullAccess permissions to the current RAM user. For more information, see Grant permissions to a RAM user by using an Alibaba Cloud account.
   Advanced Configuration
   Region	Select the region where you want to deploy the application.
   Workflow RAM Role ARN	The service role used for workflow execution. Create the role in advance and grant it the AliyunFCInvocationAccess policy.
   Function Compute service role	The service role that Function Compute uses to access other cloud services. We recommend using the default AliyunFCDefaultRole unless you have specific requirements.
   Object Storage Bucket Name	Enter the name of a bucket in the same region as the workflow and function.
   Prefix	The prefix of the directory where source videos are stored. This example uses src.
   Directory for transcoded videos	The destination directory for the transcoded videos. This example uses dst.
   OSS trigger RAM role ARN	We recommend using the AliyunOSSEventNotificationRole role. If you are using it for the first time, click Create Role to authorize and create the role.

   The application deploys in 1 to 2 minutes and automatically creates five functions and a workflow named multimedia-process-flow-3oih. You can view these resources in the Function Compute console and the CloudFlow console. The five functions are:

   • oss-trigger-workflow: Configures a trigger to detect new video uploads in a specific directory and trigger the workflow.

   • split: Segments a video into parts of a specified length in the working directory.

   • transcode: Transcodes video segments into the specified video formats.

   • merge: Merges the transcoded video segments.

   • after-process: Cleans up the working directory.

4. Verify the result.

   1. Log on to the CloudFlow console. In the top navigation bar, select a region.

   1. In the workflow list, click the target workflow multimedia-process-flow-3oih. On the workflow details page, click Start Execution. In the panel that appears, enter an Execution Name and the Execution Input. The following code provides an example input:

      {
        "oss_bucket_name": "buckettestfnf",
        "video_key": "source/test.mov",
        "output_prefix": "outputs",
        "segment_time_seconds": 15,
        "dst_formats": [
          "mp4",
          "flv"
        ]
      }

      A status of Succeeded means the execution is complete.

   2. Log on to the Object Storage Service (OSS) console. Open the target bucket and navigate to the /outputs directory to view the transcoded files.

      Transcoded files in this directory confirm that the video processing system is working correctly.

      The transcoded and merged files, merged_test.flv (19.905 MB) and merged_test.mp4 (4.516 MB), are stored in the /outputs/test/ directory.