Install MaxFrame in a DataWorks runtime image

更新时间:
复制 MD 格式

This topic describes how to use a custom runtime image in DataWorks Image Management to install MaxFrame and develop applications. This approach helps you quickly build a development environment and avoid version conflicts with other environments.

Prerequisites

Procedure

Step 1: Create a custom runtime image

  1. Log on to the DataWorks console and click Image Management in the left navigation pane.

  2. On the Custom Images tab, click Create Image. Configure the key parameters in the dialog box:

    Parameter

    Description

    Image Name

    The name of the custom runtime image.

    Image Description

    The description of the custom runtime image.

    Reference Data Type

    Select DataWorks Official Image.

    Image Namespace

    This parameter is fixed to DataWorks Default.

    Image Repository

    This parameter is fixed to DataWorks Default.

    Image Name/ID

    From the drop-down list, select an official runtime image provided by DataWorks. For this topic, select dataworks_pyodps_task_pod:prod_20251201 (System_pyodps_20251201).

    Visible Scope

    Configure the visibility of the custom image: Visible Only to Creator or Visible to all.

    Module

    Custom images are currently only supported for DataStudio.

    Supported Task Type

    • DataWorks official runtime image for Shell nodes: supports the Shell node type.

    • DataWorks official runtime image for PyODPS nodes: supports PyODPS 2 and PyODPS 3 node types. For this topic, select PyODPS 3.

    • DataWorks official runtime image for EMR DataLake Insight V5.15.1: supports EMR Spark, EMR Spark SQL, and EMR SHELL node types.

    Installation Package

    Add third-party packages as needed. The following methods are supported:

    • Quick Install: Select Python2, Python3, or Yum from the package drop-down list to directly select the environment and resources to install.

      If the package is not listed, switch to Script mode for manual installation.
    • Manual Input: Select Script from the package drop-down list. You can manually enter installation commands in the script editor. Use the following example commands to download third-party packages.

      • pip example: pip install xx (for Python 2).

      • pip3 example: /home/tops/bin/pip3 install 'urllib3<2.0' (for Python 3).

      • yum example: yum install -y git.

      • wget example: wget git.

        For more installation commands, see Installation commands.
    • For this topic, select Script. In the script command box, enter the command to install MaxFrame. If MaxFrame is already installed, this command upgrades it to the latest version. The code is as follows.

      /home/tops/bin/pip3 install --upgrade maxframe
    Important

    To install packages from the Internet, the VPC bound to the Serverless resource group must have Internet access.

    For more information, see Create a custom image.

Step 2: Publish the runtime image

On the Image Management > Custom Images tab of the DataWorks console, Publish the target image. You can publish only successfully tested images. If testing fails, click image > Modify in the Actions column to modify the image configuration.

Note the following when testing and publishing:

  • Select a serverless resource group when testing custom images.

  • For images based on ACR or personal development environments, ensure the serverless resource group VPC matches the image container VPC.

  • If your custom image fetches third-party packages from the Internet and testing fails for a long time, check whether the VPC bound to the test resource group has Internet access capability.

Step 3: Change the workspace of the runtime image

  1. On the Image Management > Custom Images tab of the DataWorks console, find the published custom image.

  2. Click image > Change Workspace in the Actions column to bind the custom image to a workspace.

Step 4: Use the runtime image

Use image in old DataStudio

  1. Enter DataStudio: Log on to the DataWorks console. After switching to the target region, click Data Development and O&M > Data Development in the left navigation pane. Select the corresponding workspace from the drop-down list and click Go to Data Development.

  2. Configure image: In DataStudio, find the task node to test with the custom image, click Properties on the right, and configure resource properties.

    • Resource Group: Select a serverless resource group.

      If the target resource group is not displayed, check whether the resource group is bound to the current workspace. You can go to the Resource Group List page, find the target resource group, and click Associate Workspace in the Actions column to complete the binding.
      Important

      The resource group must match the test resource group used during image publication.

    • Image: Select the published Custom Image.

      If you switch images, you must publish the node for the change to take effect in the production environment.

      image

  3. Debug node: Click Run with Parameters (image) in the top toolbar, configure Resource Group, CUs for Running, and Image, and then click Run.

  4. Deploy node: Click Save and Submit in the top toolbar to publish the node to the production environment.

Use image in new DataStudio

  1. Enter DataStudio: Go to the DataWorks Workspace List page, switch to the target region in the top navigation bar, find the target workspace, and click Shortcuts > Data Studio in the Actions column.

  2. Configure image: In DataStudio, find the task node to test with the custom image, click Scheduling on the right, and configure resource properties.

    • Resource Group: Select a serverless resource group.

      If the target resource group is not displayed, check whether the resource group is bound to the current workspace. You can go to the Resource Group List page, find the target resource group, and click Associate Workspace in the Actions column to complete the binding.
      Important

      The resource group must match the test resource group used during image publication.

    • Image: Select the published Custom Image.

      If you switch images, you must publish the node for the change to take effect in the production environment.

      image

  3. Debug node: In the Run Configuration panel on the right, configure Compute Resource, Resource Group, Compute CUs, Image, and Script Parameters, and then click Run in the top toolbar.

  4. Deploy node: Click Deploy in the top toolbar to publish the node to the production environment.

For more information about DataWorks custom runtime images, see Custom image.