Create an Aliyun EMR Serverless Spark compute source

Updated at:

Prerequisites

Procedure

  1. In the top navigation bar of Dataphin, choose Planning > Compute Source.

  2. On the Compute Source page, click New Compute Source, and then select Aliyun EMR Serverless Spark Compute Source.

  3. On the New Aliyun EMR Serverless Spark Compute Source page, configure the following parameters.

    Parameter

    Description

    Basic information

    Compute source type

    Select Aliyun EMR Serverless Spark.

    Compute source name

    The name can be up to 64 characters and may include Chinese characters, letters, digits, underscores (_), and hyphens (-).

    Configuration method

    Only Reference Specified Cluster is supported. You can search for a cluster by keyword. After selecting a cluster, click View to review its details on the View Aliyun EMR Serverless Spark Cluster page.

    Compute source description (Optional)

    Enter a description for the compute source, up to 128 characters.

    Connection configuration

    Workspace

    The workspace is inherited from the selected cluster and cannot be modified.

    Catalog

    Select a Catalog. The dropdown list includes all Catalogs in the selected cluster.

    Schema

    Select a Schema. The dropdown list includes all Schemas in the selected Catalog.

    Gateway configuration

    Default gateway for production jobs

    The dropdown list contains all Gateways added to the selected cluster. It displays the following information for each Gateway: status, specifications, Spark version, Kyuubi Service Engine Version, queue name, Access Method, and the number of High Availability (HA) replicas.

    Gateway for other jobs

    Gateway for priority jobs

    By default, this is the same as the gateway for production jobs. If you select Custom, you must also select a Gateway for each priority level.

  4. Click Submit.

    Once created, you can bind the Aliyun EMR Serverless Spark compute source to a Project. For more information, see Create a general-purpose project.