Create a new session cluster

更新时间:
复制 MD 格式

Create a session cluster within a resource group for Flink task development and testing.

Limits

Each tenant is limited to one session cluster.

Permissions

Only super administrators and system administrators can create session clusters.

Procedure

  1. On the Dataphin home page, click Management Center in the top menu bar.

  2. Follow the figure below to open the Session Cluster dialog box.

    image.png

  3. In the Create a new session cluster dialog box, configure the following parameters:

    Parameter

    Description

    Basic information

    Cluster name

    Enter the cluster name. Supports letters, numbers, and underscores (_). Maximum: 64 characters.

    Owner

    Select the cluster owner.

    Engine version

    Select the Flink engine version. Currently only Flink 1.15 is supported.

    Resource configuration

    Task Managers quantity

    Number of Task Managers. Positive integer. Minimum: 1. Default: 1.

    Slot quantity

    Number of slots. Positive integer. Minimum: 1. Default: 8.

    Job Manager CPU Cores

    CPU cores for the Job Manager. Positive integer. Minimum: 1. Default: 1.

    Job Manager Memory

    Memory for the Job Manager in GB. Positive integer. Minimum: 1. Default: 4.

    Task Manager CPU Cores

    CPU cores per Task Manager. Positive integer. Minimum: 1. Default: 2.

    Task Manager Memory

    Memory per Task Manager in GB. Positive integer. Minimum: 1. Default: 8.

    Runtime parameters

    Runtime parameters

    Enter runtime parameters for the session cluster. These serve as defaults when testing Flink tasks.

    Important

    Task-level runtime parameters override session cluster runtime parameters.

    Important

    Total CPU and memory must not exceed the session cluster resource group limits.

    • The session cluster's required CPU usage = (Task Managers quantity x Task Manager CPU Cores) + Job Manager CPU Cores

    • The session cluster's required memory usage = (Task Managers quantity x Task Manager Memory) + Job Manager Memory

  4. Click OK.

    The session cluster starts automatically. Monitor its status in the session cluster list.

What to do next

After the session cluster starts, develop and test Flink real-time tasks. Real-time mode testing.