Activate Realtime Compute for Apache Flink

更新时间:
复制 MD 格式

To use Realtime Compute for Apache Flink, you must activate a workspace. Each workspace has isolated compute resources and an independent development console. This topic describes how to activate a Flink workspace and important considerations during the process.

Prerequisites

You must have an Alibaba Cloud account. If you do not have one, register first. For more information, see Account registration (PC).

If you access the service using a Resource Access Management (RAM) user or RAM role, you must have the required permissions for the Realtime Compute console. For more information, see Permission Management.

Important considerations

  • Resource deduction plans offset compute resources billed on a pay-as-you-go basis. To use a resource deduction plan, activate a pay-as-you-go workspace.

  • Creating a Flink workspace involves other cloud services, including VPC, OSS, and monitoring-related services. For pricing details, see Billing items.

  • Realtime Compute for Apache Flink uses a storage-compute disaggregation architecture. The storage type can be fully managed storage (in public preview) or an OSS Bucket. To bind an OSS Bucket,when binding an OSS Bucket, note the following:

    • If you have not activated OSS, see Activate OSS to get started. The OSS bucket must be in the same region as your Flink workspace and use the Standard storage class.

    • Do not apply any policies—such as versioning or compliance retention—to the bound OSS bucket. These policies may disrupt normal Flink job execution.

      If versioning is enabled on your OSS bucket, excessive delete markers can slow down list operations. If you enable versioning, configure a lifecycle rule to periodically purge delete markers and unnecessary historical versions. This reduces storage costs and improves OSS performance. For instructions on purging delete markers, see Use lifecycle rules based on last modified time with versioning to reduce storage costs.

      Warning

      If versioning is disabled on your OSS bucket and you configure a lifecycle rule, do not apply a purge policy to Flink-related directories—including artifacts, flink-jobs, flink-savepoints, sql-artifacts, plan, and flink-sessionclusters. A purge policy may delete essential Flink files and cause job failures.

    • If you select OSS Bucket as the storage type during workspace activation, ensure the bound bucket uses zone-redundant storage to achieve zone-level high availability for your Flink workspace. (For configuration steps, see Change the storage redundancy type of a bucket.) Otherwise, if a zone becomes unavailable, state data cannot be accessed normally, and Flink cannot guarantee stateful job recovery.Fully managed storage automatically selects the optimal storage redundancy type. You do not need to manage this manually.

      Note

      If a workspace contains both zone-redundant and single-zone project spaces, upgrading the bound bucket to zone-redundant storage may increase storage costs. To balance cost and reliability, create a separate workspace dedicated to zone-redundant workloads and bind it to a zone-redundant OSS bucket. For OSS storage pricing details, see Storage fees.

    • If you select OSS Bucket as the storage type, ensure malicious file detection is disabled in Security Center. Enabling this feature may trigger OSS traffic throttling and cause Flink jobs to stall during initialization. For more information, see Malicious file detection.

Activate a subscription workspace

  1. Log on to the Realtime Compute Management Console.

  2. Click Buy Now next to Stream Processing Flink.

  3. On your first purchase, grant Flink permission to access the required cloud resources.

    Click Go to RAM for Authorization. On the Cloud Resource Access Authorization page, click Agree to Authorize. This completes automated authorization and grants access to the required cloud services.

  4. On the purchase page, enter the configuration details.

    Category

    Configuration item

    Description

    Example

    Basic configuration

    Billing method

    Select subscription. For billing details, see Subscription.

    Subscription

    Duration

    Select the subscription duration you need.

    1 month

    Auto-renewal

    If you do not check Enable, you can later modify your renewal policy in the Alibaba Cloud Management Console under Billing > Renewals.

    -

    Processor architecture

    Choose between Intel X86 and YiTian ARM architectures to meet your requirements.

    Note

    YiTian ARM architecture is currently available only for subscription workspaces in China (Beijing), China (Ulanqab), and China (Hangzhou). Availability is subject to what is displayed on the console purchase page.

    Intel X86 architecture

    Region

    We recommend selecting the same region as your upstream and downstream storage systems.

    Beijing

    Network configuration

    Deployment mode

    This option is available only for Intel X86 workspaces in specific regions with the subscription billing method. Supported deployment modes include the following:

    • Single zone (cost-effective): Flink automatically assigns the optimal compute zone in the selected region, pools inventory across zones, and schedules resources seamlessly to enhance elastic load capacity.

    • Cross-zone (high availability): Enables zone-level high availability. If the zone hosting a cross-zone project space fails, jobs automatically reschedule to another stable zone in the same region. This prevents service interruptions caused by single-zone failures and ensures job continuity and high availability.

      Important
      • End-to-end high availability requires your upstream and downstream systems to also support high availability.

      • If you did not select Cross-zone for the Deployment Mode when activating the service, you can later enable same-region high availability through other methods, but you must ensure that your workspace has sufficient cross-zone CU resources. For more information, see methods 2 and 3 in same-region high availability.

    -

    CU information

    • Deployment mode is Single Zone: you only need to configure Single Zone compute resources.

    • When the deployment mode is Multi-zone, you can clear the Enable Single-zone Hybrid Purchase option as needed.

      • If not selected: Configure only cross-zone compute resources.

      • If selected: Set separate compute resource quotas for single zone and cross-zone.

    Note

    Each Realtime Compute development console and its essential components require 2 CUs of management resources. These are configured automatically and require no manual action. For details, see Billing items.

    -

    Virtual private cloud (VPC)

    Select a VPC in the target region. You cannot change the VPC after activation. To create a VPC, see Create a VPC and vSwitch.

    • If your upstream or downstream storage resides in a VPC, your Flink workspace must use the same VPC.

    • Only the RFC-standard private CIDR blocks 192.168.0.0/16, 172.16.0.0/12, and 10.0.0.0/8—and their subnets—are supported.

    flink-test-vpc

    Virtual switch

    If deployment mode is cross-zone, select two virtual switches in different zones.

    Each workspace requires 2–3 IP addresses. Select at least one virtual switch per zone. Plan your CIDR blocks based on your Flink job scale.

    Note
    • Only existing virtual switches in the target VPC and Flink-supported zones are displayed. After activation, you can modify virtual switches.

    • The zones of the selected switches are unrelated to the zones where jobs run. They are used only for network connectivity to ensure your jobs can access network resources.

    • Do not modify or delete these switch configurations arbitrarily, as this may disrupt network connectivity and job execution.

    flinktest

    Workspace configuration

    Workspace name

    Start with a lowercase letter. Use only lowercase letters, digits, and hyphens (-). Do not end with a hyphen. Length must be 1–60 characters.

    Important

    The workspace name must be globally unique (including workspaces created by other users). If the name already exists, the system displays Workspace name already exists. Modify the name accordingly. After successful activation, you cannot change the workspace name.

    flinktestnamespace

    Storage configuration

    Storage type

    Choose fully managed storage or OSS Bucket to store job checkpoints, snapshots, logs, JAR packages, and other data.

    Important

    Fully managed storage

    Tag key and tag value

    Tags consist of a tag key and tag value and serve as identifiers for cloud resources. They help you categorize, search, and aggregate resources with common characteristics across dimensions, simplifying resource management. Manage tags centrally in Tag Management.

    -

    Monitoring service

    Monitoring service

    Choose free monitoring service or pay-as-you-go Prometheus monitoring service (for feature differences, see Feature comparison between Cloud Monitor and ARMS alerting):

    • Free monitoring service: After creating a Flink workspace, Cloud Monitor provides basic monitoring and alerting capabilities. Monitoring and alert management are free (data stored for 30 days by default). Only SMS and phone alert notifications incur charges. For details, see Free quota and Pay-as-you-go billing.

    • Prometheus monitoring service: After creating a Flink workspace, ARMS is automatically activated to provide commercial-grade Prometheus monitoring and alerting. For billing details, see Prometheus billing and Alert billing.

    Free monitoring service

  5. Select Terms Of Service, click Buy Now, and complete payment to activate the service.

    After payment, click Management Console to view the workspace being created. Workspace creation typically completes within 5–10 minutes after payment.

    Note

    If you select fully managed storage for storage type and are redirected to the Expenses and Costs payment page instead of completing payment on the current page, ensure both orders are paid. Otherwise, the Flink workspace may fail to activate.

Activate a pay-as-you-go workspace

Important
  • Flink offers a resource deduction plan billing model that offsets compute resources billed on a pay-as-you-go basis. Before activating a pay-as-you-go workspace, you can purchase or claim a resource deduction plan.

  • To activate more than 1,000 CUs at once, submit a ticket.

  1. Log on to the Realtime Compute Management Console.

  2. Click Buy Now next to Stream Processing Flink.

  3. (Optional) On your first purchase, grant Flink permission to access the required cloud resources.

    Click Go to RAM for Authorization. On the Cloud Resource Access Authorization page, click Agree to Authorize. This completes automated authorization and grants access to the required cloud services.

    Note

    If this is not your first purchase, skip this step.

  4. On the purchase page, enter the configuration details.

    Configure the workspace name (cannot be changed after creation; each workspace has isolated compute resources and an independent development console), quota limit (for example, 1,000 CUs), and storage type (OSS Bucket or fully managed storage to store job checkpoints, logs, JAR packages, and other data).

    Category

    Configuration item

    Description

    Example

    Basic configuration

    Billing method

    Select pay-as-you-go. For billing details, see Pay-as-you-go.

    Pay-as-you-go

    Region

    We recommend selecting the same region as your upstream and downstream storage systems.

    Beijing

    Resource deduction plan

    Resource deduction plans offset compute resources billed on a pay-as-you-go basis.

    Before activating a pay-as-you-go workspace, you can purchase or claim a resource deduction plan. For purchase steps and pricing, see Resource deduction plan.

    -

    Network configuration

    Virtual private cloud (VPC)

    Select a VPC in the target region. You cannot change the VPC after activation. To create a VPC, see Create a VPC and vSwitch.

    • If your upstream or downstream storage resides in a VPC, your Flink workspace must use the same VPC.

    • Only the RFC-standard private CIDR blocks 192.168.0.0/16, 172.16.0.0/12, and 10.0.0.0/8—and their subnets—are supported.

    flink-test-vpc

    Virtual switch

    If deployment mode is cross-zone, select two virtual switches in different zones.

    Each workspace requires 2–3 IP addresses. Select at least one virtual switch per zone. Plan your CIDR blocks based on your Flink job scale.

    Note
    • Only existing virtual switches in the target VPC and Flink-supported zones are displayed. After activation, you can modify virtual switches.

    • The zones of the selected switches are unrelated to the zones where jobs run. They are used only for network connectivity to ensure your jobs can access network resources.

    • Do not modify or delete these switch configurations arbitrarily, as this may disrupt network connectivity and job execution.

    flinktest

    Workspace configuration

    Workspace name

    Start with a lowercase letter. Use only lowercase letters, digits, and hyphens (-). Do not end with a hyphen. Length must be 1–60 characters.

    Important

    The workspace name must be globally unique (including workspaces created by other users). If the name already exists, the system displays Workspace name already exists. Modify the name accordingly. After successful activation, you cannot change the workspace name.

    flinktestnamespace

    Compute resource quota limit

    The default quota is 1,000 CUs. You can modify it up to a maximum of 9,999,999.

    Note

    Each Realtime Compute development console and its essential components require 2 CUs of management resources. These are configured automatically and require no manual action. For details, see Billing items.

    100

    Storage configuration

    Storage type

    Choose fully managed storage or OSS Bucket to store job checkpoints, snapshots, logs, JAR packages, and other data.

    Important

    Fully managed storage

    Tag key and tag value

    Tags consist of a tag key and tag value and serve as identifiers for cloud resources. They help you categorize, search, and aggregate resources with common characteristics across dimensions, simplifying resource management. Manage tags centrally in Tag Management.

    -

    Monitoring service

    Monitoring service

    Choose free monitoring service or pay-as-you-go Prometheus monitoring service (for feature differences, see Feature comparison between Cloud Monitor and ARMS alerting):

    • Free monitoring service: After creating a Flink workspace, Cloud Monitor provides basic monitoring and alerting capabilities. Monitoring and alert management are free (data stored for 30 days by default). Only SMS and phone alert notifications incur charges. For details, see Free quota and Pay-as-you-go billing.

    • Prometheus monitoring service: After creating a Flink workspace, ARMS is automatically activated to provide commercial-grade Prometheus monitoring and alerting. For billing details, see Prometheus billing and Alert billing.

    Free monitoring service

  5. Select Terms Of Service, click Buy Now, and complete payment to activate the service.

    After payment, click Management Console to view the workspace being created. Workspace creation typically completes within 5–10 minutes after payment.

    Note

    If you select fully managed storage for storage type and are redirected to the Expenses and Costs payment page instead of completing payment on the current page, ensure both orders are paid. Otherwise, the Flink workspace may fail to activate.

What to do next

After activating your Flink workspace, perform the following tasks as needed.

Feature

Access path

View workspace details

In the Realtime Compute Management Console, click More > Workspace Details for your target workspace to view the workspace name, workspace ID, OSS bucket (displayed only if you selected OSS Bucket for storage type during activation), VPC name and ID, virtual switches, storage type, and storage usage.

Enable and use elastic compute resources

You can configure elastic compute resources on top of a subscription plan. This hybrid billing model combines the flexibility of pay-as-you-go with the cost efficiency of subscription plans. For details, see Hybrid billing.

Basic Concepts

Understand the hierarchy of basic concepts in Realtime Compute for Apache Flink to guide job development, deployment, operations, and security management.

Permission Management

RAM users and RAM roles require RAM permissions (for example, workspace viewing and purchasing) and project space permissions (for example, job development and operations) to access the Flink console.

Add a project space

A project space is the basic unit for managing Flink jobs. All your configurations, jobs, and permissions reside within a single project space. After workspace creation, a default project space (with the suffix "default") is generated. workspace-name-default is the default single-zone project space, or workspace-name-default-ha is the default cross-zone project space. You can create additional project spaces to allocate separate resources and permissions, enabling full resource and permission isolation for multi-tenant scenarios.

Job development and Operations management

Click Console in the Actions column for your target workspace to enter the Realtime Compute development console. Perform job development and operations tasks here.

The service includes built-in connectors for various upstream and downstream systems. For details, see Supported connectors.

Resource adjustment

Adjust resources for your workspace and project spaces.

Switch between billing methods

Switch between pay-as-you-go and subscription billing methods.

Modify virtual switches

Modify virtual switches to resolve job startup failures caused by insufficient IP addresses.

Release resources

Click More > Release Resources for your target workspace. After release, the Flink workspace stops incurring charges, and associated ARMS resources are released simultaneously.

Note

Billing records may lag behind actual usage. For details, see Billing usage notes.

References