This topic summarizes the key concepts of Realtime Compute for Apache Flink. Use it as a quick reference to understand how the service organizes its resources and operations.
Hierarchy
The following diagram illustrates the hierarchy of key concepts in Realtime Compute for Apache Flink. Understanding this structure helps you plan your job development, deployments, operations, and security management.
Concepts
Concept | Description | Documentation |
Management Console | The Management Console is the central platform for managing and configuring workspaces. It provides global control over the lifecycle of workspaces and namespaces and lets you flexibly adjust resource allocation based on business needs. Key features include:
| |
Development Console | Each workspace has an independent Development Console where you can switch between target namespaces. The Development Console is for job development and operations. It helps you manage the entire lifecycle of a job, from development to production. Key features include:
| |
Workspace | A workspace is the basic unit for managing namespaces in Realtime Compute for Apache Flink. Each workspace has isolated compute resources and an independent Development Console. | |
Namespace | A namespace is the basic unit for managing jobs. You manage all your configurations, jobs, and permissions within a single namespace. You can create multiple namespaces and assign separate resources and permissions to each, enabling complete resource and permission isolation for different tenants. | |
Resource | The basic unit for computing resources in Realtime Compute for Apache Flink is the Compute Unit (CU). 1 CU = 1 vCPU + 4 GiB of memory + 20 GB of local storage (for logs, system checkpoints, and other information). The CU consumption of a job depends on the Queries Per Second (QPS) of the input data stream, the complexity of the computation, and the specific distribution of the input data. You can estimate the number of resources to purchase based on your business scale and computing requirements. | |
Draft | A draft is an SQL job created on the ETL page or a YAML job created on the Data Ingestion page of the Development Console. This concept applies only to job development within the Development Console and does not apply to development using an SDK. | |
Deployment | A deployment isolates development from production, preventing modifications to a draft from affecting a job that is running online. You can create a deployment from a job draft, an uploaded JAR, or a Python package. Each deployment has defined stream or batch properties. This concept applies to job development through both the Development Console and SDKs. | |
Job instance | A job instance is a running instance of a job in a production environment. It is generated from a deployment and has defined stream or batch properties. | None |
Connector | Realtime Compute for Apache Flink provides many built-in connectors to read from and write to various upstream and downstream systems. It also supports uploading and using custom connectors. | |
Function | Realtime Compute for Apache Flink supports two types of functions: built-in functions and user-defined functions (UDFs). | |
Catalog | A catalog in Realtime Compute for Apache Flink provides metadata from external systems. This metadata includes information about databases, tables, fields, and partitions. | |
Role | A role is a security concept in Realtime Compute for Apache Flink that represents a set of users with the same permissions. Multiple users can belong to one role, and one user can belong to multiple roles. After you grant permissions to a role, all users in that role inherit the same permissions. | |
Member | A member is a security concept in Realtime Compute for Apache Flink. You can add Alibaba Cloud accounts and RAM users as members. A member (other than the project owner) must be added to the project and granted permissions to operate on its data, jobs, resources, and functions. | |
Resource queue | You can deploy jobs to specific queues for resource isolation and management. |