Real-time instances

Section

Description

①Filter and search area

You can search for a real-time instance by its name or ID.

• My stopped instances: Instances that you own with a Stopped status.

• Instances I operated today: Instances that you own and have operated on today.

• Project: Filter instances by project. You can select multiple projects for which you have the O&M - Access Directory permission.

• Expand Filters: Click Filter to show all filter options. The filter options include:

  • Subtype: Includes Flink SQL, Flink Template SQL, Flink DataStream, Real-time Workflow.

  • Running status: Filter instances by one or more statuses. Options include Succeeded, Starting, Start Failed, Running, Failed, Stopping, and Stopped.

  • Last operator: Filter instances by the user who last operated on them.

  • Owner: Filter instances by their O&M owner.

  • Engine type: Filter instances by engine type. Options include Ververica Flink, Blink.

  • Cluster: The cluster ID.

  • Engine version: The engine version of the instance.

  • Operation date: The date an operation was performed on the instance.

• Reset: After expanding the filter options, click Reset to clear all selected filter criteria.

②List operations

• View instance statistics: Click to view detailed statistics for the instance. For more information, see View Instance Statistics.

• Export instance list: Click to export the filtered or searched instances. The list is exported in .csv format.

• Customize columns: Click the icon to select which columns to display and to adjust their order.

  Note

  • The first column and the Actions column cannot be hidden or reordered.

  • If you clear your browser's cache or use a different browser, the browser resets the column settings to their defaults.

• Refresh list: Click the icon to refresh the current list.

③Real-time instance list

The real-time instance list displays real-time instances. The list includes information such as the instance, running status, last operator, last updated time, owner, and project. You can resize each column. Your column width settings are saved locally in your browser. Clearing your browser's history and cookies removes these settings.

• Instance: The system creates a real-time instance after a real-time task is submitted to the O&M center. Its initial running status is Stopped. You can start it manually.

• Running status: The current status of the instance. Possible statuses include Succeeded, Starting, Running, Stopping, Stopped, Failed, and Start Failed. For more information, see .

• Consumption latency (s): The delay when consuming data from the data source.

• Last operator: The account that last operated on the instance.

• Last updated time: The time the last operation was performed on the instance.

• Owner: The O&M owner of the instance.

• Project: The project to which the instance belongs. The display format is Project English Name (Project Chinese Name).

• Engine type: The engine type used by the real-time instance.

• Cluster: The cluster on which the real-time instance runs. This field applies only to the Ververica Flink real-time engine and is empty for other engines.

• Resource queue: The resource queue in which the real-time instance runs. This field applies only to Blink real-time engines and is empty for other engines.

• Engine version: The engine version of the instance.

The Actions column lists the O&M operations available for real-time instances. For information about individual operations, see Individual Operations.

④Batch operations

You can use the Start, Stop, and Take Offline operations in the batch operations area to manage multiple real-time instances at once.

For information about batch O&M operations on real-time instances, see Batch Operations.

Icon

Status

Description

Succeeded

Applies to instances that process bounded stream data and have completed execution.

Starting

The instance is starting but not yet running.

Running

The instance is currently running.

Stopping

The system is stopping the instance.

Stopped

The instance is stopped.

Failed

The instance failed during execution.

Start Failed

The instance failed to start.

Operation

Description

Start

In the Actions column for the real-time instance, click the icon to start the instance. For more information, see Start a real-time instance.

Stop

In the Actions column for a Running real-time instance, click the icon to stop it. Two stop modes are available: stateless stop and stop with savepoint.

• Stateless stop: Does not create a savepoint. The job starts without any state the next time it runs.

  Important

  A stateless stop clears any saved savepoints but does not affect checkpoints.

• Stop with savepoint: Creates a savepoint to retain the current state. You can choose to start the job from this state the next time it runs.

Take offline

In the Actions column for the real-time instance, click the icon to take the instance offline from the production environment. You can take an instance offline if it is in the Stopped, Failed, or Start Failed state.

View Run Analytics

In the Actions column for the real-time instance, click the icon to view its operational information, data curves, failover details, and other parameters.

If the real-time instance uses the Ververica Flink or Blink compute engine, see View Job Details.

View Real-time Task

In the Actions column for the real-time instance, click the icon and select View Real-time Task to open the page for the job that generated the instance and view its details.

View Instance Code

In the Actions column for the real-time instance, click the icon and select View Instance Code to view the code of the job that generated the instance.

View Instance Parameters

In the Actions column for the real-time instance, click the icon and select View Instance Parameters to view the runtime parameters for the instance, such as metrics.reporter.promappmgr.port.

Operation

Description

Start

Start Real-time Development Instances and Start Real-time Workflow Instances:

1. Select one or more real-time development instances, or click Select All on This Page, then click Start and choose Start Real-time Development Instances or Start Real-time Workflow Instances.

2. In the Batch Start Real-time Development Instances or Batch Start Real-time Workflow Instances dialog box, configure the start settings.

   • Start n Instances: Shows the number and names of the selected instances, separated by commas (,).

   • Start Mode: You can choose between Stateless Start and Start from a Valid State.

     • Stateless Start: Starts the job without any initial state.

     • Start from a Valid State: Starts the job from an existing valid state.

       Note

       If your selection includes stateless instances, you can only choose Stateless Start for the batch operation.

   • Data Read Point-in-Time: Setting a specific start time causes all source tables that support startTime (such as Log Service, Kafka, MySQL, MongoDB, and OceanBase) to begin reading data from that time.

     Note

     Parameter settings in the SQL job code take precedence over this specified start time.

   • Business Date: You can set the business date to assign values to time variables. Time-based variables are calculated based on this business date, which defaults to the current day.

     Note

     Parameter settings in the SQL job code take precedence over this specified business date.

3. Click OK to start the instances.

Stop

Stop Real-time Development Instances and Stop Real-time Workflow Instances:

1. Select one or more real-time development instances, or check Select All on This Page, then click Stop and choose Stop Real-time Development Instances or Stop Real-time Workflow Instances.

2. In the Batch Stop Real-time Development Instances or Batch Stop Real-time Workflow Instances dialog box, configure the stop settings.

   • Stop n Instances: Shows the number and names of the selected instances, separated by commas (,).

   • Stop Mode:

     • Stateless stop: Does not retain the current state. The job starts without state the next time it runs.

     • Stop with savepoint: Retains the current state, allowing you to start from this state the next time the job runs.

       Note

       Real-time workflow instances only support stop with savepoint.

Take offline

Takes the selected real-time instances offline from the production environment.

Section

Description

①Node Information

Displays summary information for the currently selected node. You can also click View Node Details for more information.

Ververica Flink real-time engine: The available tabs include Node Details, Runtime Log, Operation Log, Node Code, and Task Parameters (only for compute task instances).

• Node Details: Displays basic and runtime information for the current node. Basic information includes the node name, ID, instance ID, and project. Runtime information includes the running status and runtime duration.

• Runtime Log: Displays the runtime logs for the current node. If a run fails and produces dirty data, you can download the dirty data file.

• Operation Log: Shows a history of operations performed on the node, including the operation time, operator, and the operation performed.

• Running Analysis: Displays a dashboard with various runtime metrics over a selected time period, such as the number of failures and backpressure data.

• Logs: Shows the startup log, runtime log, and exception information separately.

• Node Code: Displays the specific code for the current node.

• Task Parameters: Displays the parameters for the current node's task.

②Node Search and Filter

Allows you to quickly set the number of upstream and downstream levels to display around the main node. If the graph becomes too crowded, you can search for a node by name to quickly locate it within the current view.

③Scheduling Dependency Graph

Displays the instance's scheduling dependency graph. You can expand it to show more upstream and downstream nodes and perform O&M operations on them. Hover over a DAG node to view its name, type, scheduling cycle, O&M owner, and description.

④Canvas Adjustment

This section provides controls to adjust the DAG's display scale, including setting a specific zoom level (default is 100%), zooming in (up to 200%), zooming out (down to 20%), fitting to the canvas, and full-screen mode. This area also displays the node ID and name of the main node in the current DAG.

Operation

Description

Expand Parent Nodes

Expands dependency nodes at different levels relative to the main node in the DAG.

Expand Child Nodes

View Real-time Task

Opens the DAG of the task that generated the current instance node, where you can view task node details, upstream and downstream information, and manage the task node. For more information, see Manage real-time tasks.

View Node Code

Views the task code that generated the current instance.

View Runtime Log

Views the runtime log of the instance node. If the task generates dirty data, you can click Download Dirty Data File to download and inspect it locally. The file includes the time the dirty data was generated, error message, and cause of the error. The file is retained for 7 days by default.

View Task Parameters

Views the task parameters for the current instance node, such as execution.checkpointing.interval.

Edit Development Node

This operation applies only to the Dev-Prod development model.

Edit Node

This operation applies only in Basic mode.

View Generation Node

Views the configuration of the task in the Prod project that generated the current instance node.

View Operation Log

Views the operation log for the current instance, including the operation time, operator, and the operation performed.

View Running Analysis

Views the runtime information, data curves, failover details, and other parameters for the current real-time instance.

If the real-time instance uses the Ververica Flink or Alibaba Blink real-time engine, see View Job Details.

Start

Starts the current real-time instance. For specific steps, see Start a real-time instance.

Stop

Stops the current real-time instance, which can be in the Running or Completed state (For instances in the Completed state, only stateless stop is supported). Two stop modes are available: stateless stop and stop with savepoint.

• Stateless stop: Does not create a savepoint. The task starts without any state on the next run.

• Stop with savepoint: Creates a savepoint to retain the current state. You can choose to start the task from this state on the next run.

Decommission

Decommissions the real-time instance from the production environment. Only instances in the Stopped state can be decommissioned.