DocsICP FilingConsole
官方文档
首页

Real-time synchronization task O&M

更新时间：
复制 MD 格式
产品详情
我的收藏

After you develop and publish a task in DataStudio, you can run it in Operation Center. There, you can also monitor the task's status and view its metrics. This topic describes common O&M procedures for real-time synchronization tasks.

Prerequisites

A real-time synchronization task must be created and published. For more information, see Configure a real-time synchronization task for a single table.

Run and manage real-time synchronization tasks

After configuring the task, go to the Operation Center > Real-time Task O&M > Real-time Synchronization Task page to Start, Stop, or Undeploy it. For more information, see Run and manage a real-time synchronization task.

Note

When you start a task, you can set the checkpoint and the maximum number of failovers.

  • Checkpoint: Lets you resume the task from its last known position or start it from a specific point.

  • Failover: To prevent repeated failovers from consuming system resources, you can configure the task to automatically stop if the number of failovers within a specified period exceeds a defined limit. If you do not configure this setting, the task automatically stops if more than 100 failovers occur within a 5-minute period.

Monitor task status

To prevent task errors from causing service latency, go to the Operation Center > Real-time Task O&M > Real-time Synchronization Tasks page. Find your task and click Alarm settings in the Operations column. For more information, see Run and manage a real-time synchronization task.

  • Alert conditions: You can set monitoring alerts for Task status, Business Latency, Failover, Accumulated Messages, and Dirty Data.

  • Notification methods: Alerts can be sent to recipients through Email, Text Message, Telephone, and DingTalk.

  • Fatigue control: To prevent an excessive number of alerts in a short period, Data Integration supports fatigue control. This feature allows you to configure DataWorks to send only one alert for a specific rule within a set time interval.

Task runtime information in Logview

Logview in Data Integration collects tracing data from synchronization tasks, analyzes and processes the data, and visualizes the results. It provides detailed information about the synchronization process, such as synchronization speed and logs.

Go to the Operation Center > Real-time Synchronization Task page. Find the target task and click its name to open the Logview page.logview This page displays the following information:

Parameter

Description

Logs

View detailed logs for the real-time task. You can filter the logs by time.

Progress

View the detailed progress of the task, including trend charts for Business Latency, Synchronized Data Records, Synchronized Bytes, Synchronization Rate for Synchronized Data Records, Synchronization Speed (Bytes/s), and Window Waiting Period.

You can also perform the following operations on this page:

  • Use the time filter to view synchronization information for a specific period.

    Note

    The time filter lets you view detailed information for up to the last 15 days.

  • To the right of the Task List, click the 自定义列 icon to display more columns.

  • In the Task List section, you can switch between the worker list and task list views.

    • The worker list shows the instances started for the real-time task. A non-distributed task has only one worker, whereas a distributed task has multiple workers based on the concurrency level.

    • The task list shows the threads running within a worker. These threads are categorized as readers or writers. Depending on the configured concurrency, a worker can have multiple reader and writer tasks.

    You can also click an item in the task list to view its metric trend chart.

  • In the process list, you can click a specific item to view its metric trend chart.

Database DDL Events

View the database DDL events detected from the source.

Statistics on Database DML Events

View statistical analysis of the source and target tables.

Failover

View the failover status and trend chart for the task. You can also view the logs for each failover event in the log details.

Modify a real-time synchronization task

You can edit the real-time synchronization task in DataStudio. To apply the changes, you must first stop the running task. After you modify the task, resubmit and publish it, and then go to Operation Center to restart the task.

Modify DDL rules

Before you start the synchronization task, go to DataStudio and redefine how the task handles source DDL messages. For more information, see Configure a real-time synchronization task for an entire database.

Note

To apply changes made in DataStudio, you must first stop the running task. After you make the changes, resubmit and publish the task, and then go to Operation Center to restart it.

FAQ