Manage triggers

How it works

A DataWorks trigger monitors predefined events. When a specified event occurs, the trigger starts the associated event-triggered workflow and passes event information as parameters.

• Multiple event sources: Triggers monitor events from different sources.

  • External storage events: Monitors new object creation in an OSS bucket.

  • External message events: Monitors new messages in message queues such as Kafka and RocketMQ.

• Dynamic parameter passing: When an event is detected, the trigger captures event context and passes it to the downstream workflow as parameters.

  • For OSS events: file details.

  • For message queue events: the full message content, including the key, value, and headers.

  Workflow nodes can then use these parameters to process data based on the event.

• Supported types:

  • Message queues: Kafka, RocketMQ, and RabbitMQ.

  • Storage objects: OSS.

Usage notes

• Regions: Available only in China (Hangzhou), China (Beijing), China (Zhangjiakou), China (Ulanqab), China (Shenzhen), China (Hong Kong), and Singapore.

• Editions: Available only in DataWorks Professional Edition or later. To upgrade, see upgrade DataWorks to Professional Edition or a later edition.

Billing

When an event-triggered workflow runs with a configured trigger, EventBridge fees are incurred in addition to scheduled task run fees. Billing of event streams. You are charged based on the number of events.

Prerequisites

• activate EventBridge and grant permissions and activate Message Service.

• An event source in the same region as your workspace, such as an OSS instance or a cloud message queue (Kafka, RocketMQ, or RabbitMQ), with the required access permissions.

• A user added to the workspace with the Developer, Operator, or Workspace Administrator role. Manage members and roles.

Go to the trigger management page

1. Log on to the DataWorks console. In the target region, click Data Development and O&M > Operation Center in the left-side navigation pane. Select a workspace from the drop-down list and click Go to Operation Center.

2. In the left-side navigation pane of Operation Center, choose Others > Scheduling Configuration, and then click the Trigger Management tab.

Create a trigger

If you have the developer, operator, or workspace administrator role, you can create a trigger on the Trigger Management tab.

1. Click the Create Trigger button to go to the Create Trigger configuration page.

   Note

   • DataWorks event triggering depends on EventBridge. If you use this feature for the first time or the required service-linked role is missing, click Authorize to grant access.

   • Required permissions: ram:CreateServiceLinkedRole and arms:CheckServiceStatus. Permission list.

2. Configure the following parameters.

   Kafka trigger

   Parameter	Description
   Applicable Workspace	Select the workspace. Only workspaces with the new Data Studio are supported.
   Applicable Environment	Triggers apply only to the production environment. In the development environment, you must manually specify parameters.
   Owner	Select the trigger owner.
   Trigger Type	Select ApsaraMQ for Kafka.
   Trigger Event	The alikafka:Topic:Message event type is supported — the trigger activates when Kafka messages arrive. To learn how to send messages, see Send and receive Kafka messages.
   Kafka Instance	Select a Kafka instance in the same region as the workspace. If none exists, Purchase a Kafka instance.
   Topic	Select the topic to monitor. If none exists, Create a topic.
   Key	Optional. Specify a preset key value. The trigger activates only when the message key matches exactly. If left empty, any consumed message triggers the workflow.
   ConsumerGroupId	Select Quick Create or Use Existing. Quick Create auto-generates a group ID.
   Message Format Example	A fixed Kafka message example. You can use this in the triggered workflow. You can use ${workflow.triggerMessage} in internal tasks to obtain the full message body, or use ${workflow.triggerMessage.xxx} to obtain the value of a specific field in the message body.

   RocketMQ trigger

   Parameter	Description
   Applicable Workspace	Select the workspace. Only workspaces with the new Data Studio are supported.
   Applicable Environment	Triggers apply only to the production environment. In the development environment, you must manually specify parameters.
   Owner	Select the trigger owner.
   Trigger Type	Select ApsaraMQ for RocketMQ. Note Versions earlier than 5.x are not supported. Version 5.x is used by default.
   Trigger Event	The mq:Topic:SendMessage event type is supported — the trigger activates when RocketMQ messages are consumed.
   ApsaraMQ for RocketMQ Instance	Select a RocketMQ instance in the same region as the workspace. If none exists, Manage RocketMQ instances.
   Topic	Select the topic to monitor. If none exists, Create a topic.
   Tag	Optional. Specify a preset tag value. The trigger activates only when the message tag matches exactly. If left empty, any consumed message triggers the workflow.
   Security Group	If the RocketMQ instance uses subscription billing, you can select a security group.
   Consumer Group	Select Quick Create or Use Existing. Quick Create auto-generates a group ID.
   Message Format Example	A fixed RocketMQ message example. You can use this in the triggered workflow. You can use ${workflow.triggerMessage} in internal tasks to obtain the full message body, or use ${workflow.triggerMessage.xxx} to obtain the value of a specific field in the message body.

   RabbitMQ trigger

   Parameter	Description
   Applicable Workspace	Select the workspace. Only workspaces with the new Data Studio are supported.
   Applicable Environment	Triggers apply only to the production environment. In the development environment, you must manually specify parameters.
   Owner	Select the trigger owner.
   Trigger Type	Select ApsaraMQ for RabbitMQ.
   Trigger Event	The amqp:Queue:SendMessage event type is supported — the trigger activates when RabbitMQ messages are consumed.
   ApsaraMQ for RabbitMQ Instance	Select a RabbitMQ instance in the same region as the workspace. If none exists, Create a RabbitMQ instance.
   Vhost	The RabbitMQ virtual host, used for logical isolation of queues. If none exists, Create a vhost.
   Queue	Select the queue to monitor. If none exists, Create a queue.
   Message Format Example	A RabbitMQ message body example. You can use this in the triggered workflow. You can use ${workflow.triggerMessage} in internal tasks to obtain the full message body, or use ${workflow.triggerMessage.xxx} to obtain the value of a specific field in the message body.

   OSS trigger

   Parameter	Description
   Applicable Workspace	Select the workspace. Only workspaces with the new Data Studio are supported.
   Applicable Environment	Triggers apply only to the production environment. In the development environment, you must manually specify parameters.
   Owner	Select the trigger owner.
   Trigger Type	OSS.
   Trigger Event	Three event types are supported: oss:ObjectCreated:PutObject: Upload a file. oss:ObjectCreated:PostObject: Upload a file by using an HTML form. oss:ObjectCreated:CompleteMultipartUpload: Complete a multipart upload.
   Bucket Name	Select the OSS bucket to use as the event source. If none exists, create an OSS bucket.
   File Name	Specify the file name pattern that triggers the event. Wildcard matching is supported: Prefix matching: Example: task*. Description: An event is triggered when you upload a file with the prefix task to OSS, such as task10.txt. Suffix matching: Example: *task.txt. Description: An event is triggered when you upload a file with the suffix task.txt to OSS, such as work_task.txt. Flexible matching: Example: *task*. Description: An event is triggered when you upload a file that contains the string task to OSS, such as work_task.txt.

3. Click Confirm to complete the trigger creation.

Use a trigger

A trigger must be used with an event-triggered workflow. Only an event-triggered workflow submitted to Operation Center can be triggered.

After the event-triggered workflow is submitted to Operation Center, the tasks in the workflow are automatically triggered when the monitored event occurs.

1. Create an event-triggered workflow

2. Configure the schedule settings and specify the trigger you created in Scheduling Policy.

3. Nodes inside an event-triggered workflow are configured the same way as nodes in a regular workflow.

   The key difference is the execution mechanism: instead of running on a fixed schedule, the workflow is driven by a trigger that responds to external events.

Manage triggers

On the Trigger Management tab, find the target trigger to view referenced tasks, edit the trigger, or view and roll back versions.

• View Nodes That Reference Calendar: When the trigger is referenced by an event-triggered workflow, click View Nodes That Reference Calendar in the Actions column to see which workflows reference the trigger on the View Nodes That Reference Calendar page.

• Edit a trigger: Click Edit in the Actions column. Edit the trigger information on the Modify Trigger page and click Confirm.

  Note

  The system automatically creates a new version for the trigger after modification.

• View Version:

  1. Click Version in the Actions column to view all historical versions on the View Versions page.

  2. Click View in the Actions column to view version details.

  3. To roll back to a historical version, click Roll Back next to the version. Enter rollback Remarks and click OK.

     Note

     A rollback automatically generates a new version based on the selected historical version.

• Delete a trigger: Before deleting a trigger, undeploy and delete all tasks that reference it. Then, click Delete and click Confirm to confirm.

Next steps

After creating a trigger, use it in an event-triggered workflow. Create an event-triggered workflow.