Using Function Compute as a transform-EventBridge(EventBridge)-阿里云帮助中心

This topic shows you how to use Function Compute to transform events. By writing custom code, you can perform complex operations like enriching event data, splitting payloads, converting formats, or dynamically routing events before they reach the sink.

How it works

When you select Alibaba Cloud's Function Compute for the transform, you can write function code to perform more complex and customized processing on events. The overall process is shown in the figure.

The source pulls events and groups them into batches. After the batching conditions are met, the events move to the filter stage.
The filter stage evaluates each event in a batch against your event pattern and drops events that do not match. The remaining events move to the transform stage.
The transform stage sends the batch of filtered events to a Function Compute function. If a batch exceeds the function's payload limit, the stage splits the batch into smaller groups that comply with the limit and invokes the function sequentially for each group.
After the function is invoked, the transform stage sends the function's return value to the sink.

Usage notes

Item	Description
Function invocation mode	You can only invoke Function Compute functions synchronously. The maximum request payload is 16 MB.
Function input format	For multiple CloudEvents, the input format is an array: `[cloudEvents1,cloudEvents2]`. For a single CloudEvent, the input format is also an array: `[cloudEvents1]`. For more information about the CloudEvents format, see Event overview.
Function execution timeout	If you create a function from a template, the default timeout is 60s. If this timeout does not meet your requirements, you can change it in the Function Compute console after creating the function. For more information, see Manage functions. If you bind an existing function, the function's configuration determines the timeout.
Function return value	The function can return any value, subject to the following constraints: Due to differences in how Function Compute runtimes handle serialization, what your function code returns may not be identical to the value the event stream receives. For a detailed comparison, see the section on return value differences below. The data format sent to the sink is based on the value that the event stream receives. The size of the return value is limited by the sink's constraints. If the sink is a service other than Function Compute, such as Simple Message Queue (formerly MNS), ApsaraMQ for RocketMQ, or ApsaraMQ for Kafka, the size of each data record in the return value must not exceed the sink's maximum message size. If the return value is an array, this limit applies to each element in the array. If the return value is not an array, the limit applies to the entire value. If the sink is Function Compute, the total size of the return value cannot exceed 16 MB.
Exception handling	The transform stage uses the same exception handling policy and fault tolerance settings configured for the event stream.

Return value differences by runtime

Due to a limitation in Function Compute, what your function code returns can differ from what the event stream receives when you use different runtimes. The following table shows the differences for Python, Node.js, and Go.

Runtime	Function returns	Event stream receives
Python 3.9	["test1", "test2"]	["test1", "test2"]
	"[\"test1\", \"test2\"]"	["test1", "test2"]
	"test"	test
	"\"test\""	"test"
Node 14	["test1", "test2"]	["test1", "test2"]
	"[\"test1\", \"test2\"]"	["test1", "test2"]
	"test"	test
	"\"test\""	"test"
Go 1.x	["test1", "test2"]	["test1", "test2"]
	"[\"test1\", \"test2\"]"	"[\"test1\", \"test2\"]"
	"test"	"test"
	"\"test\""	"\"test\""

Prerequisites

Step 1: Create an event stream

Log on to the EventBridge console. In the left-side navigation pane, click Event Streams.
In the top navigation bar, select a region and click Create Event Stream.

On the Create Event Stream page, enter a Task name and an optional Description, then configure the following parameters.

Configure the event pipeline

In the Source step, select a Data Provider. This example uses ApsaraMQ for Kafka. Configure the following parameters, then click Next Step.

Parameter	Description	Example
Region	Select the region where the source ApsaraMQ for Kafka instance is located.	China (Beijing)
ApsaraMQ for Kafka Instance	Select the source instance that produces the ApsaraMQ for Kafka messages.	alikafka_post_115964845466****_ByBeUp3p
Topic	Select the topic that produces the ApsaraMQ for Kafka messages.	topic
Group ID	The consumer group of the source instance. To prevent disruptions, use a dedicated consumer group for the event source. Do not reuse a consumer group that other services already use.	Quickly Create
Consumer Offset	The position where message consumption starts.	Latest Offset
Network Configuration	Select the network type for message routing.	Basic Network
VPC	Select the VPC ID. This parameter is required only when Network Configuration is set to Self-managed Internet.	vpc-bp17fapfdj0dwzjkd****
vSwitches	Select the vSwitch ID. This parameter is required only when Network Configuration is set to Self-managed Internet.	vsw-bp1gbjhj53hdjdkg****
Security Group	Select a security group. This parameter is required only when Network Configuration is set to Self-managed Internet.	alikafka_pre-cn-7mz2****
Messages (optional)	Batching allows you to group multiple events. A batch is sent when either the Messages count or the Interval (Unit: Seconds) is reached. This parameter sets the maximum number of messages to include in a batch. A request is sent only when the number of messages reaches this value. Valid values: 1 to 10,000.	100
Interval (Unit: Seconds) (optional)	The time interval for sending aggregated messages to the function. The system aggregates messages and sends them at this interval. Valid values: 0 to 15 seconds. A value of 0 indicates that batches are sent immediately without a waiting period.	3

In the Filtering step, configure the Pattern Content. The default value is {}, which means no events are filtered. For more information, see Event patterns.
Note
The Filtering step is optional. You can click the Delete button in the upper-right corner of the step to remove it, or click +Filtering (Filter) to add it. The position of the Filtering step in the pipeline does not change.

In the Transformation step, set Alibaba Cloud Service to Function Compute and configure the following parameters. This example uses a function template to create a new service and function.

Configuration method	Parameter	Description
Create function template	Service	The name of the service in Function Compute. A system-generated random value is provided, which you can edit as needed. After the task is saved, a service with this name is created in Function Compute.
	Function	The name of the function in Function Compute. A system-generated value is provided, which you can edit. When you save the task, a function with this name is created in Function Compute.
	Function template	Four default function templates are provided: Content Splitting Content Mapping Content Enrichment Dynamic Routing You can use the provided code editor to edit and debug the template code. You do not need to save the code separately. When you save the task, the code in the editor is used to create the function.
Bind existing function	Service	Select an existing service in Function Compute.
	Function	Select a function within the selected service.
	Version and alias	Select a version or alias of the service.

In the Sink step, select a Service Type and configure the relevant parameters. This example selects Function Compute.

Parameter	Description	Example
Service	Select an existing Function Compute service.	test
Function	Select an existing Function Compute function.	test
Service version and alias	Select a service version or alias.	Default Version
Invocation Mode	Select synchronous or asynchronous invocation.	Asynchronous
Event	Select an event transformation type. For more information, see Event content transformation.	Complete Event

Task properties (optional)
Configure a retry policy and a dead-letter queue for the event stream. For more information, see Retries and dead-letter queues. This example uses a Backoff Retry policy, enables Fault Tolerance Allowed, and does not use a dead-letter queue.

After you complete the configuration, click Save.
On the Event Streams page, find the event stream that you created. In the Actions column, click Enable.
After you enable the event stream, it takes 30 to 60 seconds to start. You can monitor the progress in the Status column on the Event Streams page.

Step 2: Verify the event stream

On the Event Streams page, find the event stream that you created. In the Event Sources column, click the Kafka topic to navigate to the ApsaraMQ for Kafka console. In the upper-right corner, click Send Message.
In the Start to Send and Consume Message panel, set the Sending method, Message key, and Message Content, then click OK.
Verify that the function used in the transform stage received the request.
Return to the Event Streams page and click Details in the Actions column. In the Transformation area, click the function name.
- Click the Invocation Logs tab to view the logs. If the Invocation Results column shows Success, the transform was executed successfully.
  Note
  If Simple Log Service is not active, you must activate it first and then send another Kafka message.
- Click the Monitoring tab and check for invocations. If there are invocations and zero errors, the sink successfully received the transformed content.
Verify that the sink function received the request.
Return to the Event Streams page and click Details in the Actions column. In the Event Targets column, click the function name.
- Click the Invocation Logs tab to view the logs. If the Invocation Results column shows Success, the sink function was invoked successfully.
  Note
  If Simple Log Service is not active, you must activate it first and then send another Kafka message.
- Click the Monitoring tab and check for invocations. If there are invocations and zero errors, the sink successfully received the transformed content.

Use Function Compute as a transform