Message Queue for Apache Kafka

Procedure

1. Log on to the EventBridge console. In the left-side navigation pane, click Event Streams.

2. In the top navigation bar, select a region and click Create Event Stream.

3. On the Create Event Stream panel, enter a Task Name and Description, configure the following parameters, and then click Save.

   • Task Creation

     1. In the Source step of the wizard, set Data Provider to Message Queue for Apache Kafka. Configure the following parameters and click Next Step.

     Parameter	Description	Example
     Region	Select the region of the source Message Queue for Apache Kafka instance.	China (Beijing)
     ApsaraMQ for Kafka Instance	Select the source instance that produces Message Queue for Apache Kafka messages.	MQ_INST_115964845466****_ByBeUp3p
     Topic	Select the topic for producing Message Queue for Apache Kafka messages.	topic
     Group ID	Select the consumer group for the source instance. Note: Use a dedicated consumer group for the event source. Do not share a consumer group with other services to avoid disrupting existing message delivery.	GID_http_1
     Consumer Offset	Select the offset from which message consumption begins.	Latest Offset
     Network Configuration	Select the network type over which to route messages.	Basic Network
     VPC	Select the VPC ID. This parameter is required only if Network Configuration is set to Internet.	vpc-bp17fapfdj0dwzjkd****
     vSwitch	Select the vSwitch ID. This parameter is required only if Network Configuration is set to Internet.	vsw-bp1gbjhj53hdjdkg****
     Security Group	Select a security group. This parameter is required only if Network Configuration is set to Internet.	alikafka_pre-cn-7mz2****
     Batch Push	Batch push allows you to aggregate multiple events. A push occurs when either the Messages or Interval (Unit: Seconds) threshold is reached. For example, if you set Messages to 100 and Interval to 15s, a push is triggered after only 10s if 100 messages are collected within that time.	Enable
     Messages	The maximum number of messages to include in a batch for a function invocation. A request is sent only when the number of queued messages reaches this value. Valid values: 1 to 10,000.	100
     Interval (Unit: Seconds)	The maximum time to wait, in seconds, before the function is invoked. The system aggregates messages and sends them to Function Compute at this interval. Valid values: 0 to 15. A value of 0 means messages are delivered immediately without waiting.	3

     2. In the Filtering, Transformation, and Sink steps of the wizard, configure event filtering rules, transformation rules, and the event sink. For more information about event transformation, see Use Function Compute to clean message data.

   • Task Properties

     Configure the retry policy and dead-letter queue for the event stream. For more information, see Retry policies and dead-letter queues.

4. Return to the Event Streams page. Find the event stream you created, and in the Actions column, click Enable.

   After you enable the event stream, it may take 30 to 60 seconds to start. You can monitor the startup progress in the Status column on the Event Streams page.

Sample event

{
  "specversion": "1.0",
  "id": "8e215af8-ca18-4249-8645-f96c1026****",
  "source": "acs:alikafka",
  "type": "alikafka:Topic:Message",
  "subject": "acs:alikafka:alikafka_pre-cn-i7m2msb9****:topic:****",
  "datacontenttype": "application/json; charset=utf-8",
  "time": "2022-06-23T02:49:51.589Z",
  "aliyunaccountid": "182572506381****",
  "data": {
    "topic": "****",
    "partition": 7,
    "offset": 25,
    "timestamp": 1655952591589,
    "headers": {
      "headers": [],
      "isReadOnly": false
    },
    "key": "keytest",
    "value": "hello kafka msg"
  }
}

For descriptions of the parameters in the CloudEvents specification, see Event overview.

The following table describes the parameters in the data field.