Elasticsearch connection type,create Elasticsearch connection,connection test,Catalog mount-EventBridge(EventBridge)-阿里云帮助中心

Overview

The Elasticsearch connection type establishes connectivity between EventBridge and Elasticsearch clusters. After you create an Elasticsearch connection, you can mount it in the EventHouse Data Catalog to map Elasticsearch indexes as queryable event tables.

Supported Elasticsearch instances:

Alibaba Cloud Elasticsearch (version 6.x and later, 7.x or 8.x recommended)
Self-managed Elasticsearch clusters

Connection parameters

Parameter	Field name	Type	Required	Description
Host address	HostName	string	Yes	The Elasticsearch instance address. For Alibaba Cloud Elasticsearch, enter the instance domain name (such as `es-cn-xxx.elasticsearch.aliyuncs.com`). For self-managed clusters, enter the IP address.
Port number	Port	string	Yes	The Elasticsearch HTTP port. Default value: `9200`. Specify the HTTP port, not the transport port (9300).
Username	User	string	No	The Elasticsearch username. Required only when security authentication is enabled. The default username for Alibaba Cloud Elasticsearch is `elastic`.
Password	Password	string	No	The Elasticsearch password. Required only when security authentication is enabled.

Alibaba Cloud Elasticsearch has security authentication enabled by default. We recommend that you provide the username and password. If X-Pack Security is not enabled on a self-managed cluster, authentication credentials are not required.

Parameter examples

Security authentication enabled (recommended):

{
  "HostName": "es-cn-xxx.elasticsearch.aliyuncs.com",
  "Port": "9200",
  "User": "elastic",
  "Password": "your_password"
}

Security authentication not enabled:

{
  "HostName": "192.168.1.100",
  "Port": "9200"
}

Create an Elasticsearch connection

Create a connection in the console

Log on to the EventBridge console.
In the left-side navigation pane, choose Integration Center > Connection Management.
In the top navigation bar, select a region and click Create Connection.
Select Elasticsearch as the connection type and configure the connection parameters.
Click Test Connection to verify connectivity.
After the test is passed, click OK.

Create a connection by calling an API operation

Call the CreateConnection API operation to create an Elasticsearch connection:

{
  "ConnectionName": "my-es-conn",
  "Type": "Elasticsearch",
  "Parameters": {
    "HostName": "es-cn-xxx.elasticsearch.aliyuncs.com",
    "Port": "9200",
    "User": "elastic",
    "Password": "your_password"
  }
}

Network configuration

Network type	Description	Scenario
Internet	Connect over the Internet. No additional configuration is required.	The Elasticsearch instance has Internet access enabled. For Alibaba Cloud Elasticsearch, enable the public network access feature.
VPC	Connect over a VPC.	The Elasticsearch instance is deployed in a VPC.

For a VPC connection, the following additional parameters are required:

VPC ID: The VPC where the Elasticsearch instance is located.
vSwitch ID: A vSwitch in the VPC.
Security group ID: The inbound rules of the security group must allow access to the Elasticsearch HTTP port (9200).

EventBridge automatically establishes network connectivity with your VPC. No additional network configuration is required.

Usage scenarios

Catalog mount (supported)

Mount an Elasticsearch connection to an EventHouse Catalog to query Elasticsearch indexes directly with SQL:

-- Query a log index in Elasticsearch
SELECT * FROM my_catalog.log_namespace.app_logs LIMIT 10;

-- Cross-source JOIN: Elasticsearch logs and EventHouse events
SELECT l.trace_id, l.message, e.event_type
FROM my_catalog.log_namespace.app_logs l
JOIN my_catalog.event_namespace.api_events e
  ON l.trace_id = e.trace_id
WHERE l.level = 'ERROR';

For more information, see Data catalog.

Field type mapping

During mounting, Elasticsearch field mappings are automatically converted to Catalog column types:

Elasticsearch type	Catalog column type
text / keyword	text
long / integer / short / byte	bigint
double / float / half_float	double
date	timestamp
boolean	text
object / nested	json

Permission requirements

We recommend that you create a dedicated read-only account for EventBridge. Example:

POST /_security/role/eb_readonly
{
  "indices": [
    {
      "names": ["app_logs*", "metrics*"],
      "privileges": ["read", "view_index_metadata"]
    }
  ]
}

POST /_security/user/eb_reader
{
  "password": "your_password",
  "roles": ["eb_readonly"]
}

EventBridge only requires read permissions and does not modify data in Elasticsearch.

FAQ

Q: The connection test failed with "Connection refused".

A: Check the following:

Verify that the Elasticsearch cluster is running properly (cluster health status is green or yellow).
Verify that the host address and port are correct. Specify the HTTP port (default 9200), not the transport port (9300).
Verify that the security group or firewall allows access to port 9200.
If you use a VPC connection, verify that the VPC and vSwitch are configured correctly.

Q: Authentication failed on an instance with security authentication enabled.

A: Check the following:

Verify that the username and password are correct. The default username for Alibaba Cloud Elasticsearch is elastic.
Verify that the user has the read and view_index_metadata permissions.

Q: Can I modify the parameters after creating a connection?

A: You can modify the host address, port, username, and password. The connection name and type cannot be modified after creation.