EventHouse overview

Core Components

EventHouse consists of three core components that work both independently and collaboratively:

Core Value

Zero-ETL (Seamless Data Integration)

Directly map external data sources such as RDS and OSS and run federated queries without moving data into EventHouse. This reduces data latency and storage costs.

Unified Governance

The Open Catalog brings standardized metadata management and data lineage tracking to schema-less data in message queues, often called "dark data", helping break down data silos.

Agentic Analytics

EventHouse natively integrates the Model Context Protocol (MCP), allowing AI Agents to understand event data structures and perform analysis using natural language queries.

Unified Metadata Management

• Multi-source Mapping: Automatically discovers and registers metadata from data sources such as Kafka, RocketMQ, and RDS.

• Schema Evolution: Automatically infers and manages schema versions for event data. When upstream fields change, compatibility version control ensures downstream analysis tasks continue without interruption.

• Data Lineage Tracking: Tracks the full lifecycle of events from production (Producer) and storage (EventStore) to analysis (Analysis), supporting troubleshooting and impact assessment.

Open Ecosystem Compatibility

The Open Catalog supports open table formats such as Iceberg, Hudi, and Delta Lake. Your data is not vendor-locked, so you can freely choose your preferred compute engines.

Permissions and Security

Fine-grained access control (ACL) is available at three levels: database, table, and column.

Application Scenario: Unified Data View

In an e-commerce scenario, order data might be scattered across RocketMQ for real-time streams and MySQL for persistence. You can create a unified View in the Catalog to logically associate real-time order streams in MQ with user information tables in the database. You can then query this View directly without knowing the physical storage location of the underlying data.

Intelligent Query Engine

• Multimodal Query: Supports three query modes: SQL for structured queries, NoSQL for document queries, and External for external data source queries.

• Stream-Batch Integration: The same SQL syntax can query both historical archived data (Batch) and real-time incoming event streams (Streaming).

• Materialized View: Pre-computed materialized views cache high-frequency query results for millisecond-level response times.

Federated Query

• Cross-Source Federated Analysis: Directly associate EventHouse internal tables with external data sources such as OSS log files and RDS dimension tables by using SQL JOIN, without data migration.

• Predicate Pushdown: Filter conditions are pushed down to the source for execution, so only necessary data is pulled, improving query efficiency.

Real-time Anomaly Detection

• Built-in time window functions (Tumble, Hop, and Session) calculate metrics such as transaction success rates and latency distributions in real time.

• Combine this with a rules engine to automatically trigger an alert event when analysis results exceed a threshold, for example, "more than 100 failed orders in 1 minute".

Technical Advantages

DataAgent

Luma includes a built-in DataAgent that autonomously runs a perceive-plan-act loop:

1. Perception: An abnormal drop in transaction volume was detected.

2. Plan: Decides to query associated payment gateway logs and database connection pool status.

3. Act: Automatically generates SQL for association analysis and outputs a root cause report.

Application Scenario: E-commerce Risk Control

1. An operations team member asks, "Have there been any abnormal brushing behaviors in the last half hour?"

2. The Luma Agent retrieves Catalog information using MCP and identifies the Transaction_Table and User_Behavior_Log.

3. The Agent automatically generates association SQL (including time window, IP aggregation, and device fingerprint analysis) and executes it in the EventHouse analysis engine.

4. It returns a list of suspected brushing UserIDs and generates a risk report using the knowledge base.