Flink AI capabilities
Realtime Compute for Apache Flink integrates AI capabilities for a full range of scenarios, from SQL function calls and Python data processing to building streaming agents. You can perform large model inference, vectorization, multimodal analysis, and agent orchestration all within a single Flink job.
Built-in model service
The platform includes built-in large models from the Qwen series, which are ready to use out of the box with no API key required.
Capability | Built-in model |
Text generation, visual understanding | qwen3.6-plus, qwen3.6-flash, qwen3.5-plus, qwen3.5-flash |
Text vectorization | text-embedding-v4 |
Multimodal vectorization | qwen3-vl-embedding |
SQL AI functions
Directly call large models in Flink SQL without writing application code.
Text: Complete sentiment analysis, entity extraction, tagging and classification, translation, summary generation, and more with a single SQL statement.
Vector: Perform text vectorization and vector search. Vectorization results can be cached to reduce the overhead of repeated calls.
Multimodal: Classify images, extract information, and generate content descriptions. The model takes an image URL as input and outputs structured results.
Use cases: Real-time data cleansing and annotation, streaming text analysis, online feature engineering, and real-time image classification and information extraction.
Python DataFrame API
A high-level AI data processing interface for Python developers. Use chained calls to connect data ingestion, preprocessing, AI inference, and output.
Core capabilities:
Multimodal support: Accepts both text and images as input.
Built-in operators: Includes built-in operators for large model inference and vectorization that can be integrated with a single line of code.
Custom preprocessing: Supports custom UDFs for data preprocessing.
Use cases: E-commerce product image analysis and description generation, multimodal content understanding, and intelligent batch data processing.
Flink Agents
Build event-driven, streaming AI agents using the open-source Apache Flink Agents framework.
Core capabilities:
Distributed coordination: The framework handles event routing, partitioning, state consistency, and fault recovery.
Stateful memory: Agent memory is automatically persisted and restored at each checkpoint.
Stream processing: Agents continuously respond to incoming events with high throughput and low latency.
Dual-mode agents: Use Workflow agents for scenarios with well-defined processes and ReAct agents for scenarios that require flexible decision-making.
Supports both Python and Java development. Use cases: Real-time user review analysis, intelligent customer service, and automated business decision-making.
Agent Skills (natural language O&M)
Use Alibaba Cloud Flink Agent Skills to manage Flink resources in natural language in AI agent clients such as Claude Code and Cursor.
Instance management: Create instances, query status, and manage namespaces.
Workspace O&M: Create and validate SQL drafts, deploy and start/stop jobs, manage Session Clusters, and diagnose jobs.
No manual OpenAPI calls are required. Operations can be handled through conversation.
AI assistant
A conversational AI assistant built into the Realtime Compute console that lets you manage the entire lifecycle of your Flink jobs through natural language interaction.
Intelligent code generation: Automatically generates Flink SQL code from your business requirements. It supports building complete logic, including source table declarations, dimension table joins, and window calculations.
Troubleshooting: Analyzes job runtime exceptions and error logs to identify root causes and recommend fixes.
Performance tuning: Identifies bottlenecks such as data skew and oversized states based on Flink best practices, and suggests parameter tuning and SQL rewriting.
Third-party model services
Connect to your self-managed or third-party models through inference services like Triton Inference Server or PAI-EAS to perform online inference in your Flink jobs. This approach is ideal for scenarios when you have existing model assets and need to manage the inference services yourself.