AI_EXTRACT
Updated at:
AI_EXTRACT extracts structured information from raw text using a large language model (LLM).
Limitations
-
Supported only by the real-time computing engine Ververica Runtime (VVR) 11.4 or later.
-
To use the Flink AI service (built-in models), VVR 11.7 or later is required, and the Flink AI service must be activated. For details, see Flink AI service (built-in models).
-
Throughput is limited by the model service platform. When traffic reaches the platform's access limit, backpressure occurs on the Flink operators used in
AI_EXTRACTstatements. Severe rate limiting can trigger timeouts and cause the Flink job to restart.
Syntax
AI_EXTRACT(
MODEL => MODEL <MODEL NAME>,
INPUT => <INPUT COLUMN NAME>,
EXTRACT_SCHEMA => <EXTRACT SCHEMA>
)
Parameters
| Parameter | Data type | Description |
|---|---|---|
MODEL <MODEL NAME> |
MODEL | The name of the registered model service. For more information, see Model Settings. Note: The output type of this model must be VARIANT. |
<INPUT COLUMN NAME> |
STRING | The column containing the raw text to extract information from. |
<EXTRACT_SCHEMA> |
STRING | A JSON string that defines the fields to extract and their data types. Must be a constant. |
Return values
| Column | Data type | Description |
|---|---|---|
extracted_json |
STRING | The extracted fields as a JSON string. |
Examples
Test data
| id | description |
|---|---|
| 1 | Xiao Ming is 18 years old and lives in Hangzhou. His phone number is ******. |
SQL statements
The following example references a Flink built-in model and uses AI_EXTRACT to extract user information.
CREATE TEMPORARY MODEL general_model
INPUT (`input` STRING)
OUTPUT (`content` VARIANT)
WITH (
'provider' = 'openai-compat',
'task' = 'chat/completions',
'model' = 'qwen3.6-flash'
);
CREATE TEMPORARY VIEW infos(id, description)
AS VALUES (1, 'Xiao Ming is 18 years old and lives in Hangzhou. His phone number is ******.');
-- Positional argument style
SELECT id, extracted_json
FROM infos,
LATERAL TABLE(
AI_EXTRACT(
MODEL general_model,
description,
'{"name":"string","phone":"string","address":"string","age":"int"}'));
-- Named argument style
SELECT id, extracted_json
FROM infos,
LATERAL TABLE(
AI_EXTRACT(
MODEL => MODEL general_model,
INPUT => description,
EXTRACT_SCHEMA => '{"name":"string","phone":"string","address":"string","age":"int"}'));
Output
| id | extracted_json |
|---|---|
| 1 | {"address":"Hangzhou","age":18,"name":"Xiao Ming","phone":"******"} |
Is this page helpful?