Hive built-in functions
Realtime Compute for Apache Flink supports Hive built-in functions, letting you reuse Hive function semantics in Flink SQL jobs without rewriting business logic. This topic describes how to call Hive built-in functions in both Hive SQL drafts and Flink SQL drafts.
Limits
-
Only jobs running on Ververica Runtime (VVR) 8.0.11 or later support Hive built-in functions.
-
Hive user-defined functions (UDFs) are not supported.
Prerequisites
Before you begin, ensure that you have:
-
A Flink job running on VVR 8.0.11 or later
-
Access to the Realtime Compute for Apache Flink console
-
A Hive metastore to determine the correct Hive connector version
How it works
Hive built-in functions are mapped to Scalar functions, Table functions, or Aggregate functions in Flink's type system, depending on the resolution and execution phases. Supported functions vary by Flink version.
There are two ways to call Hive built-in functions, depending on your draft type:
| Draft type | How functions are loaded | Extra setup needed |
|---|---|---|
| Hive SQL draft | Via Hive catalog — no additional configuration | None |
| Flink SQL draft | Via Hive connector and HiveModule | Upload JAR, load HiveModule |
Call functions in a Hive SQL draft
In a Hive SQL draft, call any Hive built-in function directly through Hive catalogs. No extra configuration is required.
When a Hive built-in function shares a name with a Flink system function, VVR executes the Hive built-in function by default. This ensures consistent function behavior between your Flink job and a native Hive job.
For a step-by-step walkthrough, see Quick start with Hive dialect jobs.
Call functions in a Flink SQL draft
In a Flink SQL draft, invoke Hive built-in functions via the Hive connector and HiveModule. The setup has three stages: configure the dependency, load HiveModule, and (optionally) adjust the module resolution order.
Step 1: Configure the dependency
1. Get the Hive connector JAR.
Choose the JAR that matches both your VVR version and your Hive metastore version. Use the table below to find the correct version and download the JAR from Maven Central Repository.
Flink supports Hive versions 2.0.0–2.3.9 and 3.1.0–3.1.3. For VVR-to-Apache Flink version mappings, see Engine updates.
| Hive metastore version | Hive connector version | Download |
|---|---|---|
| 2.0.x, 2.1.x, or 2.2.x | 2.2.0 | — |
| 2.3.x | 2.3.9 | flink-sql-connector-hive-2.3.9_2.12/1.17.2 |
| 3.1.x | 3.1.3 | flink-sql-connector-hive-3.1.3_2.12/1.17.2 |
For example, if your job uses VVR 8.0.11 with Hive 2.3.x, download flink-sql-connector-hive-2.3.9_2.12-1.17.2.jar.
2. Upload the JAR and attach it to your draft.
-
Log on to the Realtime Compute for Apache Flink console.
-
In the left-side navigation pane, choose Artifacts.
-
Click Upload Artifact and upload the Hive connector JAR.
-
In the left-side navigation pane, choose Development > ETL.
-
Click New to create a draft. In the New Draft dialog box, select an engine version compatible with the Apache Flink version of the uploaded JAR.
-
On the right side of the SQL editor, click the Configurations tab. In the Additional Dependencies field, select the JAR you uploaded.
Step 2: Load HiveModule
Run the following statement to load HiveModule for your Hive version:
LOAD MODULE hive WITH('hive-version' = '2.3.9');The hive-version value must match the version in the JAR filename. For example, when using flink-sql-connector-hive-2.3.9_2.12-1.17.2.jar, set hive-version='2.3.9'. Supported features of HiveModule vary by Apache Flink version — see the Hive Module of Flink 1.17 reference or the corresponding Hive Module of Flink 1.20 reference for details on supported syntax and limitations.
Verify the HiveModule status.
Copy the following statements into the SQL editor, select them, right-click, and choose Run:
LOAD MODULE hive WITH('hive-version' = '2.3.9');
SHOW MODULES;Verify that the output lists both core and hive, confirming the module is loaded and active:
Step 3: Adjust the module resolution order (optional)
By default, Flink resolves functions in this order: core, then hive. When a Hive function shares a name with a Flink system function, Flink resolves to the Flink system function in the core module.
To make Flink resolve Hive functions first, change the resolution order:
USE MODULES `sql-gateway-module`, `hive`, `core`;sql-gateway-module is VVR's internal compilation module. Its position cannot be moved — only the order of hive and core can be changed.
After running this statement, Flink resolves functions to HiveModule first:
Check available functions
To list all Hive functions available in your current Flink session, run:
LOAD MODULE hive WITH('hive-version' = '2.3.9');
SHOW FUNCTIONS;
Supported functions vary by Flink version. For version-specific lists, see Hive Functions of Flink 1.17 or Hive Functions of Flink 1.20. For the complete catalog of Hive native functions, see LanguageManual UDF. Note that Flink supports only a subset of core functions.