Python multimodal function library

Load the Module

LOAD MODULE python;
USE MODULES python, core;

After execution, the functions in Python Module participate in function resolution of the current session.

Call functions

After loading the module, you can call image and video functions directly in SQL without executing CREATE FUNCTION.

The following example decodes image bytes into an image structure and resizes it to 224×224:

LOAD MODULE python;
USE MODULES python, core;

SELECT
    id,
    IMAGE_RESIZE(IMAGE_DECODE(image_bytes, 'raise', 'RGB'), 224, 224) AS image_224
FROM raw_images;

Parameter conventions

SQL functions accept only positional arguments, in the order defined by the signature. Optional parameters are shown in [ ]. For example, the signature of IMAGE_DECODE is:

IMAGE_DECODE(image_bytes [, on_error [, mode [, pixel_limit]]])

Follow these rules when calling the function:

• If you only need default values, omit the trailing optional parameters.

• To set a later optional parameter, also specify all preceding optional parameters.

• To leave a preceding optional parameter unset, use NULL or the default value as a placeholder. NULL is passed as the argument value; its meaning depends on the parameter description and is not necessarily equivalent to the default.

• Except for the data input column, configuration parameters (format, size, threshold, model name, language list, and so on) must be constants; table columns are not allowed.

-- Use the default on_error, mode, and pixel_limit
SELECT IMAGE_DECODE(img_bytes) FROM raw_images;

-- Set on_error and mode; use the default pixel_limit
SELECT IMAGE_DECODE(img_bytes, 'null', 'RGB') FROM raw_images;

-- Set on_error and pixel_limit only; use NULL as a placeholder for mode
SELECT IMAGE_DECODE(img_bytes, 'null', null, 67108864)
FROM raw_images;

-- Configuration parameters must be constants
SELECT IMAGE_RESIZE(img, 512, 512, 'lanczos') FROM images;

Extract Image Metadata

LOAD MODULE python;
USE MODULES python, core;

SELECT
  id,
  IMAGE_METADATA(image_bytes) AS meta
FROM raw_images;         

Decodes raw image bytes and extracts metadata such as width, height, number of channels, and format. Suitable for data cleansing and quality analysis.

Image Object Detection

LOAD MODULE python;
USE MODULES python, core;

SELECT
  id,
  IMAGE_DETECT_OBJECTS(IMAGE_DECODE(image_bytes, 'raise', 'RGB'), 'yolov8n', 0.1) AS objects
FROM raw_images;          

Performs object detection on decoded images using the yolov8n model with a confidence threshold of 0.1. Only detection results above the threshold are returned. Suitable for product recognition and content moderation.

Extract Video Metadata

LOAD MODULE python;
USE MODULES python, core;

SELECT
  id,
  VIDEO_METADATA(video_uri) AS meta
FROM source_video;         

Extracts metadata such as duration, resolution, frame rate, and codec from a video file URI.

Video Frame Extraction

LOAD MODULE python;
USE MODULES python, core;

SELECT
  id,
  VIDEO_EXTRACT_FRAMES(video_uri, 'sample', 1000, 8) AS sampled_frames
FROM source_video;

Extracts frames from a video at a sampling interval of 1000 milliseconds, returning up to 8 frames for subsequent image analysis or feature extraction.