This topic describes how to use the Parquet format and its type mappings.
Limits
The Parquet format supports reading and writing Parquet data.
Only the Object Storage Service (OSS) connector supports reading data in Parquet format.
Example
The following example shows how to create a table using OSS and the Parquet format.
CREATE TEMPORARY TABLE user_behavior (
user_id BIGINT,
item_id BIGINT,
category_id BIGINT,
behavior STRING,
ts TIMESTAMP(3),
dt STRING
) PARTITIONED BY (dt) WITH (
'connector' = 'filesystem',
'path' = 'oss://<bucket>/path',
'format' = 'parquet'
)Options
Parameter | Required | Default value | Type | Description |
format | Yes | none | String | The format to use. To use the Parquet format, set this parameter to `parquet`. |
parquet.utc-timezone | No | false | Boolean | You can convert between epoch time and LocalDateTime using the UTC time zone or the local time zone. Valid values:
|
The Parquet format also supports the configurations for ParquetOutputFormat. For example, you can set parquet.compression=GZIP to enable GZIP compression.
Type mapping
The Parquet format type mapping is compatible with Apache Hive, but differs from Apache Spark in the following ways:
Timestamp: Maps the timestamp type to `int96`, regardless of precision.
Decimal: Maps the decimal type to a fixed-length byte array, based on the precision.
The following table describes the data type mappings between Flink and Parquet.
Flink SQL type | Parquet type | Parquet logical type |
CHAR / VARCHAR / STRING | BINARY | UTF8 |
BOOLEAN | BOOLEAN | - |
BINARY / VARBINARY | BINARY | - |
DECIMAL | FIXED_LEN_BYTE_ARRAY | DECIMAL |
TINYINT | INT32 | INT_8 |
SMALLINT | INT32 | INT_16 |
INT | INT32 | - |
BIGINT | INT64 | - |
FLOAT | FLOAT | - |
DOUBLE | DOUBLE | - |
DATE | INT32 | DATE |
TIME | INT32 | TIME_MILLIS |
TIMESTAMP | INT96 | - |
ARRAY | - | LIST |
MAP | - | MAP |
ROW | - | STRUCT |