This topic describes the Frontend (FE), Backend (BE), and system parameters for StarRocks clusters on E-MapReduce (EMR), and how to modify them.
Parameters fall into two categories:
Dynamic parameters: Take effect immediately without restarting the service. All FE dynamic parameters can be modified online.
Static parameters: Take effect only after a service restart.
View current configuration
To view the current FE configuration, run the following command in a MySQL client connected to the FE:
ADMIN SHOW FRONTEND CONFIG;The output includes the parameter name, current value, and whether the parameter is mutable (can be modified online).
Modify FE dynamic parameters
Run the following command in a MySQL client connected to the FE:
ADMIN SET FRONTEND CONFIG ("key" = "value");FE dynamic parameters
Logging
Parameter | Default value | Description |
|
| Threshold for slow query detection. Queries that take longer than this value are recorded as slow queries. Unit: milliseconds. |
Metadata and cluster management
Parameter | Default value | Description |
|
| Timeout for acquiring a catalog lock. Unit: milliseconds. |
|
| Maximum number of image log entries written before a new log file is created. |
|
| Whether to ignore unknown log IDs. Set to |
|
| Whether to ignore the metadata gap. Set to |
|
| Maximum time for a BE to rejoin the cluster after disconnecting from the FE. If the BE does not reconnect within this period, it is considered permanently down. Unit: seconds. |
|
| Whether to automatically delete a BE node after decommissioning. Set to |
Query engine
Parameter | Default value | Description |
|
| Maximum number of IN predicate elements allowed in a query. |
|
| Maximum nesting depth for expressions in a query. |
|
| Maximum number of elements in the IN predicate of a DELETE statement. |
|
| Maximum length of a single row. |
|
| Whether to disable the logical cluster feature. Set to |
|
| Whether to enable materialized views. Set to |
|
| Whether to support the DECIMAL V3 data type. Set to |
|
| Whether to enforce the SQL query blacklist. When enabled, queries matching blacklist patterns cannot be executed. Set to |
|
| Interval at which the FE checks for new data and automatically creates partitions. Unit: seconds. |
|
| Whether to enable dynamic partitioning. Set to |
|
| Maximum number of partitions that can be created in a single batch. |
|
| Maximum number of query retries on the FE. |
|
| Maximum timeout for creating a table. Unit: seconds. |
|
| Maximum number of concurrent Rollup jobs per table. |
|
| Maximum number of times the optimizer rewrites a scalar operator. |
|
| Interval at which statistics tables are created. Unit: seconds. |
|
| Interval at which statistics are collected. Unit: seconds. |
|
| Default interval at which statistics are updated. Unit: seconds. |
|
| Number of rows sampled during statistics collection. |
|
| Whether to enable statistics collection. Set to |
|
| Whether the optimizer preferentially selects tablet replicas on BEs that share the same IP address as the FE. Set to |
|
| Maximum recursion depth for partition pruning. |
Loading and unloading
Parameter | Default value | Description |
|
| Maximum loading lag tolerated for a BE replica before data is cloned from other replicas. Unit: seconds. |
|
| Maximum number of pending jobs across all job types, including table creation, loading, and schema change jobs. |
|
| Maximum number of concurrent load jobs per database. |
|
| Maximum timeout for any load job. Unit: seconds. |
|
| Minimum timeout for any load job. Unit: seconds. |
|
| Number of concurrent loading instances on a single BE. |
|
| Whether to disable loading data from Hadoop. Set to |
|
| Whether to reject new load jobs when the cluster encounters an error. Set to |
|
| Interval at which the FE updates data quota usage for each database. Unit: seconds. |
|
| Maximum retention period for historical jobs. Unit: seconds. |
|
| Maximum number of load jobs retained within the period defined by |
|
| Retention period for load job labels. Unit: seconds. |
|
| Maximum number of Routine Load jobs. |
|
| Maximum number of concurrent tasks within a single Routine Load job. |
|
| Maximum number of concurrent Routine Load tasks on a single BE. Must be less than or equal to |
|
| Maximum data volume loaded by a single Routine Load task. Unit: bytes. |
|
| Maximum time a Routine Load task spends consuming data. Unit: seconds. |
|
| Timeout for a Routine Load task. Unit: seconds. |
|
| Maximum number of failed BEs allowed before Routine Load jobs can no longer be automatically recovered. |
|
| Interval at which the FE attempts to automatically recover Routine Load jobs. Unit: minutes. |
|
| Timeout for a Spark Load job. Unit: seconds. |
|
| Root directory of the Spark client. |
|
| Default timeout for a Stream Load job. Unit: seconds. |
|
| Maximum timeout for a Stream Load job. Unit: seconds. |
|
| Timeout for an INSERT INTO load operation. Unit: seconds. |
|
| Timeout for a Broker Load job. Unit: seconds. |
|
| Minimum data volume processed by a single Broker Load instance. Default: 64 MB. |
|
| Maximum number of concurrent Broker Load instances. |
|
| Maximum data volume exported from a single BE by a single unload task. Default: 256 MB. |
|
| Maximum number of concurrent data unload jobs. |
|
| Timeout for a data unload job. Default: 2 hours. Unit: seconds. |
Storage
Parameter | Default value | Description |
|
| Whether the FE checks for available storage space on BEs when creating a table. Set to |
|
| Upper disk usage threshold for a BE. When disk usage exceeds 75%, the FE stops sending table creation and clone jobs to that BE until usage drops below the threshold. |
|
| Maximum allowed storage usage percentage in a BE's storage directory. |
|
| Minimum required free space in a BE's storage directory. Default: 2 GB. |
|
| Free space threshold in a BE's storage directory below which load and restore jobs are rejected. Default: 1 GB. |
|
| Storage usage threshold above which load and restore jobs are rejected. |
|
| Retention period for metadata in the recycle bin after a table or database is deleted. Data cannot be restored after this period expires. Default: 1 day. Unit: seconds. |
|
| Timeout for a schema change job. Default: 1 day. Unit: seconds. |
|
| Safe disk usage threshold for the |
|
| Load imbalance threshold. For the |
|
| Whether to disable tablet balancing. Set to |
|
| Maximum number of tablets being scheduled concurrently. If this threshold is exceeded, tablet load balancing is skipped. |
|
| Maximum number of tablets being rebalanced concurrently. If this threshold is exceeded, rebalancing is skipped. |
|
| Whether to disable replica balancing for colocate tables. Set to |
|
| Whether to substitute an empty tablet replica when a replica is lost or corrupted. Set to |
|
| Minimum timeout for a tablet clone task. Default: 3 minutes. Unit: seconds. |
|
| Maximum timeout for a tablet clone task. Default: 2 hours. Unit: seconds. |
|
| Timeout for creating a tablet. Unit: seconds. |
|
| Timeout for deleting a tablet. Unit: seconds. |
|
| Interval between tablet replica repair attempts. Unit: seconds. |
|
| Hour at which the FE starts replica consistency checks. Default: 23:00. |
|
| Hour at which the FE stops replica consistency checks. Default: 04:00. |
|
| Timeout for a replica consistency check. Unit: seconds. |
Others
Parameter | Default value | Description |
|
| Whether to enable the plug-in feature. Plug-ins can only be installed or uninstalled on the master FE node. Set to |
|
| Maximum number of small files that can be stored. |
|
| Maximum size of a single small file. Default: 1 MB. |
|
| Timeout for a backup job. Default: 1 day. Unit: milliseconds. |
|
| Maximum number of pending reports in the queue. Reports include disk status, task status, and tablet information from BEs. |
FE static parameters
Static parameters take effect only after an FE restart.
Logging
Parameter | Default value | Description |
|
| Maximum size of a log file before a new one is created. Default: 1 GB. |
|
| Directory for system log files. |
|
| Minimum severity level for system log entries. Valid values: |
| _(empty)_ | Modules for which verbose system logs are generated. For example, setting this to |
|
| Rotation interval for system log files. |
|
| Retention period for system log files. Files older than this value are deleted. |
|
| Maximum number of system log files retained per rotation interval. |
|
| Directory for audit log files. |
|
| Maximum number of audit log files retained. |
|
| Modules for which audit logs are generated. |
|
| Rotation interval for audit log files. Valid values: |
|
| Retention period for audit log files. Files older than this value are deleted. |
|
| Directory for dump log files. |
|
| Modules for which dump logs are generated. |
|
| Rotation interval for dump log files. Files rotated daily use a |
|
| Maximum number of dump log files retained per rotation interval. |
|
| Retention period for dump log files. Files older than this value are deleted. |
Server
Parameter | Default value | Description |
|
| IP address of the FE node. |
| _(empty)_ | Preferred CIDR block for servers with multiple network interfaces. For example: |
|
| HTTP server port. |
|
| Backlog queue length for the HTTP server. |
|
| Cluster name displayed in the web UI. |
|
| Thrift server port on the FE. |
|
| Backlog queue length for the Thrift server. |
|
| Service model for the Thrift server. Valid values: |
|
| Maximum number of worker threads for the Thrift server. |
|
| Timeout for client requests. The default value |
|
| Maximum idle wait time for bRPC connections. Default: 10 seconds. Unit: milliseconds. |
|
| MySQL server port on the FE. |
|
| Whether to enable asynchronous I/O for the MySQL service. Set to |
|
| Maximum number of threads for handling MySQL I/O events. |
|
| Backlog queue length for the MySQL server. |
|
| Maximum number of threads for processing MySQL tasks. |
|
| Maximum number of threads for the connection scheduler. |
|
| Maximum number of concurrent connections to the FE across all users. |
|
| Whether to verify that the Java runtime version is compatible with the compiled Java version. Set to |
Metadata and cluster management
Parameter | Default value | Description |
|
| Directory for storing FE metadata. |
|
| Number of threads used by Heartbeat Manager to process heartbeat tasks. |
|
| Size of the blocking queue for heartbeat tasks. |
|
| Whether to forcibly reset FE metadata. Exercise caution when enabling this parameter. |
|
| Port used for communication among master, follower, and observer FE nodes. |
|
| Type of edit log. Set this to |
|
| Heartbeat timeout for the BDB JE-based FE. Unit: seconds. |
|
| Lock timeout for the BDB JE-based FE. Unit: seconds. |
|
| Maximum allowed clock offset between the master FE and follower or observer FEs. Unit: milliseconds. |
|
| Maximum number of transactions that can be rolled back. |
|
| Maximum time the master FE waits for ACK messages from follower FEs. Unit: seconds. |
|
| Policy for flushing logs to disk on the master FE. |
|
| Policy for flushing logs to disk on follower FE replicas. |
|
| Maximum lag allowed for metadata on follower and observer FEs relative to the master FE. Unit: seconds. |
|
| Cluster ID. FE and BE nodes with the same cluster ID belong to the same cluster. The default value |
Query engine
Parameter | Default value | Description |
|
| Whether to disable colocate joins. Set to |
|
| Whether to enable user-defined functions (UDFs). Set to |
|
| Interval at which version publish tasks are dispatched. Unit: milliseconds. |
|
| Maximum number of rows cached for the statistics table. |
Loading and unloading
Parameter | Default value | Description |
|
| Thread pool size for asynchronous load jobs. |
|
| Polling interval for load job status checks. Unit: seconds. |
|
| Interval at which completed transactions are cleaned up. Unit: seconds. |
|
| Interval at which load job labels are cleaned up. Unit: seconds. |
|
| Version of Spark Dynamic Partition Pruning (DPP). |
| _(empty)_ | Root directory for Spark dependency packages. |
|
| Directory for Spark launcher log files. |
|
| Path to the YARN client binary. |
|
| Directory for YARN configuration files. |
|
| Polling interval for unload job scheduling. Unit: seconds. |
|
| Thread pool size for unload jobs. |
Storage
Parameter | Default value | Description |
|
| Interval for storage media migration from SSD to HDD. Default: 30 days. Unit: seconds. |
|
| Default storage media for tables and partitions when no media is specified. Valid values: |
|
| Maximum number of tablet jobs that can run concurrently in each storage directory on a BE. |
|
| Load balancing policy for tablets. Valid values: |
|
| Interval at which the FE retrieves tablet statistics from each BE. Default: 5 minutes. Unit: seconds. |
Others
Parameter | Default value | Description |
|
| Directory for plug-in installation packages. |
|
| Root directory for small files. |
|
| Maximum number of threads in the agent task thread pool. |
|
| Base DN for searching user authentication information in LDAP. |
|
| Administrator DN for searching user authentication information in LDAP. |
|
| Password for the LDAP administrator DN. |
|
| Hostname of the LDAP server. |
|
| Port of the LDAP server. |
|
| LDAP attribute used to identify users. |
|
| Directory for temporary files such as those generated during backup and restore. |
|
| Character set encoding. |
|
| Maximum number of concurrent threads for loading Hive metadata. |
|
| Interval at which cached Hive external table metadata is refreshed. Unit: seconds. |
|
| Time-to-live for cached Hive external table metadata. Default: 2 hours. Unit: seconds. |
|
| Connection timeout for the Hive Metastore. Default: 24 hours. Unit: seconds. |
|
| Interval at which the FE syncs Elasticsearch index metadata. Unit: seconds. |
|
| Whether to enable authentication checks. Set to |
|
| Whether to periodically collect cluster metrics. Set to |
BE parameters
All BE parameters are static parameters and take effect only after a BE restart.
Parameter | Default value | Description |
|
| Thrift server port on the BE, used to receive requests from FEs. |
|
| bRPC port for viewing bRPC network statistics. |
|
| Number of bRPC worker threads. The default value |
| _(empty)_ | Preferred CIDR block for servers with multiple network interfaces. For example: |
|
| Heartbeat service port on the BE, used to receive heartbeats from FEs. |
|
| Number of threads for the heartbeat service. |
|
| Number of threads for creating tablets. |
|
| Number of threads for deleting tablets. |
|
| Number of threads for processing NORMAL-priority load jobs. |
|
| Number of threads for processing HIGH-priority load jobs. |
|
| Number of threads for publishing tablet versions. |
|
| Number of threads for clearing completed transactions. |
|
| Number of threads for schema change operations. |
|
| Number of threads for tablet cloning. |
|
| Number of threads for storage media migration, such as migrating hot data from SSD to HDD. |
|
| Number of threads for checking tablet replica consistency. |
|
| Concurrency for creating snapshots. Lower this value to reduce resource usage. |
| Number of vCPUs | Concurrency for uploading snapshots. Lower this value to reduce bandwidth and I/O usage. |
|
| Interval at which the BE reports job status (table creation, deletion, loading, and schema change) to the FE. Unit: seconds. |
|
| Interval at which the BE reports disk status and data volumes to the FE. Unit: seconds. |
|
| Interval at which the BE reports tablet version information to the FE. Unit: seconds. |
|
| Timeout for a schema change job on the BE. Unit: seconds. |
|
| Directory for BE system logs. |
|
| Directory for UDF files. |
|
| Minimum severity level for BE system log entries. Valid values: |
|
| Maximum log file size before rotation. Default: 1 GB. |
|
| Maximum number of system log files retained. |
| _(empty)_ | Modules for which verbose system logs are generated. For example, setting this to |
|
| Verbosity level for logs prefixed with |
| _(empty)_ | Log flush policy. By default, logs are buffered in memory. |
|
| Number of threads started per CPU core. |
|
| Whether to compress row batches in bRPC calls between BEs during query processing. Set to |
|
| Whether to serialize row batches in bRPC calls between BEs during query processing. Set to |
|
| Interval at which the BE reports query profiles to the FE. Unit: seconds. |
|
| Number of threads in the storage engine's scan thread pool. |
|
| Maximum number of scan tasks queued in the storage engine. |
|
| Granularity at which the storage engine splits query scan ranges. |
|
| Maximum number of scan tasks the storage engine supports. |
|
| Maximum number of data rows returned by each scan thread per call. |
|
| Maximum number of scan keys that can be split. |
|
| Ratio threshold for the STRING type below which dictionary compression is applied. |
|
| Size threshold for a column below which dictionary compression is applied. |
|
| Maximum memory per thread for schema change jobs. Unit: GB. |
|
| Interval at which unused cached file descriptors are evicted. Unit: seconds. |
|
| Interval at which disk status is checked. Unit: seconds. |
|
| Interval at which expired rowsets are cleared. Unit: seconds. |
| _(empty)_ | Root directory for data storage. |
|
| Maximum number of tablets per shard within each storage directory. |
|
| Maximum retention period for pending data in the storage engine. Unit: seconds. |
|
| Retention period for loaded data in incremental cloning scenarios. Unit: seconds. |
|
| Maximum interval for disk garbage collection. Unit: seconds. |
|
| Minimum interval for disk garbage collection. Unit: seconds. |
|
| Retention period for snapshots before they are deleted. Default: 48 hours. Unit: seconds. |
|
| Retention period for files in the recycle bin before they are deleted. Default: 72 hours. Unit: seconds. |
|
| Maximum number of cached file descriptors. |
|
| Minimum number of file descriptors for the BE process. |
|
| Cache capacity for Bloom filter, Min, and Max statistical information. |
|
| Maximum capacity of the page cache. |
|
| Whether to disable the page cache. Set to |
|
| Thread polling interval for base compaction. Unit: seconds. |
|
| Number of threads for base compaction per disk. |
|
| Ratio of cumulative file size to base file size that triggers a base compaction. |
|
| Minimum interval since the last base compaction that triggers another base compaction. Unit: seconds. |
|
| Thread polling interval for cumulative compaction. Unit: seconds. |
|
| Minimum number of singleton files that triggers a cumulative compaction. |
|
| Maximum number of singleton files that triggers a cumulative compaction. |
|
| Number of threads for cumulative compaction per disk. |
|
| Minimum interval before a failed tablet compaction can be rescheduled. Unit: seconds. |
|
| Maximum concurrency for base compactions and cumulative compactions combined. The default value |
|
| HTTP server port on the BE. |
|
| Number of worker threads for the BE HTTP server. |
|
| Interval at which Counter statistics are collected. Unit: milliseconds. |
|
| Retention period for files generated by small-scale load jobs. Unit: hours. |
|
| Retention period for load error log files. Unit: hours. |
|
| Number of threads for Stream Load. |
|
| Maximum file size for a single Stream Load. Unit: MB. |
|
| RPC timeout for Stream Load. Unit: seconds. |
|
| Number of initial threads for query execution. Additional threads are created dynamically as needed. |
|
| Maximum number of queries that can be queued on a single BE node. |
|
| Whether to enable partition-based aggregation. Set to |
|
| Whether to enable token-based authentication checks. Set to |
|
| Maximum memory for all load threads on a BE node. Default: 100 GB. |
|
| Maximum percentage of total memory used by all load threads on a BE node. |
|
| Whether to enable synchronous writes for tablet metadata. Set to |
|
| Timeout for Thrift RPC calls. Unit: milliseconds. |
|
| Timeout for transaction commit RPC calls. Unit: milliseconds. |
|
| Thread pool size for Routine Load. Must be greater than or equal to |
|
| Minimum number of new rowsets since the last tablet meta checkpoint before triggering a new checkpoint. |
|
| Thread polling interval for generating tablet meta checkpoints. Unit: seconds. |
|
| Maximum body size for bRPC messages. Default: 200 MB. |
|
| Maximum number of concurrent transactions supported by the storage engine. |
|
| Size of the tablet map shard. |
|
| Whether to enable the new on-disk storage format for the BITMAP type, which improves |