EMR Trino node-DataWorks(DataWorks)-阿里云帮助中心

Prerequisites

You have created an Alibaba Cloud EMR cluster and registered it with DataWorks. For more information, see Data Studio: Bind an EMR computing resource.
(Optional, for RAM users) The RAM user for task development has been added to the corresponding workspace and granted the Development or Workspace Administrator (this role has extensive permissions, grant with caution) role. For more information about how to add members, see Add members to a workspace.

If you use an Alibaba Cloud account, you can skip this step.

Limitations

EMR Trino nodes can run only on a serverless resource group.
To manage metadata for data lake or custom clusters in DataWorks, you must first configure EMR-HOOK on the cluster. For details, see Configure EMR-HOOK for Hive.

Note
If EMR-HOOK is not configured on the cluster, DataWorks cannot display real-time metadata, generate audit logs, show data lineage, or perform EMR-related governance tasks.
If Lightweight Directory Access Protocol (LDAP) authentication is enabled for Trino, log on to the EMR master node to download the keystore file from the /etc/taihao-apps/trino-conf directory. Log on to the DataWorks console. In the top navigation bar, select the target region, find your workspace, and click Operation in the Management column to go to the Management Center. In the left-side navigation pane, select Compute Resource. Find the Account Mapping for your EMR cluster and click Edit Account Mapping. On the page that appears, click Upload Keystore File to upload the file.

Procedure

On the EMR Trino node editing page, follow these steps.

Configure connectors

Before you query MySQL tables, configure the built-in EMR Trino connector for MySQL. For details, see MySQL Connector.
Before you query Hive tables, configure the built-in EMR Trino connector for Hive. For details, see Hive Connector.
To query tables from other data sources, see Configure connectors.

Edit SQL code

In the SQL editor, write your task code. You can define variables by using the ${variable_name} format. In the Scheduling Settings pane on the right, you can assign values to the variables in the Scheduling Parameters section. This allows you to dynamically pass parameters during scheduled runs. For more information about scheduling parameters, see Sources and expressions of scheduling parameters. The following code provides an example.

-- Usage
-- SELECT * FROM <catalog>.<schema>.<table>;
-- Parameter description
-- <catalog>: The name of the data source to connect to.
-- <schema>: The name of the database to use.
-- <table>: The table to query.
-- Example: Query data from the hive_table table in the default database of a Hive data source.
-- Query a Hive table
SELECT * FROM hive.default.hive_table;
-- Example: Query data from the rt_user table in the custom rt_data database of a MySQL data source.
-- Query a MySQL table  
SELECT * FROM mysql.rt_data.rt_user;
-- Join a Hive table and a MySQL table
SELECT DISTINCT a.id, a.name,b.rt_name FROM hive.default.hive_table a INNER JOIN mysql.rt_data.rt_user b ON a.id = b.id;
-- Query a Hive table by using a scheduling parameter
SELECT * FROM hive.default.${table_name};

(Optional) Configure advanced parameters

On the right side of the node editing page, in the Scheduling Settings pane, you can configure the following properties under EMR Node Parameters > DataWorks parameters.

Note

You can configure open-source Spark properties under EMR Node Parameters > Spark parameter in the Scheduling Settings pane on the right.

Parameter

Description

FLOW_SKIP_SQL_ANALYZE

Specifies how SQL statements are executed. Valid values:

true: Executes multiple SQL statements at a time.
false (default): Executes one SQL statement at a time.

DATAWORKS_SESSION_DISABLE

Controls JDBC connection reuse when you run tests in the development environment. Valid values:

true: Creates a new JDBC connection for each SQL statement execution.
false (default): Reuses the same JDBC connection for different SQL statements that run within the same node.

Run the SQL task

In the Run Configuration pane, under Compute Resource, select a Compute Resource and a DataWorks Resource Group.
Note
- You can also adjust the CUs for Scheduling based on the resources required by the task. The default value is 0.25.
- To access data sources over the public internet or in a Virtual Private Cloud (VPC), you must use a scheduling resource group that has passed the connectivity test for the data source. For details, see Network Connectivity Solutions.
In the parameter dialog box on the toolbar, select a data source and click Run to run the SQL task.

Note
Query results from an EMR Trino node are limited to 10,000 rows and 10 MB.

To run the node task on a regular basis, configure its scheduling properties based on your business requirements. For details, see Configure Node Scheduling.
After you configure the node, you must publish it. For details, see Publish a Node or Workflow.
After the task is published, you can view its operational status in Operation Center. For details, see Get started with Operation Center.

FAQ

Q: A connection timeout occurs when I run the node. Why?
```
EMR execute task failed!
SQL: {"name":"dw20251018","type":"TRINO_SQL","launcher":{"allocationSpec":{}},"properties":{"envs":{"FLOW_SKIP_SQL_ANALYZE":false},"arguments":["select * from default.dim_customers;"],"tags":[]},"description":"DataWorks"}
TASK-MESSAGE:
FAILED: Error executing query
```
A: This error can occur if the resource group and the cluster do not have network connectivity. To resolve this issue, go to the computing resource list page, find the resource, and click Resource Initialization. In the dialog box that appears, click Re-initialize and ensure that the resource is successfully initialized.

The dialog box displays the following message at the top: If this is the first time you bind a cluster or if the cluster service configuration (for example, the hive-site file) has changed, initialize the resource group. Otherwise, tasks may fail to run.