Create an EMR function

Procedure

1. Go to the DataStudio page.

   Log on to the DataWorks console. In the target region, click Data Development and O&M > Data Development in the left-side navigation pane. Select a workspace from the drop-down list and click Go to Data Development.

2. Create a workflow. For more information, see Create a scheduled workflow.

3. Write your program in a local Java environment, package it into a JAR file, and then create and submit a JAR resource. For more information, see Create and use EMR resources.

4. Create a function.

   1. Open the workflow you want, right-click EMR, and select Create Function.

   2. In the Create Function dialog box, enter a Name, and select an Engine Instance and a Path.

   3. Click Create.

   4. In the Register Function dialog box, configure the following parameters.

      Parameter	Description
      Function Type	Select a function type. Options include Mathematical Operation Functions, Aggregate Functions, String Processing Functions, Date Functions, Window Functions, and Other Functions.
      EMR engine instance	A default value is displayed and cannot be changed.
      EMR engine type	A default value is displayed and cannot be changed.
      EMR database	Select a database from the drop-down list. To create a new database, click Create Database. In the Create Database dialog box, configure the parameters and click OK.
      Function Name	The UDF name used in SQL statements. This name must be globally unique and cannot be changed after registration.
      Owner	The owner is displayed by default.
      Class Name	Required. The main class name that implements the UDF.
      Resources	Required. Select a resource in the current workspace. To create a resource, click Create Resource. In the Create Resource dialog box, configure the parameters and click Determine.
      Description	A brief description of the UDF.
      Expression Syntax	Example usage of the UDF, such as test.
      Parameter Description	The supported input and return parameter types.
      Return Value	Optional. The return value, such as 1.
      Example	Optional. A usage example.

5. Click the Save icon in the toolbar.

6. Submit the function.

   1. Click the Submit icon in the toolbar.

      Note

      When you submit the function, select a scheduling resource group. A serverless resource group is recommended. With a serverless resource group, DataWorks submits the function registration task to the EMR engine, which runs the task and prints execution logs for troubleshooting. If no serverless resource group is available, purchase and configure one. For more information, see Use serverless resource groups.

   2. In the Commit Node dialog box, enter a Change Description.

   3. Click OK.