Create an EMR function

Procedure

1. Go to the DataStudio page.

   Log on to the DataWorks console. In the target region, click Data Development and O&M > Data Development in the left-side navigation pane. Select a workspace from the drop-down list and click Go to Data Development.

2. Create a workflow. For more information, see Create a scheduled workflow.

3. Write your program in a local Java environment, package it into a JAR file, and then create and submit a JAR resource. For more information, see Create and use EMR resources.

4. Create a function.

   1. Open the workflow you want, right-click EMR, and select Create Function.

   2. In the Create Function dialog box, enter a Name, and select an Engine Instance and a Path.

   3. Click Create.

   4. In the Register Function dialog box, configure the following parameters.

      Parameter	Description
      Function Type	Select a function type. Options include Mathematical Operation Functions, Aggregate Functions, String Processing Functions, Date Functions, Window Functions, and Other Functions.
      EMR engine instance	A default value is displayed and cannot be changed.
      EMR engine type	A default value is displayed and cannot be changed.
      EMR database	Select a database from the drop-down list. To create a new database, click Create Database. In the Create Database dialog box, configure the parameters and click OK.
      Function Name	The UDF name used in SQL statements. This name must be globally unique and cannot be changed after registration.
      Owner	The owner is displayed by default.
      Class Name	Required. The main class name that implements the UDF.
      Resources	Required. From the drop-down list, select a resource that belongs to the current workspace. To create a resource, click Create Resource. In the Create Resource dialog box, configure the parameters and click Determine.
      Description	A brief description of the UDF.
      Expression Syntax	An example of how to use this UDF is test.
      Parameter Description	The supported input and return parameter types.
      Return Value	Optional. The return value, such as 1.
      Example	Optional. A usage example.

5. Click the Save icon in the toolbar.

6. Submit the function.

   1. Click the Submit icon in the toolbar.

      Note

      When you submit the function, you must select a scheduling resource group. We recommend using a serverless resource group. If you use a serverless resource group, DataWorks submits the function registration task to the EMR engine, which then runs the task and prints execution logs. If an error occurs during submission, you can use the logs for troubleshooting. If a serverless resource group is not available, you must purchase and configure one. For more information, see Use serverless resource groups.

   2. In the Commit Node dialog box, enter a Change Description.

   3. Click OK.