Create a MaxCompute user-defined function (UDF)-DataWorks(DataWorks)-阿里云帮助中心

If the built-in functions in MaxCompute cannot meet your business requirements, you can create a user-defined function (UDF) to extend its capabilities. This topic describes how to create a UDF in DataWorks using the visual interface.

Background

A user-defined function (UDF) extends the existing function library. You can write custom logic and calculations to be executed during queries to enable more powerful data processing. For more information, see UDF overview. In addition to using the visual interface in DataWorks, you can also create a UDF using MaxCompute Studio or by running commands in MaxCompute. For more information, see Create a user-defined function in MaxCompute Studio and Create a user-defined function in MaxCompute by running commands.

Prerequisites

Functions in MaxCompute are based on resources. Before you create a function, you must first create a MaxCompute resource and upload or add it to DataWorks. For more information, see Create a MaxCompute resource.

Note

When you create a MaxCompute resource, you can refer to Develop a UDF (Java) and Develop a UDF (Python 3) to prepare the MaxCompute resource files.

Limitations

In DataWorks, you can only view and manage user-defined functions that are uploaded through the visual interface. For UDFs that are added to the MaxCompute engine by using another tool, such as MaxCompute Studio, you must manually load them into DataWorks by using the MaxCompute Functions feature. Once loaded, you can view and manage these functions in DataWorks. For more information, see Manage MaxCompute functions.

Register a function

Log on to the DataWorks console. In the target region, click Data Development and O&M > Data Development in the left-side navigation pane. Select a workspace from the drop-down list and click Go to Data Development.
Create a workflow. For more information, see Create an auto triggered workflow.

Create a function.

Open the workflow, right-click MaxCompute, and then select Create Function.
In the Create Function dialog box, enter a Name and select a Path.
Click Create.

In the Register Function dialog box, configure the parameters.

Parameter	Description
Function Type	Select a function type. Valid values are Mathematical Operation Functions, Aggregate Functions, String Processing Functions, Date Functions, Window Functions, and Other Functions. For more information, see Use built-in functions.
MaxCompute Engine Instance	This parameter is read-only.
Function Name	The name used to reference the function in SQL statements. The name must be globally unique and cannot be changed after registration.
Owner	The owner of the function. By default, this is the current user, but you can select another account.
Class Name	The class name of the UDF. The format is `ResourceName.ClassName`. The resource name can be a Java package name or a Python resource name. DataWorks supports creating user-defined functions with MaxCompute resources of the JAR and Python types. The class name format varies based on the resource type: For JAR resources, the format for the Class Name is `PackageName.ActualClassName`. You can obtain this value in IntelliJ IDEA by using the `copy reference` action. For example, if the Java package name is `com.aliyun.odps.examples.udf` and the actual class name is `UDAFExample`, set the Class Name parameter to `com.aliyun.odps.examples.udf.UDAFExample`. For Python resources, the format for the Class Name is `PythonResourceName.ActualClassName`. For example, if the Python resource name is `LcLognormDist_sh` and the actual class name is `LcLognormDist_sh`, set the Class Name parameter to `LcLognormDist_sh.LcLognormDist_sh`. Note Do not include the .jar or .py suffix in the resource name. You must submit and publish the resource before it can be used. For more information, see Create and use MaxCompute resources.
Resources	Select the resources required to register the function. Visual mode: You can select only resources that have been uploaded or added to DataWorks. Code editor: You can select all resources in the corresponding MaxCompute engine instance. Note You do not need to enter the path of an added resource. If the UDF calls multiple resources, separate the resource names with a comma (,).
Description	A brief description of the UDF.
Expression Syntax	An example of how to use the UDF, such as `test`.
Parameter Description	A description of the supported input and return parameter types.
Return Value	An example of the return value, such as 1. This parameter is optional.
Example	A usage example. This parameter is optional.

Click the icon in the toolbar to save the function.
Submit the function.
1. Click the icon in the toolbar.
2. In the Submission dialog box, enter a Change Description.
3. Click OK.

To view the functions in a MaxCompute engine instance and their change history, see Manage MaxCompute functions.

Version history and rollback

Right-click the function name and select View Earlier Versions to view the version history or perform a rollback.

In the panel that appears, the version list shows information such as the submission time and change description for each version. You can select a target version and perform a rollback.

Use a user-defined function in a node

You can reference a user-defined function by its name. You can also select the resource, right-click it, and choose Insert Function to quickly insert the function name into the current node editor.

Appendix 1: View UDF list

You can run the SHOW FUNCTIONS command to list all registered user-defined functions in the MaxCompute project that is bound to DataStudio in your DataWorks workspace.
MaxCompute provides many built-in functions. For more information, see Overview of built-in functions.

// View the functions in the current project.
SHOW FUNCTIONS;

Appendix 2: View UDF details

You can run the DESCRIBE or DESC command followed by a function name to view the details of a user-defined function.
```
// Use the short form to view the details of a user-defined function.
DESC FUNCTION <function_name>;
```
In DataWorks, if the existing functions cannot meet your data processing requirements in a workflow, you can write a MaxCompute user-defined function. Then, you can upload and associate the corresponding resources, such as JAR packages or Python files, to manage and extend your data processing capabilities. For more information, see Manage MaxCompute resources.

Best practices

After you create a user-defined function, see Best practices for allowing a specified user to access a specific UDF to implement access control for it.

FAQ

Q: After I upload a resource to DataWorks and define it as a UDF, can I use it in SQL queries in DataAnalysis in addition to using it in ODPS SQL nodes in DataStudio?

A: Yes, you can. UDFs that you register in DataWorks are stored in the corresponding MaxCompute project. Therefore, they can be used not only in ODPS SQL nodes but also in SQL queries in DataAnalysis.

Create and use a user-defined function