Java UDF best practices
Dataphin Enterprise Edition supports custom Java user-defined functions (UDFs) for complex data development scenarios. This tutorial uses the built-in Java toLowerCase function as an example to show you how to create a custom Java UDF in Dataphin.
Prerequisites
You have downloaded the JAR package.
Background information
This tutorial uses the downloaded JAR package to create a custom Java UDF that converts uppercase letters to lowercase. You can also write your own Java UDF code to implement more features. For more information, see Best practices for developing Java UDFs in IntelliJ IDEA.
To reference a UDF across projects in MaxCompute, use the format project_name:function_name. For example, if the `test_dev` project has a custom function named `to_normal`, you can use it in the `xyz_dev` project by calling test_dev:to_normal.
The code for the JAR package in this tutorial is shown below.
package org.alidata.odps.udf.examples;
import com.aliyun.odps.udf.UDF;
public final class javaudf extends UDF {
public String evaluate(String s) {
if (s == null) {
return null;
}
return s.toLowerCase();
}
}Where:
The package path is org.alidata.odps.udf.examples.
The class file name is javaudf.
Step 1: Upload the JAR package
Log in to the Dataphin console.
On the Dataphin console, select the region where your workspace is located and click Enter Dataphin>>.
Go to the Resource Management page.
In the top menu bar, choose Develop > Data Studio.
In the navigation pane on the left, choose Data Processing > Resource.
Above the list of resources, click the
icon.
In the New Resource dialog box, configure the following parameters.
Parameter
Description
Type
Select jar.
Name
The name of the uploaded file must end with the file extension. For example, javaudf.jar.
Description
Enter a description for the resource.
Upload File
Select the local JAR file, such as javaudf.jar.
Compute Type
Select MaxCompute.
Select Directory
Select the folder to store the JAR package. The system default is Resource Management. Keep the default setting.
To submit the resource, click Submit.
In the Submit Note dialog box, enter a note for the submission.
Click OK and Submit.
(Optional) Publish the resource to the production environment.
If you use the Dev-Prod development mode, you must publish the resource to the production environment. For more information, see Manage publishing tasks.
If you use the Basic development mode, the resource is automatically published to the production environment after you submit it.
Step 2: Create a MAXC function
On the Develop page, in the navigation pane on the left, choose Data Studio > Function.
Above the function list, click the
icon and select MAXC Function.In the New Function dialog box, configure the following parameters.
Parameter
Description
Name
Enter a name for the function, such as Java.
Select Resource
Select the uploaded resource javaudf.jar.
Class Name
The class name format is Package Path.Class File Name. Enter org.alidata.odps.udf.examples.javaudf.
Type
The type of the function. Select String.
Command Format
Define the format for calling the function. Enter to_char(string i).
Usage Document
Enter a description of how to use the function, such as javaudf.
Select Directory
The default is MAXC Function-User-defined Function. Keep the default setting.
Click Submit.
In the Submit Note dialog box, enter a note for the submission.
Click OK and Submit.
(Optional) Publish the function to the production environment.
If you use the Dev-Prod development mode, you must publish the function to the production environment. For more information, see Manage publishing tasks.
If you use the Basic development mode, the function is automatically published to the production environment after you submit it.
Step 3: Create an SQL node
On the Develop page, in the navigation pane on the left, choose Data Processing > Script Task.
Above the list of compute nodes, click the
icon and select MAX_COMPUTE_SQL.In the New MAX_COMPUTE_SQL Node dialog box, configure the following parameters.
Parameter
Description
Name
Enter a name for the compute node, such as javaudf.
Scheduling Type
Set the scheduling type for the task to Recurring Task Node.
Select Directory
Compute Job is selected by default.
Use Template
Click the switch to specify whether to use a template. If you enable this option, you must also select a Template and Template Version.
Description
Enter a brief description for the job.
Click OK.
Step 4: Use the Java UDF
On the code editor page for the SQL node, enter the code. For example,
select java('ABCGDfagHH');.Above the code editor, click Run to run the current code and view the result.
(Optional) Schedule and run the node
To run the SQL node periodically, configure its scheduling properties and publish it to the production environment.
On the code editor page, click Property in the right-side sidebar.
In the Property panel, configure the Scheduling properties. For more information about how to configure scheduling properties, see Configure scheduling properties for a logical table.
Save, submit, and publish SQL jobs.
Above the code editor, click the Save icon to save the code.
Above the code editor, click the Submit icon to submit the code.
In the Submitting Log dialog box, confirm the submission content and pre-check results, and enter a note for the submission.
Click OK and Submit.
(Optional) Publish the SQL node to the production environment.
If you use the Dev-Prod development mode, you must publish the SQL node to the production environment. For more information, see Manage publishing tasks.
If you use the Basic development mode, the SQL node is automatically published to the production environment after you submit it.