Configure data computing and data filtering
Define fields, configure expressions, and apply filters to preprocess parsed data for downstream analysis or output.
Prerequisites
Ensure the source node is configured.
Configure data computing
You can configure expressions to preprocess the output fields from the preceding node.
-
In the middle canvas, click the add
icon following the current node. -
From the node list that appears, click the Data Computing node.
-
Click the Data Computing node in the canvas to display the original field list in the right-side configuration panel.
-
Adjust the Field Name, Data Type, Precision, Decimal Place, and the Output To Next Node option as needed.
NoteOnly fields with the Output To Next Node option selected can be used in subsequent node configurations.
-
(Optional) Click Add Field, and in the dialog box that appears, configure the Field Name, Data Type, Precision, Decimal Places, Output To Next Node, and Expression following the page prompts.
Important-
Expressions support functions such as Date functions, Mathematical functions, and String functions. Nested use of different function types is permitted.
-
Use the Calculation Formula column's Original field for expressions.
-
You can use JSONPath in expressions to retrieve property values from JSON data, including arrays and nested JSON objects.
For example, given
{a:{key1:v1, key2:v2}}, you can retrieve the valuev2by accessinga.key2.For more information about JSONPath, see LanguageManual UDF.
-
Distinguish between single and double quotes when using variables in expressions:
-
Single quotes denote constants.
-
Double quotes or no quotes indicate variables.
-
-
-
(Optional) Repeat the previous step to add additional fields as needed.
To generate additional data from newly added fields, configure multiple expressions by repeating the steps above.
-
Click Save in the upper right corner of the data parsing workspace to finalize the Data Computing node configuration.
Configure data filtering
You can add filter conditions to refine the output fields from the preceding node.
-
In the middle canvas, click the add
icon following the current node. -
From the node list that appears, click the Data Filtering node.
-
Click the Data Filtering node in the canvas. In the right-side configuration panel, use the table below to add filter conditions.
Item
Description
Field
Select the field to filter.
Data Type
The data type is automatically determined when you select a field and cannot be changed.
Operator
Choose from operators such as ≠ (Not Equal To), = (Equal To), > (Greater Than), < (Less Than), >= (Greater Than Or Equal To), <= (Less Than Or Equal To), Between, Like, and isNotNull.
Filter Value
Enter the filter value.
-
(Optional) Click Add AND Condition to include multiple AND conditions, following the steps above.
-
Click Save in the upper right corner of the data parsing workspace to finalize the Data Filtering node configuration.
What to do next
After configuring the data computing and filtering nodes, set up other processing nodes or configure the target node to complete the data parsing task.