Standardized rules and datasets

Rule Sources

Standardized rules use the SPL syntax to map incoming raw logs to standardized log structures and define key field mappings. This provides the data foundation for generating alerts from normalized logs.

Create custom standardized rules

1. Access the Security Center console - Agentic SOC - Management - Access Settings. In the upper-left corner of the page, select the region where the assets to be protected are located: Chinese Mainland or Outside Chinese Mainland.

2. Configure the following parameters for the basic settings.

   Parameter	Description
   Rule Name	A custom name for the rule.
   Service Provider	Predefined vendors: Alibaba Cloud, Fortinet, Chaitin, Microsoft, Sangfor, Tencent Cloud, Huawei Cloud, Hillstone Networks, DbappSecurity, Microsoft Cloud, and others. Custom vendors: For information about how to create vendors, see Product integration.
   Service	Automatically lists all products under the selected vendor, such as Security Center for Alibaba Cloud and Fortinet Firewall. For the list of supported products, see Integration settings.
   Remarks	Add a description for the standardized rule to facilitate search and improve readability.

3. On the Log Standardization Test tab, configure the field mappings for the standardization rule.

   1. Select a log sample source:

      • Based on data source

        • Data source: Automatically synchronizes existing data source instances.

        • Log sample: Automatically queries log samples from the last 7 days. You can also select a custom time range.

      • Based on manual input: Paste raw log data into the JSON editor. The editor supports formatting and validation.

   2. Create a parsing plan by using either the AI-recommended configuration or manual configuration.

      AI-recommended configuration

      Invoke the Security AI Assistant from the console. Agentic SOC automatically assigns a log standardization agent to analyze log samples, recommend a standardization classification, structure, and field mapping, and generate SPL syntax.

      1. In the Parse test section, click Security AI Assistant. The AI assistant dialog panel appears on the right.

      2. Obtain the recommended configuration: After analyzing the log sample, the Security AI Assistant returns the recommended configuration.

         • Standardization classification and structure: The AI identifies the log type based on the log content and recommends a classification.

           • Standardization classifications: Network Log, Host Log, Security Log, Audit Log, Snapshot Log, Logon Log, and Other Log.

           • Standardization structure:

             • Each standardization classification contains multiple standardization structures.

             • Each standardization structure corresponds to a set of standardized fields and a dataset (StoreView). A dataset can be mapped to multiple standardization structures.

             • For information about how to view the dataset and standardized fields for a classification or structure, see View standardized fields and datasets.

         • Mapping results table: Displays the mapping between each log field and its standardized field, and indicates whether the field is required or optional. For fields that the AI does not automatically match, select a standardized field from the drop-down list.

      3. Generate SPL syntax: Below the recommended configuration, click Generate SPL. The AI generates the SPL syntax based on the mapping results, and provides a mapping decision table and a compliance self-check.

      4. Populate the configuration form: Below the generated SPL syntax, click Apply. The AI-recommended standardization classification, structure, and SPL syntax are automatically filled into the rule editing form.

      Manual configuration

      • Standardization classification and structure: For the default standardization classifications and structures supported by Agentic SOC, and their field descriptions, see View standardized fields and datasets.

      • SPL syntax:

        • Use predefined rules and existing custom rules as templates. View the SPL syntax for each rule on its details page.

        • Or write custom syntax by referencing SPL syntax documentation.

   3. Configure extended fields and syntax type

      • Extended field ingestion:

        • Pass-through: Raw log fields that are not mapped to standardized fields are stored directly in the dataset as key-value pairs. This preserves all original data but increases log volume and traffic costs.

        • Not Ingested (Default): Raw log fields that are not mapped by the standardized rules are discarded.

      • Syntax type:

        • General syntax: When you configure a log ingestion policy, you can select either Real-time Consumption or Scan Query. For more information, see the standardization method comparison in Product logs.

        • Real-time consumption: When you configure a log ingestion policy, the standardization method supports only Real-time Consumption.

   4. Parse test

      1. After configuring the standardization classification and parsing syntax, click Parse.

      2. View the parse results in the Parse Results section.

      3. Manage extended fields (optional)

         If the SPL parsing results contain fields that are not mapped to standardized fields and extended field ingestion is set to Pass-through, we recommend creating indexes for these fields to facilitate quick searches after ingestion. For more information, see Manage standardized rules.

4. After the test passes, click Complete in the lower-left corner.

Manage standardized rules

View standardized fields and datasets

1. Access the Security Center console - Agentic SOC - Management - Access Settings. In the upper-left corner of the page, select the region where the assets to be protected are located: Chinese Mainland or Outside Chinese Mainland.

2. In the Standard Fields list on the left, the items under Log Activity Category are the Standardization Category supported by Agentic SOC. The root nodes represent the Standardization Structure.

3. Click the expand icon next to a Standardization Category (for example, Network Log) to view the Standardization Structure under it (for example, 5-tuple Log, DNS Log, and HTTP Log).

4. Click a Standardization Structure (for example, API Risk Log). The following information appears on the right:

   • Standardized Field: Displays the standardized fields for the selected structure, including field name, type, description, required status, and sample values.

   • Extended Fields: Displays user-defined extended fields, including field name, type, description, tokenization status, and update time.

   • Dataset: Displays the Dataset for the selected structure.

Manage extended fields

Dataset

Mapping between datasets and standardized fields

Standardized rules require a Standardization Classification/Structure configuration. Each standardization classification contains multiple standardization structures. Each structure corresponds to a dataset (StoreView) and a set of standardized fields. A dataset (StoreView) can also be mapped to multiple standardization structures. The following figure illustrates the mapping: