Advanced preview

Procedure

1. Log on to the Simple Log Service console.

2. Navigate to the data transformation page.

   1. In the Projects section, click the project you want.

   2. On the Log Storage > Logstores tab, click the logstore you want.

   3. On the query and analysis page, click Data Transformation.

3. In the upper-right corner of the page, select a time range.

   After you select a time range, make sure that logs appear on the Raw Logs tab.

4. In the edit box, enter a data transformation statement.

   For more information, see Data processing syntax.

   Note

   You can add comments to the data transformation statement to debug the statement line by line.

5. Preview the data.

   1. In the upper-right corner, click Advanced.

   2. Click Preview Data.

   3. In the Add Preview Settings dialog box, configure the following parameters and click OK.

      You must configure these parameters the first time you preview data. To change them later, click Modify Preview Settings.

      Parameter	Description
      authorization method	You can use one of the following methods to grant permissions to a data transformation job to read data from the source Logstore: Default Role: The job assumes the AliyunLogETLRole system role to read data from the source logstore. Custom Role: The job assumes a custom role to read data from the source logstore. First, grant the custom role the permissions to read data from the source logstore. Then, enter the Alibaba Cloud Resource Name (ARN) of the custom role in the Role ARN field. For more information, see Use a custom role to access data.
      Advanced parameter settings	If your transformation statements require sensitive information, such as a database password, you can store it as a key-value pair. You can then reference the value in your statements by using a variable such as ${key}. Click + to add multiple key-value pairs. For example, config.vpc.vpc_id.test1:vpc-uf6mskb0b****n9yj can represent the ID of the VPC where an RDS instance resides.

      After you configure the preview settings, you can view the transformation result on the Transformation Results tab.

      • If a data transformation fails due to a syntax error or incorrect permissions, follow the on-screen prompts to resolve the issue.

      • If the transformation result is as expected, you can save the settings as a data transformation job. For more information, see Create a data transformation job.

Example

• Transformation statements

  # e_set("insert_field", "test_value")
  e_table_map(
      res_rds_mysql(
          address="rm-uf6wjk5****.mysql.rds.aliyuncs.com",
          username="test_username",
          password="****",
          database="test_db",
          table="test_table",
      ),
      "ip",
      ["country", "province"],
  )

• Raw log

  {
    "__source__": "192.0.2.0",
    "__time__": 1624523917,
    "__topic__": "topic",
    "id": "1001",
    "ip": "127.0.0.1"
  }

• Advanced preview settings: In the Add Preview Settings dialog box, set authorization method to default role. The system automatically populates the role ARN in the format acs:ram::<Account ID>:role/aliyunlogetlrole.

• Transformation result: The raw log now includes the fields country:China and province:Shanghai. The output target is target0. One log was transformed successfully, with no failures.