How to process and analyze data using Transforms-Elasticsearch(ES)-阿里云帮助中心

Elasticsearch Transforms reshapes raw data into a new, summarized index, which helps you prepare data for analysis and visualization without modifying the original source. This topic shows you how to use a transform to aggregate shot data and view the results.

Prerequisites

Download the sample data. This topic uses the Kaggle datasetNBA shot logs. This dataset includes fields such as shot time, shooter, shot location, closest defender, and the distance between the shooter and the closest defender. To download the dataset, see shot_logs.csv.
Create an Alibaba Cloud Elasticsearch cluster. For more information, see Create an Alibaba Cloud Elasticsearch cluster. This topic uses an Elasticsearch 7.10.0 cluster.

Note
Alibaba Cloud Elasticsearch version 8.5 clusters are not supported.
Log on to the Kibana console. For more information, see Log on to the Kibana console.
Create the nba_shot_logs index and an index pattern by importing the sample data.
1. In the upper-left corner of the Kibana page, choose > Kibana > Machine Learning.
2. Click the Data Visualizer tab.
3. In the Import data section, click Upload file.
4. Click the icon.
5. Select theshot_logs file.
6. In the lower-right corner of the page, click Import.
7. On the Simple tab, enternba_shot_logs for Index name and select Create index pattern.
8. Click Import.
  
  The figure below shows the page after the data is imported.

Procedure

This topic describes three methods to transform and view data.

Method 1: Create a transform in Kibana

In the upper-left corner of the Kibana page, choose > Management > Stack Management.
In the left-side navigation pane, click Data > Transforms.
Click Create your first transform.
Select thenba_shot_logs index.
In the Configuration section, select histogram(GAME_ID) for Group by. For Aggregations, select DRIBBLES.sum, DRIBBLES.avg, and DRIBBLES.max.
Note
- Group by GAME_ID: Groups data by game ID.
- DRIBBLES.sum: The total number of dribbles by all players in each game.
- DRIBBLES.avg: The average number of dribbles per player in each game.
- DRIBBLES.max: The maximum number of dribbles by a single player in each game.
In the lower-right corner of the Configuration section, click Next.
In the Transform details section, specify the Transform ID and Destination index, and then click Next on the right side of the page.
In the Create section, click Create and start.

Note
The transform is created when the progress bar reaches 100%.
Click Discover to view the data in the destination index.

Method 2: Preview with the API

In the upper-left corner of the Kibana page, choose > Management > Dev Tools.

On the Console tab, run the following command to create a transform.

PUT _transform/test2_nba_shot_logs
{
    "source": {
    "index": "nba_shot_logs"
  },
  "dest" : { 
    "index" : "test2_nba_shot_logs"
  },
  "pivot": {
    "group_by": { 
      "game_id": { "terms": { "field": "GAME_ID" }}
    },
    "aggregations": {
      "dribbles_sum": { "sum": { "field": "DRIBBLES" }},
      "dribbles_avg": { "avg": { "field": "DRIBBLES" }},
      "dribbles_max": { "max": { "field": "DRIBBLES" }}
    }
  }
}

Run the following command to preview the transformed data.

POST _transform/_preview
{
  "source": {
    "index": "nba_shot_logs"
  },
  "dest" : { 
    "index" : "test2_nba_shot_logs"
  },
  "pivot": {
    "group_by": { 
      "game_id": { "terms": { "field": "GAME_ID" }}
    },
    "aggregations": {
      "dribbles_sum": { "sum": { "field": "DRIBBLES" }},
      "dribbles_avg": { "avg": { "field": "DRIBBLES" }},
      "dribbles_max": { "max": { "field": "DRIBBLES" }}
    }
  }
}

Method 3: View in Discover with the API

Create and start the transform.

In the upper-left corner of the Kibana page, choose > Management > Dev Tools.

On the Console tab, run the following command to create a transform.

PUT _transform/test2_nba_shot_logs
{
    "source": {
    "index": "nba_shot_logs"
  },
  "dest" : { 
    "index" : "test2_nba_shot_logs"
  },
  "pivot": {
    "group_by": { 
      "game_id": { "terms": { "field": "GAME_ID" }}
    },
    "aggregations": {
      "dribbles_sum": { "sum": { "field": "DRIBBLES" }},
      "dribbles_avg": { "avg": { "field": "DRIBBLES" }},
      "dribbles_max": { "max": { "field": "DRIBBLES" }}
    }
  }
}

Start the transform.
```
POST _transform/test2_nba_shot_logs/_start
```
Note
Transforms are stopped by default.

Create an index pattern.

Note
Before you can view data in Discover, you must create an index pattern.
1. In the upper-left corner of the Kibana page, choose > Management > Stack Management.
2. In the left-side navigation pane, click Kibana > Index Patterns.
3. In the upper-right corner of the page, click Create index pattern.
4. In the Index pattern name field, enter the transform's destination index, test2_nba_shot_logs, and click Next step.
5. Click Create index pattern.
View the data in Discover.
1. In the upper-left corner of the Kibana page, choose > Kibana > Discover.
2. Select the destination index to view the transformed data.