Group statistics: aggregate clause

A keyword query may return tens of thousands of documents, and users are unlikely to browse all of them to find the information they need. In some cases, users may be more interested in statistical data.

Syntax

Format of the statistics clause:

group_key:field, range:number1~number2, agg_fun:func1#func2, max_group:number2, 
agg_filter:filter_clause, max_group:number

Parameters:

Parameter	Type	Required	Valid values	Default value	Description
group_key:field	field must be an attribute field	Yes	Statistics are currently supported only for fields of the INT, LITERAL, INT_ARRAY, and LITERAL_ARRAY types. (Note: For the INT_ARRAY and LITERAL_ARRAY field types, statistics are calculated by grouping identical values in the field.)		field specifies the name of the field on which statistics are calculated. It must be configured as an attribute field.
agg_fun		Yes	count(), sum(id), max(id), min(id), distinct_count (five system functions in total)		func can be one of five system functions: count(), sum(id), max(id), min(id), and distinct_count(id). They respectively count the number of documents, calculate the sum of the id field, obtain the maximum value of the id field, obtain the minimum value of the id field, and count the number of distinct values of id. Multiple functions can be calculated at the same time, separated by number signs (#). The content of sum, max, and min supports basic arithmetic operations.
range		No	Indicates the intervals from number1 to number2 and greater than number2. (Note: Distribution statistics are not supported for fields of the string type.)		Specifies range-based statistics, which can be used for distribution statistics. Only a single range parameter is supported.
agg_filter		No			Specifies that only documents that meet specific conditions are counted.
agg_sampler_threshold	int	No			, the threshold for sampling-based statistics. Documents before this value are counted one by one, and documents after this value are counted by sampling;
agg_sampler_step	int	No			The step size for sampling-based statistics. Starting after agg_sampler_threshold, one document is counted every agg_sampler_step documents. For sum and count statistics, the sampling result after the threshold is multiplied by the step size to produce an estimate, and this estimate is added to the statistics result before the threshold to obtain the final result.
max_group	int	No		1000	The maximum number of groups to return

Usage notes

The aggregate clause is optional.
Fields that appear in the aggregate clause must be configured as attribute fields when you define the application schema.
The aggregate results are displayed in the facet node of the search results. The field names of the specific values are the names of the agg_fun functions, such as sum and count.
The aggregate clause supports statistics on multiple keys. Separate multiple statistics with semicolons (;).

Reference format:

group_key:field1,agg_fun:func1;group_key:field2,agg_fun:func2

The information of this statistics clause is displayed in facet. You must set format in the config clause to fulljson for the facet content to be returned and displayed.
The distinct_count feature is supported only in exclusive clusters, and you must add the enable_accurate_statistics:true parameter to the kvpairs clause. When you use the distinct_count feature, the query no longer retrieves specific result data and returns only the statistical data in facet.
In exclusive clusters, accurate statistics are supported for count, max, min, and sum. You must add the enable_accurate_statistics:true parameter to the kvpairs clause.
Due to engine performance limits, the aggregate statistics clause can guarantee accurate document count statistics only when up to 100,000 documents are retrieved. Accuracy is not guaranteed when more than 100,000 documents are counted. (In exclusive clusters, you can add the enable_accurate_statistics:true parameter to the kvpairs clause to improve statistical accuracy.)

Examples

Search for documents that contain "Zhejiang University", calculate statistics by the group_id field, including the sum and the maximum value of the price field, and count the documents by company_id:

query=default:'Zhejiang University'&&aggregate=group_key:group_id,agg_fun:sum(price)#max(price);group_key:company_id,agg_fun:count()

Results:

{
　　status: "OK",
　　result: {
　　　　searchtime: 0.015634,
　　　　total: 5,
　　　　num: 1,
　　　　viewtotal: 5,
　　　　items: [        //Specific search results
　　　　　　{ ... }
　　　　],
　　　　facet: [
　　　　　　{
　　　　　　　　key: "group_id",
　　　　　　　　items: [
　　　　　　　　　　{
　　　　　　　　　　　　value: 43,
　　　　　　　　　　　　sum: 81,
　　　　　　　　　　　　max: 20,
　　　　　　　　　　},
　　　　　　　　　　{
　　　　　　　　　　　　value: 63,
　　　　　　　　　　　　sum: 91,
　　　　　　　　　　　　max: 50,
　　　　　　　　　　},
　　　　　　　　],
　　　　　　},
　　　　　　{
　　　　　　　　key: "company_id",
　　　　　　　　items: [
　　　　　　　　　　{
　　　　　　　　　　　　value: 13,
　　　　　　　　　　　　count: 4,
　　　　　　　　　　},
　　　　　　　　　　{
　　　　　　　　　　　　value: 10,
　　　　　　　　　　　　count: 1,
　　　　　　　　　　},
　　　　　　　　],
　　　　　　},
　　　　],
　　},
　　errors: [ ],
　　tracer: "",
},

Search for documents that contain "Zhejiang University" and calculate statistics by the group_id field, including the sum of the price field. Documents after the first 10,000 are sampled with a step size of 5:
```
query=default:'Zhejiang University'&&aggregate=group_key:group_id,agg_fun:sum(price), agg_sampler_threshold:10000, agg_sampler_step:5
```
Search for documents that contain "Zhejiang University" and count the documents by the group_id field, with the statistics dimensions being the numbers of documents less than 10, from 10 to 50, and greater than 50:
```
query=default:'Zhejiang University'&&aggregate=group_key:group_id,agg_fun:count(),range:10~50
```
Search for documents that contain "Zhejiang University" and calculate the maximum value of the sum of hits and replies by the group_id field, counting only documents whose create_timestamp is greater than 1423456781:
```
query=default:'Zhejiang University'&&aggregate=group_key:group_id,agg_fun:max(hits+replies),agg_filter:create_timestamp>1423456781
```