阿里云Elasticsearch团队自研的TimeStream时序引擎增强插件,支持使用PromQL查询Elasticsearch数据,无缝对接Prometheus+Grafana。本文介绍如何基于Elasticsearch TimeStream使用Prometheus集成接口。
背景信息
TimeStream是阿里云Elasticsearch团队自研,并结合Elastic社区时序类产品特性共建的时序引擎。阿里云Elasticsearch提供的TimeStream时序增强功能插件,优化了Elasticsearch在存储指标数据方面的DSL(Domain-Specific Language)查询复杂且慢以及存储成本过高等问题。详细信息请参见TimeStream时序增强引擎介绍。
前提条件
已创建阿里云Elasticsearch实例,且实例版本为7.10(内核版本为1.8.0及以上)或7.16及以上(内核版本为1.7.0及以上)。具体操作请参见创建阿里云Elasticsearch实例。
Prometheus API说明
Prometheus API除了可以访问通过Prometheus Remote Write方式写入的数据,也可以访问任意TimeStream索引。
Prometheus API支持去掉返回的指标和维度的前后缀关键字。如果使用默认的数据模型,指标前缀metric.会被去掉,维度前缀label.会被去掉。
{
"labels": {
"instance": "127.0.0.1:9114",
"job": "elasticsearch"
},
"metrics": {
"up": 0.0,
"scrape_samples_post_metric_relabeling": 0.0,
"scrape_samples_scraped": 0.0,
"scrape_duration_seconds": 4.45999E-4,
"scrape_series_added": 0.0
},
"@timestamp": "1655717638795"
}{
"status" : "success",
"data" : [
"__name__",
"instance",
"job"
]
}
{
"status" : "success",
"data" : {
"scrape_samples_post_metric_relabeling" : [
{
"type" : "gauge",
"help" : "",
"unit" : ""
}
],
"scrape_samples_scraped" : [
{
"type" : "gauge",
"help" : "",
"unit" : ""
}
],
"scrape_duration_seconds" : [
{
"type" : "gauge",
"help" : "",
"unit" : ""
}
],
"scrape_series_added" : [
{
"type" : "gauge",
"help" : "",
"unit" : ""
}
],
"up" : [
{
"type" : "gauge",
"help" : "",
"unit" : ""
}
]
}
}PUT _time_stream/{name}
{
"time_stream": {
"labels_fields": "@labels.*_l",
"metrics_fields": "@metrics.*_m",
"label_prefix": "@labels.",
"label_suffix": "_l",
"metric_prefix": "@metrics.",
"metric_suffix": "_m"
}
}RemoteWrite接口
API
POST /_time_stream/prom_write/{index}
---PB data---使用说明
remote_write:
- url: "http://127.0.0.1:9200/_time_stream/prom_write/prom_index"
basic_auth:
username: elastic
password: xxxxPrometheus通过Remote Write写入数据到TimeStream索引,只支持默认的时序数据模型,即维度前缀是label.,指标前缀是metric.。
Instant Query接口
API
GET /_time_stream/prom/{index}/query
POST /_time_stream/prom/{index}/query使用说明
Prometheus Instant Query接口,详细信息请参见Prometheus instant queries。
Range Query接口
API
GET /_time_stream/prom/{index}/query_range
POST /_time_stream/prom/{index}/query_range使用说明
Prometheus Range Query接口,详细信息请参见Prometheus range queries。
Series Metadata接口
API
GET /_time_stream/prom/{index}/series
POST /_time_stream/prom/{index}/series使用说明
Prometheus查询时间线的接口,详细信息请参见Prometheus finding series by label matchers。
Labels Metadata接口
API
GET /_time_stream/prom/{index}/labels
POST /_time_stream/prom/{index}/labels使用说明
Prometheus查询维度(label)列表的接口,详细信息请参见Prometheus getting label names。
Label Values Metadata接口
API
GET /_time_stream/prom/{index}/label/<label_name>/values使用说明
Prometheus查询维度对应values列表的接口,详细信息请参见Prometheus querying label values。
Metric Metadata接口
API
GET /_time_stream/prom/{index}/metadata使用说明
Prometheus查询指标(Metrics)列表的接口,详细信息请参见Prometheus querying metric metadata。
TimeStream PromQL支持情况
表达式支持情况
| 表达式 | 类型 | 是否支持 | 说明 |
| powOp | binaryOperator | 是 | 幂乘 |
| unaryOp | binaryOperator | 是 | 正、负 |
| multOp | binaryOperator | 是 | 乘、除、取模 |
| addOp | binaryOperator | 是 | 加、减 |
| compareOp | binaryOperator | 是 | 比较(==,>=, >, <=, <, !=) |
| andUnlessOp | binaryOperator | 是 | 集合运算(and、or、unless) |
| orOp | binaryOperator | 是 | 集合运算(or) |
| function | vector | 是 | 函数 |
| aggregation | vector | 是 | 聚合运算 |
| instantSelector | vector | 是 | 瞬时选择器 |
| matrixSelector | vector | 是 | 向量选择器 |
| offset | vector | 是 | 偏移量 |
| literal | vector | 是 | 常量 |
| labelMatcher | vector | 是 | 标签匹配 |
| grouping | group | 否 | group聚合,包括on、ignoring、group_left、group_right |
Aggregation operator支持情况
| Aggregation operator | 是否支持 |
| sum(calculate sum over dimensions) | 是 |
| min(select minimum over dimensions) | 是 |
| max(select maximum over dimensions) | 是 |
| avg(calculate the average over dimensions) | 是 |
| group(all values in the resulting vector are 1) | 否 |
| stddev(calculate population standard deviation over dimensions) | 否 |
| stdvar(calculate population standard variance over dimensions) | 否 |
| count(count number of elements in the vector) | 是 |
| count_values(count number of elements with the same value) | 是 |
| bottomk(smallest k elements by sample value) | 是 |
| topk(largest k elements by sample value) | 是 |
| quantile(calculate φ-quantile (0 ≤ φ ≤ 1) over dimensions) | 是 |
Function支持情况
| Function | 是否支持 |
| abs(v instant-vector) | 是 |
| absent(v instant-vector) | 否 |
| absent_over_time(v range-vector) | 否 |
| ceil(v instant-vector) | 是 |
| changes(v range-vector) | 否 |
| clamp(v instant-vector, min scalar, max scalar) | 是 |
| clamp_max(v instant-vector, max scalar) | 是 |
| clamp_min(v instant-vector, min scalar) | 是 |
| day_of_month(v=vector(time()) instant-vector) | 是 |
| day_of_week(v=vector(time()) instant-vector) | 是 |
| days_in_month(v=vector(time()) instant-vector) | 否 |
| delta(v range-vector) | 是 |
| deriv(v range-vector) | 否 |
| exp(v instant-vector) | 是 |
| floor(v instant-vector) | 是 |
| histogram_quantile(φ scalar, b instant-vector) | 否 |
| holt_winters(v range-vector, sf scalar, tf scalar) | 否 |
| hour(v=vector(time()) instant-vector) | 是 |
| idelta(v range-vector) | 是 |
| increase(v range-vector) | 是 |
| irate(v range-vector) | 是 |
| label_join(v instant-vector, dst_label string, separator string, src_label_1 string, src_label_2 string, ...) | 否 |
| label_replace(v instant-vector, dst_label string, replacement string, src_label string, regex string) | 否 |
| ln(v instant-vector) | 是 |
| log2(v instant-vector) | 否 |
| log10(v instant-vector) | 是 |
| minute(v=vector(time()) instant-vector) | 是 |
| month(v=vector(time()) instant-vector) | 是 |
| predict_linear(v range-vector, t scalar) | 否 |
| rate(v range-vector) | 是 |
| resets(v range-vector) | 否 |
| round(v instant-vector, to_nearest=1 scalar) | 否 |
| scalar(v instant-vector) | 否 |
| sgn(v instant-vector) | 是 |
| sort(v instant-vector) | 否 |
| sort_desc() | 否 |
| sqrt(v instant-vector) | 是 |
| time() | 否 |
| timestamp(v instant-vector) | 是 |
| vector(s scalar) | 否 |
| year(v=vector(time()) instant-vector) | 是 |
| avg_over_time(range-vector) | 是 |
| min_over_time(range-vector) | 是 |
| max_over_time(range-vector) | 是 |
| sum_over_time(range-vector) | 是 |
| count_over_time(range-vector) | 是 |
| quantile_over_time(scalar, range-vector) | 是 |
| stddev_over_time(range-vector) | 否 |
| stdvar_over_time(range-vector) | 否 |
| last_over_time(range-vector) | 是 |
| present_over_time(range-vector) | 否 |
| acos(v instant-vector) | 是 |
| acosh(v instant-vector) | 是 |
| asin(v instant-vector) | 是 |
| asinh(v instant-vector) | 否 |
| atan(v instant-vector) | 是 |
| atanh(v instant-vector) | 否 |
| cos(v instant-vector) | 是 |
| cosh(v instant-vector) | 是 |
| sin(v instant-vector) | 是 |
| sinh(v instant-vector) | 是 |
| tan(v instant-vector) | 是 |
| tanh(v instant-vector) | 是 |
| deg(v instant-vector) | 是 |
| pi() | 是 |
| rad(v instant-vector) | 是 |