AI 助理
文档备案控制台
官方文档
首页 可观测监控 Prometheus 版 实践教程 通过ServiceMonitor创建服务发现

通过ServiceMonitor创建服务发现

更新时间：
复制 MD 格式
产品详情
我的收藏

可观测监控 Prometheus 版支持使用CRD ServiceMonitor的方式来满足您自定义服务发现的采集需求。通过使用ServiceMonitor，您可以自行定义Pod发现的Namespace范围以及通过matchLabel来选择监听的Service。本文将基于SpringBoot框架演示如何通过ServiceMonitor创建服务发现。

Demo

您可以通过下载Demo工程，同步体验通过ServiceMonitor创建服务发现的完整过程。

步骤一：创建基础代码依赖

  1. 创建一个Maven应用，并在pom.xml文件中添加以下依赖。

        <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>io.micrometer</groupId>
            <artifactId>micrometer-registry-prometheus</artifactId>
            <version>1.6.6</version>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-configuration-processor</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>

  2. 在项目的src/resources/applications.properties文件中添加以下配置。

    management.endpoints.web.exposure.include=prometheus

  3. 启动项目，通过浏览器访问http://{host}:{port}/actuator/prometheus。

    返回的 Prometheus 格式指标数据示例如下。

    # HELP jvm_threads_daemon_threads The current number of live daemon threads
# TYPE jvm_threads_daemon_threads gauge
jvm_threads_daemon_threads 23.0
# HELP tomcat_sessions_rejected_sessions_total
# TYPE tomcat_sessions_rejected_sessions_total counter
tomcat_sessions_rejected_sessions_total 0.0
# HELP jvm_memory_committed_bytes The amount of memory in bytes that is committed for the Java virtual machine to use
# TYPE jvm_memory_committed_bytes gauge
jvm_memory_committed_bytes{area="heap",id="PSSurvivorSpace",} 1.31072E7
jvm_memory_committed_bytes{area="heap",id="PSOldGen",} 1.30023424E8
jvm_memory_committed_bytes{area="heap",id="PSEdenSpace",} 1.56762112E8
jvm_memory_committed_bytes{area="nonheap",id="Metaspace",} 3.670016E7
jvm_memory_committed_bytes{area="nonheap",id="CodeCache",} 7143424.0
jvm_memory_committed_bytes{area="nonheap",id="CompressedClassSpace",} 5242880.0
# HELP jvm_classes_loaded_classes The number of classes that are currently loaded in the Java virtual machine
# TYPE jvm_classes_loaded_classes gauge
jvm_classes_loaded_classes 6877.0
# HELP jvm_threads_peak_threads The peak live thread count since the Java virtual machine started or peak was reset
# TYPE jvm_threads_peak_threads gauge
jvm_threads_peak_threads 28.0
# HELP system_cpu_count The number of processors available to the Java virtual machine
# TYPE system_cpu_count gauge
system_cpu_count 12.0
# HELP tomcat_sessions_expired_sessions_total
# TYPE tomcat_sessions_expired_sessions_total counter
tomcat_sessions_expired_sessions_total 0.0
# HELP process_files_max_files The maximum file descriptor count
# TYPE process_files_max_files gauge
process_files_max_files 10240.0
# HELP jvm_buffer_total_capacity_bytes An estimate of the total capacity of the buffers in this pool
# TYPE jvm_buffer_total_capacity_bytes gauge
jvm_buffer_total_capacity_bytes{id="direct",} 8192.0
jvm_buffer_total_capacity_bytes{id="mapped",} 0.0
# HELP jvm_threads_states_threads The current number of threads having NEW state
# TYPE jvm_threads_states_threads gauge
jvm_threads_states_threads{state="runnable",} 9.0
jvm_threads_states_threads{state="blocked",} 0.0
jvm_threads_states_threads{state="waiting",} 12.0
jvm_threads_states_threads{state="timed-waiting",} 6.0
jvm_threads_states_threads{state="new",} 0.0
jvm_threads_states_threads{state="terminated",} 0.0
# HELP jvm_buffer_count_buffers An estimate of the number of buffers in the pool
# TYPE jvm_buffer_count_buffers gauge
jvm_buffer_count_buffers{id="direct",} 1.0
jvm_buffer_count_buffers{id="mapped",} 0.0
# HELP process_files_open_files The open file descriptor count
# TYPE process_files_open_files gauge
process_files_open_files 93.0
# HELP jvm_gc_memory_promoted_bytes_total Count of positive increases in the size of the old generation memory pool before GC to after GC
# TYPE jvm_gc_memory_promoted_bytes_total counter
jvm_gc_memory_promoted_bytes_total 5225384.0
# HELP jvm_memory_max_bytes The maximum amount of memory in bytes that can be used for memory management
# TYPE jvm_memory_max_bytes gauge
jvm_memory_max_bytes{area="heap",id="PSSurvivorSpace",} 1.31072E7
jvm_memory_max_bytes{area="heap",id="PSOldGen",} 2.863661056E9
jvm_memory_max_bytes{area="heap",id="PSEdenSpace",} 1.40509184E9
jvm_memory_max_bytes{area="nonheap",id="Metaspace",} -1.0
jvm_memory_max_bytes{area="nonheap",id="CodeCache",} 2.5165824E8
jvm_memory_max_bytes{area="nonheap",id="CompressedClassSpace",} 1.073741824E9
# HELP jvm_gc_memory_allocated_bytes_total Incremented for an increase in the size of the (young) heap memory pool after one GC to before the next
# TYPE jvm_gc_memory_allocated_bytes_total counter
jvm_gc_memory_allocated_bytes_total 1.14749824E8
# HELP jvm_classes_unloaded_classes_total The total number of classes unloaded since the Java virtual machine has started execution
# TYPE jvm_classes_unloaded_classes_total counter
jvm_classes_unloaded_classes_total 10.0
# HELP tomcat_sessions_created_sessions_total
# TYPE tomcat_sessions_created_sessions_total counter

步骤二：部署Kubernetes集群

  1. 构建一个镜像，并将构建镜像的Dockerfile文件上传至镜像仓库。有关镜像的更多信息，请参见绑定源代码托管平台。

  2. 参考以下内容创建Deployment。

    apiVersion: apps/v1
kind: Deployment
metadata:
  name: micrometer-prometheus
  namespace: default
  labels:
    app: demo-prometheus
spec:
  replicas: 3
  selector:
    matchLabels:
      app: demo-prometheus
  template:
    metadata:
      labels:
        app: demo-prometheus
    spec:
      containers:
        - name: micrometer-prometheus
          image: manjusakalza/micrometer-prometheus:latest
          ports:
            - containerPort: 8080

  3. 参考以下内容创建Service。

    apiVersion: v1
kind: Service
metadata:
  name: prometheus-metrics-demo
  namespace: default
  labels:
    micrometer-prometheus-discovery: 'true'
spec:
  selector:
    app: demo-prometheus
  ports:
    - protocol: TCP
      port: 8080
      targetPort: 8080
      name: metrics

步骤三：创建ServiceMonitor

  1. 将写好的YAML文件保存至本地，并执行kubectl apply -f {YAML文件所在的路径}使YAML文件生效。

ServiceMonitor的YAML文件示例如下：

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: micrometer-demo
  namespace: default
spec:
  endpoints:
    - interval: 15s
      path: /actuator/prometheus
      port: metrics    #注意：这里port配置的是端口名，并非端口号。
  namespaceSelector:
    any: true
  selector:
    matchLabels:
      micrometer-prometheus-discovery: 'true'

在这段YAML文件中，各代码段的含义如下：

  • metadata下的name和namespace将指定ServiceMonitor所需的一些关键元信息。

  • spec的endpoints为服务端点，代表Prometheus所需的采集Metrics的地址。endpoints为一个数组，同时可以创建多个endpoints。每个endpoints包含三个字段，每个字段的含义如下：

    • interval：指定Prometheus对当前endpoints采集的周期。单位为秒，在本次示例中设定为15s。

    • path：指定Prometheus的采集路径。在本次示例中，指定为/actuator/prometheus。

    • port：指定采集数据需要通过的端口，设置的端口为步骤二创建Service时端口所设置的name。在本次示例中，设定为metrics。

      重要

      这里port配置的是端口名，并非端口号。

  • spec的namespaceSelector为需要发现的Service的范围。namespaceSelector包含两个互斥字段，字段的含义如下：

    • any：有且仅有一个值true，当该字段被设置时，将监听所有符合Selector过滤条件的Service的变动。

    • matchNames：数组值，指定需要监听的namespace的范围。例如，只想监听default和arms-prom两个命名空间中的Service，那么matchNames设置如下：

      namespaceSelector:
  matchNames:
  - default
  - arms-prom

  • spec的selector用于选择Service。

    在本次示例所使用的Service有micrometer-prometheus-discovery: 'true' Label，所以selector设置如下：

    selector:
  matchLabels:
    micrometer-prometheus-discovery: 'true'

如需使用 basic auth 功能，请参考如下YAML文件。

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: cloud-open-api-monitor     #  设置 ServiceMonitor 唯一的名字
  namespace: default   #  设置当前 ServiceMonitor 所在的命名空间
spec:
  endpoints:
  - interval: 30s
    # 指定 Prometheus 对当前 endpoints 采集的周期
    port: tcp-8080
    # 填写 Prometheus exporter代码中暴露的地址
    path: /api/actuator/prometheus
    basicAuth:
      password:
        name: basic-auth
        key: <password>
      username:
        name: basic-auth
        key: <userName>
    scheme: http
  namespaceSelector:
    any: true
  selector:
    matchLabels:
      # 匹配具有如下标签的 Service
      edas.oam.acname: cloud-open-api

如果使用 basic auth 功能时没有权限， 需要在集群里添加一个有对应权限的 ClusterRole，然后将此 ClusterRole 通过 ClusterRoleBinding 绑定到 arms-prom 命名空间下名字为 arms-prom-operator 的 ServiceAccount 上，这样可以让 Prometheus Agent 具有对应权限。

  • ClusterRole YAML 文件

    apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: prometheus-agent-role
  labels:
    app: prometheus-agent
rules:
  - apiGroups: [""]
    resources: ["pods", "services", "endpoints", "nodes"]
    verbs: ["get", "list", "watch"]  #根据具体需求调整
  - apiGroups: ["monitoring.coreos.com"]  # 根据需要调整apiGroups
    resources: ["*"]
    verbs: ["get", "list", "watch", "create", "update", "delete"]  # 根据需要调整

  • ClusterRoleBinding YAML 文件

    apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: prometheus-agent-binding
subjects:
  - kind: ServiceAccount
    name: arms-prom-operator  # 服务账户名称
    namespace: arms-prom      # 服务账户所在的命名空间
roleRef:
  kind: ClusterRole
  name: prometheus-agent-role  # 上面定义的 ClusterRole 名称
  apiGroup: rbac.authorization.k8s.io

步骤四：验证ServiceMonitor

通过以下操作，验证Prometheus是否成功进行服务发现。

  1. 登录Prometheus控制台，在左侧导航栏单击接入管理。

  2. 在已接入环境页签，查看容器环境列表，单击目标容器环境名称。

  3. 单击自监控页签，然后单击Targets页签。

    在Targets页签，查看是否存在名称为{namespace}/{serviceMonitorName}/x的Target。

    页面中显示目标组 default/micrometer-demo/0 (3/3 up)，表示 ServiceMonitor 已被 Prometheus 正确识别，所有目标的 State 均为绿色 UP，验证通过。

  4. 单击{namespace}/{serviceMonitorName}/x所在行展开Target，然后单击Endpoint链接。

    页面显示 Prometheus 格式的 JVM 与 Tomcat 运行时指标数据，表明 ServiceMonitor 已正确配置。

    # HELP jvm_buffer_total_capacity_bytes An estimate of the total capacity of the buffers in this pool
# TYPE jvm_buffer_total_capacity_bytes gauge
jvm_buffer_total_capacity_bytes{id="mapped",} 0.0
jvm_buffer_total_capacity_bytes{id="direct",} 81968.0
# HELP jvm_threads_daemon_threads The current number of live daemon threads
# TYPE jvm_threads_daemon_threads gauge
jvm_threads_daemon_threads 29.0
# HELP jvm_memory_committed_bytes The amount of memory in bytes that is committed for the Java virtual machine to use
# TYPE jvm_memory_committed_bytes gauge
jvm_memory_committed_bytes{area="nonheap",id="miscellaneousnon-heapstorage",} 2.3068672E7
jvm_memory_committed_bytes{area="nonheap",id="classstorage",} 2.8471712E7
jvm_memory_committed_bytes{area="nonheap",id="JITcodecache",} 2.68435456E8
jvm_memory_committed_bytes{area="heap",id="tenured-LOA",} 671744.0
jvm_memory_committed_bytes{area="nonheap",id="JITdatacache",} 2097152.0
jvm_memory_committed_bytes{area="heap",id="nursery-survivor",} 1638400.0
jvm_memory_committed_bytes{area="heap",id="nursery-allocate",} 6881280.0
jvm_memory_committed_bytes{area="heap",id="tenured-SOA",} 1.2763136E7
# HELP process_start_time_seconds Start time of the process since the unix epoch.
# TYPE process_start_time_seconds gauge
process_start_time_seconds 1.622297808811E9
# HELP tomcat_sessions_expired_sessions_total
# TYPE tomcat_sessions_expired_sessions_total counter
tomcat_sessions_expired_sessions_total 0.0
# HELP jvm_threads_states_threads The current number of threads having NEW state
# TYPE jvm_threads_states_threads gauge
jvm_threads_states_threads{state="runnable",} 20.0
jvm_threads_states_threads{state="blocked",} 0.0
jvm_threads_states_threads{state="waiting",} 10.0
jvm_threads_states_threads{state="timed-waiting",} 3.0
jvm_threads_states_threads{state="new",} 0.0
jvm_threads_states_threads{state="terminated",} 0.0
# HELP jvm_classes_unloaded_classes_total The total number of classes unloaded since the Java virtual machine has started execution
# TYPE jvm_classes_unloaded_classes_total counter
jvm_classes_unloaded_classes_total 0.0
# HELP tomcat_sessions_rejected_sessions_total
# TYPE tomcat_sessions_rejected_sessions_total counter
tomcat_sessions_rejected_sessions_total 0.0
# HELP tomcat_sessions_created_sessions_total
# TYPE tomcat_sessions_created_sessions_total counter
tomcat_sessions_created_sessions_total 0.0
# HELP jvm_threads_live_threads The current number of live threads including both daemon and non-daemon threads
# TYPE jvm_threads_live_threads gauge
jvm_threads_live_threads 33.0
# HELP jvm_memory_max_bytes The maximum amount of memory in bytes that can be used for memory management
# TYPE jvm_memory_max_bytes gauge

上一篇：将Prometheus的监控数据接入本地Grafana下一篇：使用智能检测算子发现异常数据