如果您希望根据Pod实际的资源使用情况,动态调整和优化资源配置,以确保Pod能够调度到资源充足的节点上,建议您为业务容器开启垂直伸缩VPA(Vertical Pod Autoscaler)功能。VPA能够监控Pod的资源消耗模式,灵活推荐CPU和内存资源分配的配置,并在适当的情况下自动进行调整,而不调整Pod的副本数量。这种能力更适用于需要稳定资源配置的有状态应用的扩容等场景。
阅读前提示
为了帮助您更好地使用VPA功能,建议您在阅读本文前参见Kubernetes社区VPA介绍了解VPA的基本信息、使用流程、已知限制等。
此外,ACK集群提供了多种工作负载伸缩(调度层弹性)和节点伸缩(资源层弹性)方案,建议您在使用本文档前阅读弹性伸缩概述,了解不同方案的适用场景、使用限制等。
ack-vertical-pod-autoscaler组件介绍
VPA基于ack-vertical-pod-autoscaler组件实现,以根据实际资源需求推荐并自动调整Pod中容器的CPU及内存的Request和Limit参数。同时,VPA也会遵循容器初始配置中定义的资源Request和Limit占比。ack-vertical-pod-autoscaler组件有以下组成部分:
Admission Controller组件:为新建Pod设置正确的资源请求值Requests。安装admission-controller组件前,您需要用此脚本为Webhook生成证书。
Recommender组件:监控容器目前和过去的资源使用情况,并基于检测数据推荐资源配置,以更准确地匹配资源使用需求。
Updater组件:检查VPA管理的Pod资源量配置是否正确。如果不正确,Updater会终止这些Pod,以便后续重建。
注意事项
VPA功能目前处于Beta阶段,其性能未在大型集群中测试,请谨慎使用。如遇相关问题或有相关产品建议,请提交工单联系容器服务团队。
更新正在运行的Pod资源配置会导致Pod的重建和重启,且有可能会被调度到其他节点上。VPA动态更新现有Pod的资源Request时,支持无需重启的更新机制,但此机制仍处于测试阶段。
VPA不会驱逐没有在副本控制器(Replication Controller)管理下的Pod。在VPA中,此类Pod的Auto模式等同于Initial模式,即VPA会为新的Pod设置资源请求和限制,但不会自动删除和重新创建那些资源分配不当的运行中的Pod。
不推荐您为同一个工作负载部署多个VPA。多个VPA同时匹配同一个Pod时,可能会造成不可预测的行为。
VPA对Pod资源Request的修改值可能超过实际的资源上限,例如节点资源上限、空闲资源或资源配额,从而造成Pod处于Pending状态无法被调度。此时,使用节点自动伸缩可能可以解决这个问题:当Pod的资源使用率超出扩容阈值后,Pod将扩容,但此时节点资源不足,导致Pod调度失败并处于Pending状态,继而触发节点扩容。更多信息,请参见启用节点自动伸缩。
目前,VPA和HPA的Controller无法完全兼容。如果您同时使用VPA和HPA来监控CPU和内存使用情况,这可能会产生冲突。如果您的HPA仅监控其他定制化的或者外部的资源使用情况,可以避免此冲突。
VPA使用Admission Webhook作为其准入控制器。请确保集群中存在的其他Admission Webhook不会与VPA发生冲突。
说明准入控制器的执行顺序定义可在API Server的配置参数中查找。
VPA会处理出现的绝大多数OOM(Out Of Memory)的事件,但无法保证所有的场景下都有效。
关于VPA更多的已知限制,请参见VPA已知限制。
步骤一:安装ack-vertical-pod-autoscaler组件
您可以通过控制台(v1.26及以上集群)或kubectl(v1.26以下集群)的方式安装ack-vertical-pod-autoscaler组件。
通过控制台安装(推荐)
推荐您通过容器服务管理控制台安装ack-vertical-pod-autoscaler组件,屏蔽组件底层复杂性,使用门槛和维护成本更低。
前提条件
已创建v1.26及以上的ACK托管集群。具体操作,请参见创建Kubernetes托管版集群。如需升级集群,请参见手动升级集群。
已使用命令行工具连接集群,请参见获取集群KubeConfig并通过kubectl工具连接集群。
如果您之前通过kubectl的方式安装了VPA,建议您卸载该VPA,通过控制台重新安装。具体操作,请参见下文如何通过控制台管理使用kubectl安装的VPA?。
安装步骤
登录容器服务管理控制台,在左侧导航栏选择集群。
在集群列表页面,单击目标集群名称,然后在左侧导航栏,选择运维管理 > 组件管理。
在组件管理页面,定位ack-vertical-pod-autoscaler组件,然后按照页面提示完成安装。
通过kubectl安装
前提条件
已创建v1.26以下的ACK托管集群。具体操作,请参见创建Kubernetes托管版集群。
已使用命令行工具连接集群,请参见获取集群KubeConfig并通过kubectl工具连接集群。
如果您已在集群中部署VPA,请卸载该VPA,以避免新安装的VPA与旧版VPA冲突。
操作步骤
保存RBAC权限的YAML文件,并执行命令,创建RBAC权限文件。
展开查看RBAC YAML文件
apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: system:metrics-reader rules: - apiGroups: - "metrics.k8s.io" resources: - pods verbs: - get - list --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: system:vpa-actor rules: - apiGroups: - "" resources: - pods - nodes - limitranges verbs: - get - list - watch - apiGroups: - "" resources: - events verbs: - get - list - watch - create - apiGroups: - "poc.autoscaling.k8s.io" resources: - verticalpodautoscalers verbs: - get - list - watch - apiGroups: - "autoscaling.k8s.io" resources: - verticalpodautoscalers verbs: - get - list - watch --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: system:vpa-status-actor rules: - apiGroups: - "autoscaling.k8s.io" resources: - verticalpodautoscalers/status verbs: - get - patch --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: system:vpa-checkpoint-actor rules: - apiGroups: - "poc.autoscaling.k8s.io" resources: - verticalpodautoscalercheckpoints verbs: - get - list - watch - create - patch - delete - apiGroups: - "autoscaling.k8s.io" resources: - verticalpodautoscalercheckpoints verbs: - get - list - watch - create - patch - delete - apiGroups: - "" resources: - namespaces verbs: - get - list --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: system:evictioner rules: - apiGroups: - "apps" - "extensions" resources: - replicasets verbs: - get - apiGroups: - "" resources: - pods/eviction verbs: - create --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: system:metrics-reader roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:metrics-reader subjects: - kind: ServiceAccount name: vpa-recommender namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: system:vpa-actor roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:vpa-actor subjects: - kind: ServiceAccount name: vpa-recommender namespace: kube-system - kind: ServiceAccount name: vpa-updater namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: system:vpa-status-actor roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:vpa-status-actor subjects: - kind: ServiceAccount name: vpa-recommender namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: system:vpa-checkpoint-actor roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:vpa-checkpoint-actor subjects: - kind: ServiceAccount name: vpa-recommender namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: system:vpa-target-reader rules: - apiGroups: - '*' resources: - '*/scale' verbs: - get - watch - apiGroups: - "" resources: - replicationcontrollers verbs: - get - list - watch - apiGroups: - apps resources: - daemonsets - deployments - replicasets - statefulsets verbs: - get - list - watch - apiGroups: - batch resources: - jobs - cronjobs verbs: - get - list - watch --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: system:vpa-target-reader-binding roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:vpa-target-reader subjects: - kind: ServiceAccount name: vpa-recommender namespace: kube-system - kind: ServiceAccount name: vpa-admission-controller namespace: kube-system - kind: ServiceAccount name: vpa-updater namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: system:vpa-evictioner-binding roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:evictioner subjects: - kind: ServiceAccount name: vpa-updater namespace: kube-system --- apiVersion: v1 kind: ServiceAccount metadata: name: vpa-admission-controller namespace: kube-system --- apiVersion: v1 kind: ServiceAccount metadata: name: vpa-recommender namespace: kube-system --- apiVersion: v1 kind: ServiceAccount metadata: name: vpa-updater namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: system:vpa-admission-controller rules: - apiGroups: - "" resources: - pods - configmaps - nodes - limitranges verbs: - get - list - watch - apiGroups: - "admissionregistration.k8s.io" resources: - mutatingwebhookconfigurations verbs: - create - delete - get - list - apiGroups: - "poc.autoscaling.k8s.io" resources: - verticalpodautoscalers verbs: - get - list - watch - apiGroups: - "autoscaling.k8s.io" resources: - verticalpodautoscalers verbs: - get - list - watch - apiGroups: - "coordination.k8s.io" resources: - leases verbs: - create - update - get - list - watch --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: system:vpa-admission-controller roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:vpa-admission-controller subjects: - kind: ServiceAccount name: vpa-admission-controller namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: system:vpa-status-reader rules: - apiGroups: - "coordination.k8s.io" resources: - leases verbs: - get - list - watch --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: system:vpa-status-reader-binding roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:vpa-status-reader subjects: - kind: ServiceAccount name: vpa-updater namespace: kube-system
kubectl apply -f rbac.yaml
保存CRD的YAML文件,并执行命令,创建ack-vertical-pod-autoscaler组件的CRD。
说明CRD可以提高Kubernetes的扩展能力,详情请参见Extend the Kubernetes API with CustomResourceDefinitions。
CRD YAML定义如下:
1.22≤ 集群版本<1.26
展开查看CRD YAML
apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: api-approved.kubernetes.io: https://github.com/kubernetes/kubernetes/pull/63797 controller-gen.kubebuilder.io/version: v0.9.2 creationTimestamp: null name: verticalpodautoscalercheckpoints.autoscaling.k8s.io spec: group: autoscaling.k8s.io names: kind: VerticalPodAutoscalerCheckpoint listKind: VerticalPodAutoscalerCheckpointList plural: verticalpodautoscalercheckpoints shortNames: - vpacheckpoint singular: verticalpodautoscalercheckpoint scope: Namespaced versions: - name: v1 schema: openAPIV3Schema: description: VerticalPodAutoscalerCheckpoint is the checkpoint of the internal state of VPA that is used for recovery after recommender's restart. properties: apiVersion: description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: description: 'Specification of the checkpoint. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status.' properties: containerName: description: Name of the checkpointed container. type: string vpaObjectName: description: Name of the VPA object that stored VerticalPodAutoscalerCheckpoint object. type: string type: object status: description: Data of the checkpoint. properties: cpuHistogram: description: Checkpoint of histogram for consumption of CPU. properties: bucketWeights: description: Map from bucket index to bucket weight. type: object x-kubernetes-preserve-unknown-fields: true referenceTimestamp: description: Reference timestamp for samples collected within this histogram. format: date-time nullable: true type: string totalWeight: description: Sum of samples to be used as denominator for weights from BucketWeights. type: number type: object firstSampleStart: description: Timestamp of the fist sample from the histograms. format: date-time nullable: true type: string lastSampleStart: description: Timestamp of the last sample from the histograms. format: date-time nullable: true type: string lastUpdateTime: description: The time when the status was last refreshed. format: date-time nullable: true type: string memoryHistogram: description: Checkpoint of histogram for consumption of memory. properties: bucketWeights: description: Map from bucket index to bucket weight. type: object x-kubernetes-preserve-unknown-fields: true referenceTimestamp: description: Reference timestamp for samples collected within this histogram. format: date-time nullable: true type: string totalWeight: description: Sum of samples to be used as denominator for weights from BucketWeights. type: number type: object totalSamplesCount: description: Total number of samples in the histograms. type: integer version: description: Version of the format of the stored data. type: string type: object type: object served: true storage: true - name: v1beta2 schema: openAPIV3Schema: description: VerticalPodAutoscalerCheckpoint is the checkpoint of the internal state of VPA that is used for recovery after recommender's restart. properties: apiVersion: description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: description: 'Specification of the checkpoint. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status.' properties: containerName: description: Name of the checkpointed container. type: string vpaObjectName: description: Name of the VPA object that stored VerticalPodAutoscalerCheckpoint object. type: string type: object status: description: Data of the checkpoint. properties: cpuHistogram: description: Checkpoint of histogram for consumption of CPU. properties: bucketWeights: description: Map from bucket index to bucket weight. type: object x-kubernetes-preserve-unknown-fields: true referenceTimestamp: description: Reference timestamp for samples collected within this histogram. format: date-time nullable: true type: string totalWeight: description: Sum of samples to be used as denominator for weights from BucketWeights. type: number type: object firstSampleStart: description: Timestamp of the fist sample from the histograms. format: date-time nullable: true type: string lastSampleStart: description: Timestamp of the last sample from the histograms. format: date-time nullable: true type: string lastUpdateTime: description: The time when the status was last refreshed. format: date-time nullable: true type: string memoryHistogram: description: Checkpoint of histogram for consumption of memory. properties: bucketWeights: description: Map from bucket index to bucket weight. type: object x-kubernetes-preserve-unknown-fields: true referenceTimestamp: description: Reference timestamp for samples collected within this histogram. format: date-time nullable: true type: string totalWeight: description: Sum of samples to be used as denominator for weights from BucketWeights. type: number type: object totalSamplesCount: description: Total number of samples in the histograms. type: integer version: description: Version of the format of the stored data. type: string type: object type: object served: true storage: false --- apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: annotations: api-approved.kubernetes.io: https://github.com/kubernetes/kubernetes/pull/63797 controller-gen.kubebuilder.io/version: v0.9.2 creationTimestamp: null name: verticalpodautoscalers.autoscaling.k8s.io spec: group: autoscaling.k8s.io names: kind: VerticalPodAutoscaler listKind: VerticalPodAutoscalerList plural: verticalpodautoscalers shortNames: - vpa singular: verticalpodautoscaler scope: Namespaced versions: - additionalPrinterColumns: - jsonPath: .spec.updatePolicy.updateMode name: Mode type: string - jsonPath: .status.recommendation.containerRecommendations[0].target.cpu name: CPU type: string - jsonPath: .status.recommendation.containerRecommendations[0].target.memory name: Mem type: string - jsonPath: .status.conditions[?(@.type=='RecommendationProvided')].status name: Provided type: string - jsonPath: .metadata.creationTimestamp name: Age type: date name: v1 schema: openAPIV3Schema: description: VerticalPodAutoscaler is the configuration for a vertical pod autoscaler, which automatically manages pod resources based on historical and real time resource utilization. properties: apiVersion: description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: description: 'Specification of the behavior of the autoscaler. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status.' properties: recommenders: description: Recommender responsible for generating recommendation for this object. List should be empty (then the default recommender will generate the recommendation) or contain exactly one recommender. items: description: VerticalPodAutoscalerRecommenderSelector points to a specific Vertical Pod Autoscaler recommender. In the future it might pass parameters to the recommender. properties: name: description: Name of the recommender responsible for generating recommendation for this object. type: string required: - name type: object type: array resourcePolicy: description: Controls how the autoscaler computes recommended resources. The resource policy may be used to set constraints on the recommendations for individual containers. If not specified, the autoscaler computes recommended resources for all containers in the pod, without additional constraints. properties: containerPolicies: description: Per-container resource policies. items: description: ContainerResourcePolicy controls how autoscaler computes the recommended resources for a specific container. properties: containerName: description: Name of the container or DefaultContainerResourcePolicy, in which case the policy is used by the containers that don't have their own policy specified. type: string controlledResources: description: Specifies the type of recommendations that will be computed (and possibly applied) by VPA. If not specified, the default of [ResourceCPU, ResourceMemory] will be used. items: description: ResourceName is the name identifying various resources in a ResourceList. type: string type: array controlledValues: description: Specifies which resource values should be controlled. The default is "RequestsAndLimits". enum: - RequestsAndLimits - RequestsOnly type: string maxAllowed: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true description: Specifies the maximum amount of resources that will be recommended for the container. The default is no maximum. type: object minAllowed: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true description: Specifies the minimal amount of resources that will be recommended for the container. The default is no minimum. type: object mode: description: Whether autoscaler is enabled for the container. The default is "Auto". enum: - Auto - "Off" type: string type: object type: array type: object targetRef: description: TargetRef points to the controller managing the set of pods for the autoscaler to control - e.g. Deployment, StatefulSet. VerticalPodAutoscaler can be targeted at controller implementing scale subresource (the pod set is retrieved from the controller's ScaleStatus) or some well known controllers (e.g. for DaemonSet the pod set is read from the controller's spec). If VerticalPodAutoscaler cannot use specified target it will report ConfigUnsupported condition. Note that VerticalPodAutoscaler does not require full implementation of scale subresource - it will not use it to modify the replica count. The only thing retrieved is a label selector matching pods grouped by the target resource. properties: apiVersion: description: API version of the referent type: string kind: description: 'Kind of the referent; More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds"' type: string name: description: 'Name of the referent; More info: http://kubernetes.io/docs/user-guide/identifiers#names' type: string required: - kind - name type: object x-kubernetes-map-type: atomic updatePolicy: description: Describes the rules on how changes are applied to the pods. If not specified, all fields in the `PodUpdatePolicy` are set to their default values. properties: minReplicas: description: Minimal number of replicas which need to be alive for Updater to attempt pod eviction (pending other checks like PDB). Only positive values are allowed. Overrides global '--min-replicas' flag. format: int32 type: integer updateMode: description: Controls when autoscaler applies changes to the pod resources. The default is 'Auto'. enum: - "Off" - Initial - Recreate - Auto type: string type: object required: - targetRef type: object status: description: Current information about the autoscaler. properties: conditions: description: Conditions is the set of conditions required for this autoscaler to scale its target, and indicates whether or not those conditions are met. items: description: VerticalPodAutoscalerCondition describes the state of a VerticalPodAutoscaler at a certain point. properties: lastTransitionTime: description: lastTransitionTime is the last time the condition transitioned from one status to another format: date-time type: string message: description: message is a human-readable explanation containing details about the transition type: string reason: description: reason is the reason for the condition's last transition. type: string status: description: status is the status of the condition (True, False, Unknown) type: string type: description: type describes the current condition type: string required: - status - type type: object type: array recommendation: description: The most recently computed amount of resources recommended by the autoscaler for the controlled pods. properties: containerRecommendations: description: Resources recommended by the autoscaler for each container. items: description: RecommendedContainerResources is the recommendation of resources computed by autoscaler for a specific container. Respects the container resource policy if present in the spec. In particular the recommendation is not produced for containers with `ContainerScalingMode` set to 'Off'. properties: containerName: description: Name of the container. type: string lowerBound: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true description: Minimum recommended amount of resources. Observes ContainerResourcePolicy. This amount is not guaranteed to be sufficient for the application to operate in a stable way, however running with less resources is likely to have significant impact on performance/availability. type: object target: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true description: Recommended amount of resources. Observes ContainerResourcePolicy. type: object uncappedTarget: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true description: The most recent recommended resources target computed by the autoscaler for the controlled pods, based only on actual resource usage, not taking into account the ContainerResourcePolicy. May differ from the Recommendation if the actual resource usage causes the target to violate the ContainerResourcePolicy (lower than MinAllowed or higher that MaxAllowed). Used only as status indication, will not affect actual resource assignment. type: object upperBound: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true description: Maximum recommended amount of resources. Observes ContainerResourcePolicy. Any resources allocated beyond this value are likely wasted. This value may be larger than the maximum amount of application is actually capable of consuming. type: object required: - target type: object type: array type: object type: object required: - spec type: object served: true storage: true subresources: {} - deprecated: true deprecationWarning: autoscaling.k8s.io/v1beta2 API is deprecated name: v1beta2 schema: openAPIV3Schema: description: VerticalPodAutoscaler is the configuration for a vertical pod autoscaler, which automatically manages pod resources based on historical and real time resource utilization. properties: apiVersion: description: 'APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' type: string kind: description: 'Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' type: string metadata: type: object spec: description: 'Specification of the behavior of the autoscaler. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status.' properties: resourcePolicy: description: Controls how the autoscaler computes recommended resources. The resource policy may be used to set constraints on the recommendations for individual containers. If not specified, the autoscaler computes recommended resources for all containers in the pod, without additional constraints. properties: containerPolicies: description: Per-container resource policies. items: description: ContainerResourcePolicy controls how autoscaler computes the recommended resources for a specific container. properties: containerName: description: Name of the container or DefaultContainerResourcePolicy, in which case the policy is used by the containers that don't have their own policy specified. type: string maxAllowed: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true description: Specifies the maximum amount of resources that will be recommended for the container. The default is no maximum. type: object minAllowed: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true description: Specifies the minimal amount of resources that will be recommended for the container. The default is no minimum. type: object mode: description: Whether autoscaler is enabled for the container. The default is "Auto". enum: - Auto - "Off" type: string type: object type: array type: object targetRef: description: TargetRef points to the controller managing the set of pods for the autoscaler to control - e.g. Deployment, StatefulSet. VerticalPodAutoscaler can be targeted at controller implementing scale subresource (the pod set is retrieved from the controller's ScaleStatus) or some well known controllers (e.g. for DaemonSet the pod set is read from the controller's spec). If VerticalPodAutoscaler cannot use specified target it will report ConfigUnsupported condition. Note that VerticalPodAutoscaler does not require full implementation of scale subresource - it will not use it to modify the replica count. The only thing retrieved is a label selector matching pods grouped by the target resource. properties: apiVersion: description: API version of the referent type: string kind: description: 'Kind of the referent; More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds"' type: string name: description: 'Name of the referent; More info: http://kubernetes.io/docs/user-guide/identifiers#names' type: string required: - kind - name type: object x-kubernetes-map-type: atomic updatePolicy: description: Describes the rules on how changes are applied to the pods. If not specified, all fields in the `PodUpdatePolicy` are set to their default values. properties: updateMode: description: Controls when autoscaler applies changes to the pod resources. The default is 'Auto'. enum: - "Off" - Initial - Recreate - Auto type: string type: object required: - targetRef type: object status: description: Current information about the autoscaler. properties: conditions: description: Conditions is the set of conditions required for this autoscaler to scale its target, and indicates whether or not those conditions are met. items: description: VerticalPodAutoscalerCondition describes the state of a VerticalPodAutoscaler at a certain point. properties: lastTransitionTime: description: lastTransitionTime is the last time the condition transitioned from one status to another format: date-time type: string message: description: message is a human-readable explanation containing details about the transition type: string reason: description: reason is the reason for the condition's last transition. type: string status: description: status is the status of the condition (True, False, Unknown) type: string type: description: type describes the current condition type: string required: - status - type type: object type: array recommendation: description: The most recently computed amount of resources recommended by the autoscaler for the controlled pods. properties: containerRecommendations: description: Resources recommended by the autoscaler for each container. items: description: RecommendedContainerResources is the recommendation of resources computed by autoscaler for a specific container. Respects the container resource policy if present in the spec. In particular the recommendation is not produced for containers with `ContainerScalingMode` set to 'Off'. properties: containerName: description: Name of the container. type: string lowerBound: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true description: Minimum recommended amount of resources. Observes ContainerResourcePolicy. This amount is not guaranteed to be sufficient for the application to operate in a stable way, however running with less resources is likely to have significant impact on performance/availability. type: object target: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true description: Recommended amount of resources. Observes ContainerResourcePolicy. type: object uncappedTarget: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true description: The most recent recommended resources target computed by the autoscaler for the controlled pods, based only on actual resource usage, not taking into account the ContainerResourcePolicy. May differ from the Recommendation if the actual resource usage causes the target to violate the ContainerResourcePolicy (lower than MinAllowed or higher that MaxAllowed). Used only as status indication, will not affect actual resource assignment. type: object upperBound: additionalProperties: anyOf: - type: integer - type: string pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true description: Maximum recommended amount of resources. Observes ContainerResourcePolicy. Any resources allocated beyond this value are likely wasted. This value may be larger than the maximum amount of application is actually capable of consuming. type: object required: - target type: object type: array type: object type: object required: - spec type: object served: true storage: false
集群版本<1.22
展开查看CRD YAML
apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: verticalpodautoscalers.autoscaling.k8s.io annotations: "api-approved.kubernetes.io": "https://github.com/kubernetes/kubernetes/pull/63797" spec: group: autoscaling.k8s.io scope: Namespaced names: plural: verticalpodautoscalers singular: verticalpodautoscaler kind: VerticalPodAutoscaler shortNames: - vpa version: v1beta1 versions: - name: v1beta1 served: false storage: false - name: v1beta2 served: true storage: true - name: v1 served: true storage: false validation: # openAPIV3Schema is the schema for validating custom objects. openAPIV3Schema: type: object properties: spec: type: object required: [] properties: targetRef: type: object updatePolicy: type: object properties: updateMode: type: string resourcePolicy: type: object properties: containerPolicies: type: array items: type: object properties: containerName: type: string controlledValues: type: string enum: ["RequestsAndLimits", "RequestsOnly"] mode: type: string enum: ["Auto", "Off"] minAllowed: type: object maxAllowed: type: object controlledResources: type: array items: type: string enum: ["cpu", "memory"] --- apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: verticalpodautoscalercheckpoints.autoscaling.k8s.io annotations: "api-approved.kubernetes.io": "https://github.com/kubernetes/kubernetes/pull/63797" spec: group: autoscaling.k8s.io scope: Namespaced names: plural: verticalpodautoscalercheckpoints singular: verticalpodautoscalercheckpoint kind: VerticalPodAutoscalerCheckpoint shortNames: - vpacheckpoint version: v1beta1 versions: - name: v1beta1 served: false storage: false - name: v1beta2 served: true storage: true - name: v1 served: true storage: false
kubectl apply -f crd.yaml
安装ack-vertical-pod-autoscaler组件的Admission Controller、Recommender和Updater组件。
1.22≤ 集群版本<1.26
Admission-controller组件
apiVersion: apps/v1 kind: Deployment metadata: name: vpa-admission-controller namespace: kube-system spec: replicas: 1 selector: matchLabels: app: vpa-admission-controller template: metadata: labels: app: vpa-admission-controller spec: serviceAccountName: vpa-admission-controller securityContext: runAsNonRoot: true runAsUser: 65534 # nobody containers: - name: admission-controller image: registry.cn-hangzhou.aliyuncs.com/acs/vpa-admission-controller:0.13.0 imagePullPolicy: Always env: - name: NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace volumeMounts: - name: tls-certs mountPath: "/etc/tls-certs" readOnly: true resources: limits: cpu: 200m memory: 500Mi requests: cpu: 50m memory: 200Mi ports: - containerPort: 8000 - name: prometheus containerPort: 8944 volumes: - name: tls-certs secret: secretName: vpa-tls-certs --- apiVersion: v1 kind: Service metadata: name: vpa-webhook namespace: kube-system spec: ports: - port: 443 targetPort: 8000 selector: app: vpa-admission-controller
Recommender组件
apiVersion: apps/v1 kind: Deployment metadata: name: vpa-recommender namespace: kube-system spec: replicas: 1 selector: matchLabels: app: vpa-recommender template: metadata: labels: app: vpa-recommender spec: serviceAccountName: vpa-recommender securityContext: runAsNonRoot: true runAsUser: 65534 # nobody containers: - name: recommender image: registry.cn-hangzhou.aliyuncs.com/acs/vpa-recommender:0.13.0 imagePullPolicy: Always resources: limits: cpu: 200m memory: 1000Mi requests: cpu: 50m memory: 500Mi ports: - name: prometheus containerPort: 8942
Updater组件
apiVersion: apps/v1 kind: Deployment metadata: name: vpa-updater namespace: kube-system spec: replicas: 1 selector: matchLabels: app: vpa-updater template: metadata: labels: app: vpa-updater spec: serviceAccountName: vpa-updater securityContext: runAsNonRoot: true runAsUser: 65534 # nobody containers: - name: updater image: registry.cn-hangzhou.aliyuncs.com/acs/vpa-updater:0.13.0 imagePullPolicy: Always env: - name: NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace resources: limits: cpu: 200m memory: 1000Mi requests: cpu: 50m memory: 500Mi ports: - name: prometheus containerPort: 8943
集群版本<1.22
Admission-controller组件
apiVersion: apps/v1 kind: Deployment metadata: name: vpa-admission-controller namespace: kube-system spec: replicas: 1 selector: matchLabels: app: vpa-admission-controller template: metadata: labels: app: vpa-admission-controller spec: serviceAccountName: admin containers: - name: admission-controller image: registry.cn-hangzhou.aliyuncs.com/acs/vpa-admission-controller:0.7.0 imagePullPolicy: Always env: - name: NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace volumeMounts: - name: tls-certs mountPath: "/etc/tls-certs" readOnly: true resources: limits: cpu: 200m memory: 500Mi requests: cpu: 50m memory: 200Mi ports: - containerPort: 8000 volumes: - name: tls-certs secret: secretName: vpa-tls-certs --- apiVersion: v1 kind: Service metadata: name: vpa-webhook namespace: kube-system spec: ports: - port: 443 targetPort: 8000 selector: app: vpa-admission-controller
Recommender组件
apiVersion: apps/v1 kind: Deployment metadata: name: vpa-recommender namespace: kube-system spec: replicas: 1 selector: matchLabels: app: vpa-recommender template: metadata: labels: app: vpa-recommender spec: serviceAccountName: admin containers: - name: recommender image: registry.cn-hangzhou.aliyuncs.com/acs/vpa-recommender:0.7.0 imagePullPolicy: Always resources: limits: cpu: 200m memory: 1000Mi requests: cpu: 50m memory: 500Mi ports: - containerPort: 8080
Updater组件
apiVersion: apps/v1 kind: Deployment metadata: name: vpa-updater namespace: kube-system spec: replicas: 1 selector: matchLabels: app: vpa-updater template: metadata: labels: app: vpa-updater spec: serviceAccountName: admin containers: - name: updater image: registry.cn-hangzhou.aliyuncs.com/acs/vpa-updater:0.7.0 imagePullPolicy: Always resources: limits: cpu: 200m memory: 1000Mi requests: cpu: 50m memory: 500Mi ports: - containerPort: 8080
步骤二:验证安装VPA
保存以下YAML文件,执行命令创建名为nginx-deployment-basic的Deployment。
展开查看nginx-deployment-basic.yaml文件
说明将Deployment中的资源
requests
和limits
留空。apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment-basic labels: app: nginx spec: replicas: 2 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.7.9 ports: - containerPort: 80
kubectl apply -f nginx-deployment-basic.yaml
保存以下YAML文件,执行命令创建名为nginx-deployment-basic-vpa的VPA资源。
展开查看nginx-deployment-basic-vpa.yaml文件
说明您可以按需设置
updateMode
为Off
或者Auto
。Off
(推荐):根据集群资源消耗推荐Pod的资源Request和Limit配置,但不会自动更新Pod资源配置。Auto
:根据集群资源消耗推荐Pod的资源Request和Limit配置,并自动更新Pod资源配置。
apiVersion: autoscaling.k8s.io/v1 kind: VerticalPodAutoscaler metadata: name: nginx-deployment-basic-vpa spec: targetRef: apiVersion: "apps/v1" kind: Deployment name: nginx-deployment-basic updatePolicy: updateMode: "Off" # 设置为off。
kubectl apply -f nginx-deployment-basic-vpa.yaml
执行以下命令,查询VPA为Deployment推荐的CPU和内存资源的
requests
值。kubectl describe vpa nginx-deployment-basic-vpa
预期输出需要等待两分钟左右。正常情况下,预期输出如下,展示了VPA为Deployment推荐的值。
展开查看预期输出
Recommendation: Container Recommendations: Container Name: nginx Lower Bound: Cpu: 25m Memory: 262144k Target: Cpu: 25m Memory: 262144k Uncapped Target: Cpu: 25m Memory: 262144k Upper Bound: Cpu: 11601m Memory: 12128573170
您可以根据VPA的推荐值来配置Deployment中实际资源的
requests
。VPA会持续监控应用资源的使用情况,并提供优化建议。
如何通过控制台管理使用kubectl安装的VPA?
对于v1.26及以上的集群,推荐您使用控制台进行ack-vertical-pod-autoscaler组件的安装和VPA的管理。为避免新旧VPA冲突,请卸载旧的VPA(通过kubectl安装的VPA),并通过控制台重新安装。
展开查看完整的操作步骤
步骤1:卸载集群中已经部署的VPA
卸载集群中已经部署的VPA,包括清理Deployment、RBAC、Secret、CRD、Service等资源,以避免新安装的VPA与旧版VPA冲突而导致VPA不可用。
在旧版VPA的YAML文件中(本小节以下方示例nginx-deployment-basic-vpa.yaml为例),删除
metadata
中的信息,仅保留name
和namespace
字段即可,同时删除status
字段信息。然后,保存此YAML文件供后续步骤使用。apiVersion: autoscaling.k8s.io/v1 kind: VerticalPodAutoscaler metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"autoscaling.k8s.io/v1","kind":"VerticalPodAutoscaler","metadata":{"annotations":{},"name":"nginx-deployment-basic-vpa","namespace":"default"},"spec":{"targetRef":{"apiVersion":"apps/v1","kind":"Deployment","name":"nginx-deployment-basic"},"updatePolicy":{"updateMode":"Off"}}} creationTimestamp: "2024-02-29T06:03:35Z" generation: 1 name: nginx-deployment-basic-vpa namespace: default resourceVersion: "56264" uid: 9f128737-d12e-46f6-b254-c1a7505c19c6 spec: targetRef: apiVersion: apps/v1 kind: Deployment name: nginx-deployment-basic updatePolicy: updateMode: "Off" status: conditions: - lastTransitionTime: "2024-02-29T06:03:55Z" status: "True" type: RecommendationProvided recommendation: containerRecommendations: - containerName: nginx lowerBound: cpu: 25m memory: 262144k target: cpu: 25m memory: 262144k uncappedTarget: cpu: 25m memory: 262144k upperBound: cpu: 25m memory: 262144k
kubectl get vpa nginx-deployment-basic-vpa -oyaml
执行以下命令,删除VPA组件安装的资源。
// 删除Deployment和Service kubectl delete deployment vpa-admission-controller vpa-recommender vpa-updater -n kube-system kubectl delete svc vpa-webhook -n kube-system // 删除ClusterRole kubectl delete clusterrole system:metrics-reader system:vpa-actor system:vpa-status-actor system:vpa-checkpoint-actor system:evictioner system:vpa-target-reader system:vpa-admission-controller system:vpa-status-reader // 删除ClusterRoleBinding kubectl delete clusterrolebinding system:metrics-reader system:vpa-actor system:vpa-status-actor system:vpa-checkpoint-actor system:vpa-target-reader-binding system:vpa-evictioner-binding system:vpa-admission-controller system:vpa-status-reader-binding // 删除ServiceAccount kubectl delete sa vpa-admission-controller vpa-recommender vpa-updater -n kube-system // 删除Secret kubectl delete secret vpa-tls-certs -n kube-system //删除CRD kubectl delete crd verticalpodautoscalercheckpoints.autoscaling.k8s.io verticalpodautoscalers.autoscaling.k8s.io
步骤2:安装ack-vertical-pod-autoscaler组件
登录容器服务管理控制台,在左侧导航栏选择集群。
在集群列表页面,单击目标集群名称,然后在左侧导航栏,选择运维管理 > 组件管理。
在组件管理页面,定位ack-vertical-pod-autoscaler组件,然后按照页面提示完成安装。
步骤3:重新部署VPA YAML文件
执行以下命令,重新部署步骤1:卸载集群中已经部署的VPA保存的VPA YAML文件,本步骤以nginx-deployment-basic-vpa.yaml为例。
kubectl apply -f nginx-deployment-basic-vpa.yaml
相关文档
如果您需要基于CPU使用率、内存使用率或其他自定义指标实现Pod的扩缩容,请参见使用容器水平伸缩(HPA)。
如果您的应用资源使用率存在周期性变化,需要按照类似Crontab的策略定时对Pod进行扩缩容,请参见使用容器定时水平伸缩(CronHPA)。
如果您的应用资源使用率存在周期性变化,但难以通过规则定义,您可以选择AHPA,以根据业务历史指标自动识别业务水位周期,进行Pod扩缩容,请参见AHPA概述。
如需基于消息队列、定时策略、自定义指标等Kubernetes事件灵活自定义扩缩容策略,对Pod进行扩缩容,请参见事件驱动弹性。