Set thresholds in a StorageAutoScalerPolicy so storage-auto-expander in storage-operator auto-expands disk volumes.
How it works
When storage-auto-expander detects that PVC disk usage exceeds the threshold, it expands the volume in this sequence:
-
Monitors disk usage on PVCs matching the policy's label selector.
-
When usage exceeds the threshold, selects the first matching action and submits a resize request to the CSI (Container Storage Interface) driver.
-
The CSI driver resizes the cloud disk and signals Kubernetes to resize the file system.
-
PVC capacity updates after the file system resize completes.
Timing constraints: The trigger check interval is at most 2 minutes, and expansion takes about 1 minute. Do not fill the disk within 3 minutes of a previous expansion.
Limitations
| Limitation | Detail |
|---|---|
| Kubernetes version | Requires Kubernetes 1.16 or later. Upgrade the cluster if needed. |
| Disk type | Basic disks are not supported. See ResizeDisk for eligible disk types. |
| StorageClass | The PVC's StorageClass must have allowVolumeExpansion: true. ACK-provided StorageClasses enable this by default. For custom StorageClasses, set the parameter at creation time — existing StorageClasses cannot be modified. |
| Pod state | The application pod using the disk must be in Running state. |
Enable the storage-auto-expander module
The storage-auto-expander module in storage-operator handles automatic expansion. To check whether storage-operator is installed and find its version, go to Operations > Add-ons > Volumes.
-
v1.33.1 and later: The module is enabled by default. No action needed.
-
Earlier than v1.33.1: Enable the module:
kubectl patch configmap/storage-operator \ -n kube-system \ --type merge \ -p '{"data":{"storage-auto-expander":"{\"imageRep\":\"acs/storage-auto-expander\",\"imageTag\":\"\",\"install\":\"true\",\"template\":\"/acs/templates/storage-auto-expander/install.yaml\",\"type\":\"deployment\"}"}}'To upgrade
storage-operator, see Upgrade add-ons.
Configure an automatic expansion policy
Step 1: Verify that volume expansion is enabled on your StorageClass
kubectl get sc
The ALLOWVOLUMEEXPANSION column must be true. Example output:
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
alicloud-disk-efficiency diskplugin.csi.alibabacloud.com Delete Immediate true 26h
alicloud-disk-essd diskplugin.csi.alibabacloud.com Delete Immediate true 26h
alicloud-disk-ssd diskplugin.csi.alibabacloud.com Delete Immediate true 26h
alicloud-disk-topology-alltype diskplugin.csi.alibabacloud.com Delete WaitForFirstConsumer true 26h
Use alicloud-disk-topology-alltype when possible. It auto-selects the disk type based on instance type and zone availability, reducing provisioning failures. See Use dynamically provisioned volumes.
Step 2: Create a StorageAutoScalerPolicy
Create a file named StorageAutoScalerPolicy.yaml:
apiVersion: storage.alibabacloud.com/v1alpha1
kind: StorageAutoScalerPolicy
metadata:
name: hybrid-expand-policy
spec:
pvcSelector:
matchLabels:
app: nginx
namespaces:
- default
- nginx
conditions:
- name: condition1
key: volume-capacity-used-percentage
operator: Gt
values:
- "80"
actions:
- name: action1
type: volume-expand
params:
scale: 50Gi
limits: 100Gi
- name: action2
type: volume-expand
params:
scale: 50%
limits: 300Gi
Key fields:
| Field | Description |
|---|---|
pvcSelector |
Selects PVCs by label. Example: app: nginx. |
namespaces |
Target PVC namespaces. Multiple values use OR logic. Default: default. |
conditions |
Trigger conditions. Multiple conditions use AND logic. key: metric (volume-capacity-used-percentage = disk usage percentage). operator: Gt (greater than), Lt (less than), Eq (equal), or Ne (not equal) — case-insensitive. values: threshold as a string. |
actions |
Actions to run when conditions are met. The first matching action runs; the rest are skipped. type: must be volume-expand. params.scale: expansion amount — absolute (e.g., 50Gi) or percentage (e.g., 50%). params.limits: maximum PVC size for this action. |
The policy triggers when PVC usage exceeds 80%:
-
action1 adds 50 GiB (max 100 GiB). Applies while PVC is under 100 GiB.
-
action2 adds 50% of current size (max 300 GiB). Applies once PVC reaches 100 GiB.
Step 3: Apply the policy
kubectl create -f StorageAutoScalerPolicy.yaml
Verify automatic expansion
The following steps trigger five expansion events with a test StatefulSet, using the policy defined above.
Step 1: Create a test StatefulSet
Create a file named StatefulSet.yaml:
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: nginx
spec:
selector:
matchLabels:
app: nginx
serviceName: nginx
replicas: 1
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: anolis-registry.cn-zhangjiakou.cr.aliyuncs.com/openanolis/nginx:1.14.1-8.6
ports:
- containerPort: 80
volumeMounts:
- name: pvc-disk
mountPath: /data
volumes:
- name: pvc-disk
persistentVolumeClaim:
claimName: disk-pvc
volumeClaimTemplates:
- metadata:
name: pvc-disk
labels:
app: nginx
spec:
accessModes: [ "ReadWriteOnce" ]
storageClassName: "alicloud-disk-topology-alltype"
resources:
requests:
storage: 25Gi
Apply it and confirm the pod is running:
kubectl create -f StatefulSet.yaml
kubectl get pod -l app=nginx
Expected output:
NAME READY STATUS RESTARTS AGE
nginx-0 1/1 Running 0 99s
Check the initial disk capacity:
kubectl exec -it nginx-0 -- df -h /data
Expected output:
Filesystem Size Used Avail Use% Mounted on
/dev/vdb 25G 24K 25G 1% /data
Step 2: Trigger the first expansion (25 GiB to 75 GiB)
Write 22 GB of data to push usage above 80%:
kubectl exec -it nginx-0 -- fallocate -l 22G /data/test1
Check expansion events:
kubectl get events
The disk is 25 GiB and usage exceeds 80%, so action1 adds 50 GiB:
2m1s Warning StartExpand persistentvolumeclaim/pvc-disk-nginx-0 Start to expand of pvc pvc-disk-nginx-0 from 25Gi to 75Gi, usedCapacityPercentage:90%, freeSize:2498MB.
2m1s Normal ExternalExpanding persistentvolumeclaim/pvc-disk-nginx-0 waiting for an external controller to expand this PVC
2m1s Normal Resizing persistentvolumeclaim/pvc-disk-nginx-0 External resizer is resizing volume d-uf66kkzltnq6xgi9****
118s Normal FileSystemResizeRequired persistentvolumeclaim/pvc-disk-nginx-0 Require file system resize of volume on node
116s Warning SkipExpand persistentvolumeclaim/pvc-disk-nginx-0 Pvc pvc-disk-nginx-0 is expanding status from 25Gi to 75Gi, this action action2 will skip.
Verify the PVC capacity:
kubectl get pvc
Expected output:
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE
pvc-disk-nginx-0 Bound d-uf66kkzltnq6xgi9**** 75Gi RWO alicloud-disk-topology-alltype <unset> 26m
Step 3: Trigger the second expansion (75 GiB to 100 GiB)
Write 40 GB to push usage above 80% again:
kubectl exec -it nginx-0 -- fallocate -l 40G /data/test2
action1 fires again, but its 100 GiB limit caps expansion at 100 GiB:
kubectl get events
7m4s Warning StartExpand persistentvolumeclaim/pvc-disk-nginx-0 Start to expand of pvc pvc-disk-nginx-0 from 75Gi to 100Gi, usedCapacityPercentage:84%, freeSize:11927MB.
7m4s Normal ExternalExpanding persistentvolumeclaim/pvc-disk-nginx-0 waiting for an external controller to expand this PVC
7m4s Normal Resizing persistentvolumeclaim/pvc-disk-nginx-0 External resizer is resizing volume d-uf66kkzltnq6xgi9****
7m1s Normal FileSystemResizeRequired persistentvolumeclaim/pvc-disk-nginx-0 Require file system resize of volume on node
5m59s Warning SkipExpand persistentvolumeclaim/pvc-disk-nginx-0 Pvc pvc-disk-nginx-0 is expanding status from 75Gi to 100Gi, this action action2 will skip.
Step 4: Trigger the third expansion (100 GiB to 150 GiB)
At 100 GiB, action1's limit is reached, so action2 takes over and adds 50% of current size:
kubectl exec -it nginx-0 -- fallocate -l 20G /data/test3
kubectl get events
2m40s Warning StartExpand persistentvolumeclaim/pvc-disk-nginx-0 Start to expand of pvc pvc-disk-nginx-0 from 100Gi to 150Gi, usedCapacityPercentage:83%, freeSize:16637MB.
2m40s Normal ExternalExpanding persistentvolumeclaim/pvc-disk-nginx-0 waiting for an external controller to expand this PVC
2m40s Normal Resizing persistentvolumeclaim/pvc-disk-nginx-0 External resizer is resizing volume d-uf66kkzltnq6xgi9****
2m37s Normal FileSystemResizeRequired persistentvolumeclaim/pvc-disk-nginx-0 Require file system resize of volume on node
109s Warning SkipExpand persistentvolumeclaim/pvc-disk-nginx-0 Pvc pvc-disk-nginx-0 is expanding status from 100Gi to 150Gi, this action action2 will skip.
Step 5: Trigger the fourth expansion (150 GiB to 225 GiB)
action2 fires again, adding 50% of 150 GiB = 75 GiB:
kubectl exec -it nginx-0 -- fallocate -l 50G /data/test4
kubectl get events
2m42s Warning StartExpand persistentvolumeclaim/pvc-disk-nginx-0 Start to expand of pvc pvc-disk-nginx-0 from 150Gi to 225Gi, usedCapacityPercentage:87%, freeSize:19621MB.
2m42s Normal ExternalExpanding persistentvolumeclaim/pvc-disk-nginx-0 waiting for an external controller to expand this PVC
2m42s Normal Resizing persistentvolumeclaim/pvc-disk-nginx-0 External resizer is resizing volume d-uf66kkzltnq6xgi9****
2m38s Normal FileSystemResizeRequired persistentvolumeclaim/pvc-disk-nginx-0 Require file system resize of volume on node
114s Warning SkipExpand persistentvolumeclaim/pvc-disk-nginx-0 Pvc pvc-disk-nginx-0 is expanding status from 150Gi to 225Gi, this action action2 will skip.
Step 6: Trigger the fifth expansion (225 GiB to 300 GiB)
action2 fires one final time, capped at its 300 GiB limit:
kubectl exec -it nginx-0 -- fallocate -l 50G /data/test5
kubectl get events
17m Warning StartExpand persistentvolumeclaim/pvc-disk-nginx-0 Start to expand of pvc pvc-disk-nginx-0 from 225Gi to 300Gi, usedCapacityPercentage:82%, freeSize:40351MB.
17m Normal ExternalExpanding persistentvolumeclaim/pvc-disk-nginx-0 waiting for an external controller to expand this PVC
17m Normal Resizing persistentvolumeclaim/pvc-disk-nginx-0 External resizer is resizing volume d-uf66kkzltnq6xgi9****
17m Normal FileSystemResizeRequired persistentvolumeclaim/pvc-disk-nginx-0 Require file system resize of volume on node
Both actions have reached their limits. No further expansion occurs.
Next steps
For cloud disk volume issues, see Cloud disk volume FAQ.