Application distribution status and FAQ
This topic explains how application distribution works, describes distribution statuses, and provides troubleshooting for common issues.
How it works
-
Resource matching: The system matches application resources with distribution policies. If a distribution policy selects an existing application resource in the cluster, a
ResourceBindingobject is created. This object has a one-to-one correspondence with the application resource. TheResourceBindingobject is named using the${resource name}-${resource Kind}format, allowing you to query it directly by name. For cluster-scoped application resources, the corresponding object is aClusterResourceBinding. -
Multi-cluster scheduling: The system schedules the
ResourceBindingobject based on the created distribution policy. After scheduling is complete, theSCHEDULEDcondition in the status of theResourceBindingis set toTrue. -
Differentiated rendering: If you create a differentiated policy for the application resource, the policy takes effect at this stage.
-
Application resource creation on sub-clusters: The application resource is created on the target sub-clusters. After successful creation, the
FULLYAPPLIEDcondition in the status of theResourceBindingis set toTrue. -
Application resources run on sub-clusters: When workload-type application resources run on a sub-cluster, their status changes to
Ready. When an application resource is ready, theALLAVAILABLEcondition in the status of theResourceBindingis set toTrue.
Application distribution to multiple clusters
Step 1: Create resources and policies
-
Use the following sample YAML to create a resource and a distribution policy.
apiVersion: apps/v1 kind: Deployment metadata: name: web-demo spec: replicas: 5 selector: matchLabels: app: web-demo template: metadata: labels: app: web-demo spec: containers: - name: nginx image: registry-cn-hangzhou.ack.aliyuncs.com/acs/web-demo:0.5.0 ports: - containerPort: 80 --- apiVersion: policy.one.alibabacloud.com/v1alpha1 kind: PropagationPolicy metadata: name: web-demo spec: # If this parameter is set to true, resources on sub-clusters are retained when you delete the resource from the Fleet instance. To also delete the resources on sub-clusters, set this parameter to false. preserveResourcesOnDeletion: true resourceSelectors: - apiVersion: apps/v1 kind: Deployment name: web-demo placement: clusterAffinity: clusterNames: - ${cluster1-id} # Your cluster ID. - ${cluster2-id} replicaScheduling: replicaSchedulingType: Duplicated -
(Optional) Use the following sample YAML to create a differentiated policy.
apiVersion: one.alibabacloud.com/v1alpha1 kind: OverridePolicy metadata: name: demo spec: resourceSelectors: - apiVersion: apps/v1 kind: Deployment name: web-demo overrideRules: - targetCluster: clusterNames: - ${cluster1-id} # Your cluster ID. overriders: plaintext: - operator: replace path: /spec/replicas value: 3
Step 2: Check distribution status
-
Run the following command to check the distribution status.
kubectl get rb web-demo-deploymentExpected output:
NAME SCHEDULED FULLYAPPLIED OVERRIDDEN ALLAVAILABLE AGE web-demo-deployment True True True True 17hParameter
Description
SCHEDULED
Indicates whether the resource was successfully scheduled.
FULLYAPPLIED
Indicates whether the resource was successfully created on the sub-clusters.
OVERRIDDEN
Indicates whether the differentiated policy was applied. This status is True by default and becomes False only if a defined policy fails to apply.
ALLAVAILABLE
Indicates whether the resource has reached its desired state on the sub-clusters.
-
(Optional) Verify the differentiated policy.
-
Run the
kubectl get rb web-demo-deploymentcommand and check theOVERRIDDENcolumn in the output. If the value isTrue, the differentiated policy has taken effect. -
Run the
kubectl describe deploy web-democommand and check the events to determine if the differentiated policy was applied. The following output indicates success:Normal ApplyOverridePolicySucceed 2m37s (x3 over 49m) override-manager Apply override policy(default/demo) for cluster(cxxxxxxx) succeed.
-
Step 3: View specific resource status
Run the following command to view the status of specific application resources.
kubectl get rb web-demo-deployment -oyaml
Expected output:
...
status:
aggregatedStatus:
- applied: true
appliedMessage: 'Applied Condition Message: Apply manifest complete; AvailableCondition
Message:Resource is available'
clusterName: ${cluster1-id}
health: Healthy
status:
availableReplicas: 3
generation: 7
observedGeneration: 7
readyReplicas: 3
replicas: 3
resourceTemplateGeneration: 2
updatedReplicas: 3
- applied: true
appliedMessage: 'Applied Condition Message: Apply manifest complete; AvailableCondition
Message:Resource is available'
clusterName: ${cluster2-id}
health: Healthy
status:
availableReplicas: 5
generation: 1
observedGeneration: 1
readyReplicas: 5
replicas: 5
resourceTemplateGeneration: 2
updatedReplicas: 5
The aggregatedStatus section shows the status of the deployment on both clusters, including the cluster name and the corresponding deployment status details. Note that the differentiated policy set the replica count for Cluster1 to 3.
FAQ
Why is no ResourceBinding or ClusterResourceBinding created?
This issue usually occurs when the resourceSelector parameter in the distribution policy does not match the application resource. Check whether the conditions declared in ResourceSelectors match your application resource.
How to handle scheduling failures?
status:
conditions:
- lastTransitionTime: "2025-03-03T02:06:21Z"
message: '0/2 clusters are available: 2 cluster(s) did not match the placement
cluster affinity constraint.'
reason: NoClusterFit
status: "False"
type: Scheduled
You can run the following command to view the details of the ResourceBinding:
kubectl get rb xxx -oyaml -n xxx
Scheduling typically fails because the clusterAffinity parameter in your distribution policy does not match any of your sub-clusters. Ensure that you specify the correct cluster IDs.
How to handle a failed differentiated policy?
First, check if the resourceSelector parameter of the differentiated policy selects the intended application resource. You can troubleshoot this issue in one of the following ways:
-
Run the following command to check the
messagefield in theResourceBindingfor failure details.kubectl get rb ${resource name}-${resource kind} -oyaml -n ${namespace}Expected output:
- lastTransitionTime: "2025-03-04T08:46:46Z" message: 'Failed to apply OverridePolicy/ClusterOverridePolicy, err is: failed to apply overrides(default/demo) for resource(default/web-demo), error: replace operation does not apply: doc is missing key: /spec/replicas: missing value' reason: OverriddenFailed status: "False" type: Overridden -
Run the following command to view the events related to the resource. Look for key events like
ApplyOverridePolicyFailedandSyncWorkFailed. Themessagefield provides details about the failure.kubectl describe ${resource-kind} ${resource-name} -n ${resource-namespace}
A common reason a differentiated policy fails is an incorrect path configuration, such as specifying a path that does not exist in the resource.
Why does scheduling succeed but creation fails?
If scheduling is successful but the FULLYAPPLIED status is False, view the status of specific application resources. Check the appliedMessage field for each cluster to find the reason for the failure.
This failure often occurs when you try to distribute an unsupported application resource. You can submit a ticket for assistance.
Why is a created resource not ready?
If an application resource is created on a sub-cluster but the ALLAVAILABLE status is False, you can view the status of specific application resources. Check the appliedMessage field for each cluster to find out why the resource is not ready.
Typically, resources like ConfigMaps and Secrets are considered ready as soon as they are created. However, for workload-type application resources such as Deployments and StatefulSets, the ready state can be affected by various factors, including insufficient underlying resources or scheduling failures on the sub-cluster. You can connect to your sub-cluster or use AMC to investigate and resolve the issue.