Empty-push protection

更新时间:
复制 MD 格式

A service consumer subscribes to a service provider's instance list from a registry. Subscription exceptions can occur due to various factors, such as registry changes, unexpected events, or connection jitter between the service provider and the registry. These exceptions can cause the service consumer to receive an empty list of service provider instances. You can enable the empty-push protection feature on a Nacos client or an MSE Nacos server to improve the availability of the entire system.

Prerequisites

Enable empty-push protection on the client

Limits

Nacos Java Client 1.4.1 and later support client-side empty-push protection.

Note

To avoid using versions with potential risks, see Version recommendations and Application of MSE Nacos SDK and restricted Nacos SDK versions to select appropriate versions for your client, Spring Cloud, and Dubbo.

Procedure

  • If you use the Nacos-Client for Java:

    1. Import the nacos-client dependency:

      <!-- ${nacos-client.version} must be 1.4.1 or later -->
      <dependency>
          <groupId>com.alibaba.nacos</groupId>
          <artifactId>nacos-client</artifactId>
          <version>${nacos-client.version}</version>
      </dependency>
    2. Configure the settings in your program code as follows:

      Properties properties = new Properties();
      properties.put(PropertyKeyConst.SERVER_ADDR, "${mseNacosInstanceDomainName}");
      properties.put(PropertyKeyConst.NAMING_PUSH_EMPTY_PROTECTION, "true");
      NamingService naming = NamingFactory.createNamingService(properties);
  • If you use the Spring Cloud Alibaba framework, add the following configuration to the application configuration file.

    spring.cloud.nacos.discovery.namingPushEmptyProtection=true
  • If you use the Dubbo framework, add the following configuration to the registry link URL in the Dubbo configuration file.

    dubbo.registry.address=nacos://${mseNacosInstanceDomainName}:8848?namingPushEmptyProtection=true

Enable empty-push protection on the server

Note

Because many multi-language clients and older versions of the Java client are still in use, MSE enhances the capabilities of open source Nacos by providing server-side empty-push protection.

Limits

The Nacos database engine version must be 2.1.0.0 or later. For more information, see Upgrade the Nacos database engine version.

Procedure

The method to enable empty-push protection varies based on the Nacos database engine version.

  • If your MSE Nacos database engine version is 2.1.0.0 or 2.1.0.1, empty-push protection is enabled automatically. No action is required.

  • If your MSE Nacos database engine version is 2.1.0.2 or later, perform the following steps to enable empty-push protection.

    1. Log on to the MSE console, and select a region in the top navigation bar.

    2. In the left-side navigation pane, choose Microservices Registry > Instances.

    3. On the Instances page, click the name of the instance.

    4. In the navigation pane on the left, click Parameter Settings. On the Parameter Settings page, in the Real-time Effective Parameters section, click Edit in the Actions column for forcePushEmptyProtectionForAllService.

    5. In the Edit Parameters dialog box, set Value to Yes and click OK.

      Note

      To ensure maximum business availability, we recommend that you enable empty-push protection by setting the Value of forcePushEmptyProtectionForAllService to Yes.

Verify the results

When a service triggers empty-push protection, you can see the message Trigger push empty protection for Service in the ${user.home}/logs/nacos/naming.log file on the consumer's node.

Normally, this message does not appear frequently. If it does, you can troubleshoot based on the following scenarios.

  • Subscribing to a non-existent service.

    If an application subscribes to a non-existent service, check whether the application still needs this dependency. If not, delete the invalid dependency. If the dependency is still needed, republish the service provider.

  • The service provider to which the application subscribed is empty.

    • Check whether the service provider frequently goes offline.

    • Check whether the network connectivity between the service provider and the MSE Nacos cluster is unstable. An unstable connection can cause Nacos to consider the provider offline.

Locate the service that triggers empty-push protection

The error message contains the service that triggered the protection. You can identify which empty service the consumer has subscribed to from Service{namespace='XXX', group='xx', name='xxxxxxx', ephemeral=true, revision=0}.

Common scenarios that trigger empty-push protection

Scenario

Trigger reason

Solution

In a Dubbo 2 multi-subscription compatibility scenario, the old-format service name is empty.

Dubbo 2.7 versions earlier than 2.7.6 use a different service name format than later versions. Later Dubbo 2.7 versions support multi-subscription, which subscribes to both old and new format service names. When all providers are later versions, the old-format service does not exist in Nacos. This triggers empty-push protection when subscribing to the old format.

Upgrade the database engine to 2.1.0.1 or later, or upgrade Dubbo to 2.7.17 or later. For more information, see Upgrade the Nacos database engine version.

In a Dubbo 3 multi-subscription compatibility scenario, the interface-level service is empty.

Dubbo 3 supports application-level service discovery, where the registered service name is the application name, not the interface name. To support smooth upgrades from earlier versions, it subscribes to both application-level and interface-level service names. When all providers are Dubbo 3 versions, the interface-level service does not exist. This triggers empty-push protection when subscribing to the interface-level service.

Set the Dubbo configuration dubbo.application.service-discovery.migration=FORCE_APPLICATION. For more information, see the Dubbo official website.

When the NacosWatch feature in Spring Cloud starts for the first time, the service is empty if no other instance replicas exist.

Spring Cloud Alibaba added the NacosWatch feature in a recent version to monitor its own service status. This feature listens to its own service when the application starts. When this feature starts for the first time and there are no other instance replicas, the registry does not have this service. The subscription triggers empty-push protection. The service recovers after the application starts.

Ignore the error on the first startup with no other replicas, or disable the Watch feature by setting spring.cloud.nacos.discovery.watch.enabled=false. For more information, see Spring Cloud Alibaba .

A service is completely unpublished in Spring Cloud Gateway, but its status is not synchronized.

Spring Cloud Gateway queries the registry for a list of all current services at startup and subscribes to all of them. For example, if a service is completely unpublished (all instances removed) after the Gateway starts, the registry automatically removes the unpublished service. However, Spring Cloud Gateway is not aware of this change and continues to subscribe, which triggers empty-push protection.

Currently, Spring Cloud Gateway does not support dynamically detecting that a service has been completely removed and then unsubscribing. We recommend that you restart Spring Cloud Gateway.

Scenarios where empty-push protection is enforced

In MSE Nacos 2.1.0.2 and later, the optimized empty-push protection logic protects all empty service lists only when forcePushEmptyProtectionForAllService is enabled. However, in the following scenarios, the system automatically enables empty-push protection to ensure availability and stability, even if the forcePushEmptyProtectionForAllService switch is off.

  • When an engine node is being upgraded, restarted, or undergoing automatic fault recovery, the node automatically enables empty-push protection for all services for 2 minutes to ensure business availability during this period.

  • When all instances of a service are removed, the cluster automatically enables empty-push protection for that service for 1 minute. This prevents empty pushes caused by issues such as network jitter or short-term service provider failures, which ensures business availability.

Note

To ensure maximum business availability, we recommend that you enable empty-push protection for all services by turning on the forcePushEmptyProtectionForAllService switch.