Automatically create topics for cloud migration or testing-ApsaraMQ for Kafka(Kafka)-阿里云帮助中心

Prerequisites

You have purchased and deployed a Message Queue for Apache Kafka instance that meets the following requirements:

Item	Description
Status	Running
Version	The major version is 2.2.0 or later, and the minor version is the latest.

Note

You can view the status and version of your instance in the Basic Information section on the Instance Details page of the Message Queue for Apache Kafka console.

Usage notes

Important: This feature is intended for migration and testing scenarios only. We strongly recommend that you do not enable automatic topic creation in production environments. If you must enable it for a production environment, do not use it for critical production services.
For a Professional Edition instance with ACL enabled, you cannot automatically create a topic by sending a message.

Procedure

Log on to the ApsaraMQ for Kafka console. In the Resource Distribution section of the Overview page, select the region where the ApsaraMQ for Kafka instance that you want to manage resides.
On the Instances page, click the name of the instance that you want to manage.
On the Instance Details page, in the Configuration Information section, turn on the Automatic Topic Creation switch.
Note
Naming conventions for automatically created topics:
- Must be 3 to 64 characters in length.
- Can contain only letters, digits, hyphens (-), and underscores (_), and must contain at least one letter or digit.
On a Standard Edition instance with this feature enabled, sending a message to a non-existent topic automatically creates it. The new topic then appears in the console.

FAQ

Can topics be created automatically?

Yes, but we strongly recommend against it. In Message Queue for Apache Kafka, you should manage topics using the Message Queue for Apache Kafka console and OpenAPI. We advise against enabling automatic topic creation in a production environment for the following reasons:

Automatic topic creation does not support fine-grained permission control or resource operation auditing, posing security and governance risks.
It can cause uncontrolled resource consumption, compromising system stability.
Automatic topic creation is not guaranteed to succeed, and failures can disrupt your services. For more information, see What should I do if automatic topic creation fails?

When should I enable automatic topic creation?

Enable this feature only for specific use cases such as rapid cloud migration and testing. Do not use it beyond these scenarios.

Handling automatic topic creation failures

If automatic topic creation fails, you can try the following solutions:

If you only need to create a few topics, we recommend creating them manually in the console or using OpenAPI.
Check the logs or troubleshoot the issue based on the following possible causes:
- Invalid operations: An invalid operation can prevent the topic from being created or used correctly, affecting your services. Examples include:
  - Invalid topic name: The topic name specified by the application does not meet the naming conventions. For example, the name contains invalid characters or exceeds the length limit.
  - Invalid topic configuration: The broker engine is improperly selected, or the quotas for topics or partitions are insufficient.
  - ACL enabled: If the ACL feature is enabled, you cannot automatically create a topic by sending a message.
- Uncontrollable factors: External factors such as network jitter, server upgrades, management platform maintenance, or ECS instance failures can significantly increase topic creation time and lead to failures.
- Service conflicts: Automatic topic creation can cause naming conflicts between services, mixing data across services. Such issues are difficult to detect and recover from, potentially causing irreversible data loss.
- Latency issues: Creating a topic on demand takes significantly longer than using an existing topic, potentially causing brief service interruptions.
- Quota limits: topic creation fails if the quota for topics or partitions is exceeded.

References

For detailed steps on cloud migration, see Cloud migration.
If you want to create topics using the console or API, see Create a topic and CreateTopic.