Message query

Updated at:

To troubleshoot consumption issues in Kafka, such as message failures, loss, or delays, use the Message Query feature to inspect message content and attributes.

Usage notes

  • In the Message Queue for Apache Kafka console, up to 1 KB of content is displayed for each queried message. Content exceeding 1 KB is truncated. To view the full message content, download the message. The maximum size of a downloaded message is 10 MB. If a message exceeds 10 MB, only the first 10 MB is downloaded.

  • Query results are limited to a maximum of 10 messages and a total size of 10 MB.

    • For example, if the total size of three messages exceeds 10 MB, the console displays only two.

    • If the total size of 10 messages is less than 10 MB, you can still view a maximum of 10 messages in the console. In this case, the actual number of messages consumed is determined by the client.

  • The ability to query a message depends on the message cleanup policies of Message Queue for Apache Kafka:

    • If disk usage is below 75%, expired messages are deleted in a batch at 04:00 daily.

    • If disk usage is between 75% and 85%, expired messages are deleted periodically until disk usage falls below 75%.

    • If disk usage is between 85% and 90%, messages are deleted in the order they were stored, regardless of their retention period.

    • If disk usage is 90% or higher, the service enables write protection to ensure stability, preventing new messages from being written.

    Important

    When Message Queue for Apache Kafka deletes messages, it retains at least one storage file. Therefore, you may still be able to query expired messages.

Query messages by offset

  1. Log on to the ApsaraMQ for Kafka console.

  2. On the Overview page, select a region in the Resource Distribution section.

  3. On the Instances page, click the name of the target instance.

  4. In the left-side navigation pane, click Message Query.

  5. On the Message Query page, select Search by offset from the Search Method list.

  6. From the Topic list, select the topic. From the Partition list, select the partition. In the Offset box, enter the message offset, and then click Search.

    The results show consecutive messages starting from the specified offset. For example, if you set the partition and start offset to 5, the results start from offset 5 in partition 5.

    Table 1. Query result parameters

    Parameter

    Description

    Partition

    The topic partition where the message is stored.

    Offset

    The offset of the message in the partition.

    Key

    The message key. It is converted to a string for display.

    Value

    The message content (value). It is converted to a string for display.

    Headers

    The message headers, displayed as key-value pairs, such as [trace-id=123]. If no headers are configured, this field defaults to [].

    Compression type

    The message compression type. The default value none indicates no compression.

    Created At

    The time the message was created. This time is either included by the client or specified in the ProducerRecord when the producer sends the message.

    Note
    • If this field is configured, the configured value is displayed.

    • If this field is not configured, the system time when the message was sent is used by default.

    • A value such as 1970/x/x x:x:x indicates that the creation time was set to 0 or another invalid value.

    • Clients for Message Queue for Apache Kafka version 0.9 or earlier do not support setting this time.

    Actions

    • Click Download Key: Downloads the key of the message.

    • Click Download Value: Downloads the content of the message.

Query messages by time

This method is useful if you know when a message was sent but not its partition. Specify a point in time to query messages sent around that time.

  1. On the Message Query page, select Search by Point in Time from the Search Method list.

  2. From the Topic list, select the topic. From the Partition list, select the partition. In the Point in Time field, select a time, and then click Search.

    The results show consecutive messages from the specified point in time onward. For descriptions of the result parameters, see Table 1. Query result parameters.

    • If you select All Partitions, messages from all partitions are displayed.

    • If you select a specific partition, only messages from that partition are displayed.

Related documents