Query Message Service (MNS) logs
After you send logs from Simple Message Queue (formerly MNS) to Log Service, you can query them in real time. This topic shows how to query logs for common use cases. You can also combine multiple keywords to create more complex queries.
Prerequisites
-
Logs from Simple Message Queue (formerly MNS) have been collected. For more information, see Message Service (MNS) logs.
-
An index has been configured. For more information, see Create an index.
Background information
Simple Message Queue (formerly MNS) logs contain operational data for queue and topic messages, covering the entire message lifecycle, such as timestamps, clients, and operations. You can analyze the logs by using real-time queries, real-time computing, and offline computing.
-
Real-time query: You can perform real-time queries in the Log Service console. For example, you can query message traces, the number of sent messages, and the number of deleted messages.
-
Real-time computing: You can use tools like Spark, Storm, StreamCompute, or Consumer Library to perform real-time computing on Simple Message Queue (formerly MNS) logs. For example, you can identify the producers and consumers of the top 10 messages in a queue, calculate message production and consumption rates to check for imbalances, or measure consumer processing latency to identify potential bottlenecks.
-
Offline computing: You can use services like MaxCompute, E-MapReduce, or Hive to perform calculations over long time spans. For example, you can calculate the average latency of messages from publication to consumption over the last week.
Query a queue message trace
Log on to the Simple Log Service console.
In the Projects section, click the one you want.
On the tab, click the logstore you want.
-
Enter a search statement and click Search & Analyze. Then, click 15 Minutes (Relative) to set the time range for the query.
In this example, to query the message trace of a queue message, enter the queue name and MessageId. The search statement must be in the format of $QueueName and $MessageId, such as log and EED287A265726135146E6A9CADC880FA.
The query results trace a message from being sent to being received. The query returns two MNSLogging log entries, both with the MessageId
EED287A265726135146E6A9CADC880FA. One entry has an Action of SendMessage (timestamp: 2020-10-21 16:50:53), and the other has an Action of ReceiveMessage (timestamp: 2020-10-21 16:50:57). This shows the message's complete journey within the queuesgw-express-sync-queue-gw-0008q1m4pv7qjse4na31-5796.
Query the sent count of queue messages
Log on to the Simple Log Service console.
In the Projects section, click the one you want.
On the tab, click the logstore you want.
-
Enter a search statement and click Search & Analyze. Then, click 15 Minutes (Relative) to set the time range for the query.
In this example, to query the number of sent queue messages, enter the queue name and the send operation. The search statement must be in the format of $QueueName and (SendMessage or BatchSendMessage), such as log and (SendMessage or BatchSendMessage).
The query returns three Exact Match log entries. Each log entry includes fields such as Action (for example,
SendMessage, highlighted in red), QueueName, Time, MessageId, RequestId, and RemoteAddress.
Query the consumed count of queue messages
Log on to the Simple Log Service console.
In the Projects section, click the one you want.
On the tab, click the logstore you want.
-
Enter a search statement and click Search & Analyze. Then, click 15 Minutes (Relative) to set the time range for the query.
In this example, to query the number of consumed queue messages, enter the queue name and the consumption operation. The search statement must be in the format of $QueueName and (ReceiveMessage or BatchReceiveMessage), such as log and (ReceiveMessage or BatchReceiveMessage).
The results show that consumers received 12 messages from the log queue within the specified time range. The query returns a total of 12 log entries. Each raw log includes fields such as Action (for example,
ReceiveMessage), QueueName (for example,log), MessageId, RequestId, NextVisibleTime, ProcessTime, and RemoteAddress.
Query the deleted count of queue messages
Log on to the Simple Log Service console.
In the Projects section, click the one you want.
On the tab, click the logstore you want.
-
Enter a search statement and click Search & Analyze. Then, click 15 Minutes (Relative) to set the time range for the query.
In this example, to query the number of deleted queue messages, enter the queue name and the delete operation. The search statement must be in the format of $QueueName and (DeleteMessage or BatchDeleteMessage), such as log and (DeleteMessage or BatchDeleteMessage).
The query returns 61 log entries, which indicates that 61 messages were deleted from the queue within the specified time range.
Query a topic message trace
Log on to the Simple Log Service console.
In the Projects section, click the one you want.
On the tab, click the logstore you want.
-
Enter a search statement and click Search & Analyze. Then, click 15 Minutes (Relative) to set the time range for the query.
In this example, to query the message trace of a topic message, enter the topic name and MessageId. The search statement must be in the format of $TopicName and $MessageId, such as logtest and 8798453B65727FC6433E6AB4F746D4CE.
The query returns two log entries with Action values of PublishMessage and Notify, showing the message was published and then delivered as a notification.
1 10-21 17:17:24 ... MNSLogging 1603271947 ProcessTime :1 TopicName :logtest Time :2020-10-21 17:17:24.679000 AccountId :xxx Action :PublishMessage MessageId :8798453B65727FC6433E6AB4F746D4CE RequestId :5F8FFCA4453241C3099B4BC9 RemoteAddress :xxx MessageTag : 1 10-21 17:17:24 ... MNSLogging 1603271947 ProcessTime :1 TopicName :logtest Time :2020-10-21 17:17:24.679000 AccountId :xxx Action : Notify MessageId :8798453B65727FC6433E6AB4F746D4CE RequestId :5F8FFCA4453241C3099B4BC9 RemoteAddress :xxx MessageTag :
Query the published count of topic messages
Log on to the Simple Log Service console.
In the Projects section, click the one you want.
On the tab, click the logstore you want.
-
Enter a search statement and click Search & Analyze. Then, click 15 Minutes (Relative) to set the time range for the query.
In this example, to query the number of published topic messages, enter the topic name and the publish operation. The search statement must be in the format of $TopicName and PublishMessage, such as logtest and PublishMessage.
The results show that producers published three messages to the logtest topic within the specified time range. The query returns three exact-match log entries. The log details include fields such as TopicName (with the value logtest), Action (with the value PublishMessage), MessageId, RequestId, and RemoteAddress.
Query the message processing count for a client
Log on to the Simple Log Service console.
In the Projects section, click the one you want.
On the tab, click the logstore you want.
-
Enter a search statement and click Search & Analyze. Then, click 15 Minutes (Relative) to set the time range for the query.
In this example, to query the number of messages processed by a specific client, enter the client's IP address (from the RemoteAddress field). The search statement must be in the format of an IP address, such as 10.10.10.5.
To query logs for a specific type of operation for a client, you can combine keywords. For example, use the search statement 10.10.10.5 and (SendMessage or BatchSendMessage).
The results show that the client processed 66 messages within the specified time range.
