The data dashboard displays the following modules: Reception Overview, Knowledge Analysis, Dialog Studio Analysis, Service Capability, and Session History.
Reception overview
You can view the reception status of a selected chatbot for a specific period. You can also see how long the selected chatbot has been active on the Chatbot platform up to the current date.
The following list describes the metrics in the preceding figure:
Robot: Select the robot whose data you want to view.
Time: Select a specific day or a time range. The dashboard displays reception data from the last 90 days, regardless of the selected time range. This setting applies to all subsequent sections.
Receptions: The number of user sessions for the selected chatbot within the selected time range. In the Chatbot console, this metric represents the number of times the test window was opened.
Conversation rounds: The conversation round count increases by one each time a user sends a request to the chatbot, such as asking a question. A session typically consists of multiple conversation rounds. This metric measures the chatbot's conversation volume.
Average conversation rounds: The total number of conversation rounds divided by the number of receptions. This metric indicates the average number of rounds per session and reflects user engagement. A higher average indicates greater engagement.
Resolved receptions: The number of service sessions (receptions), excluding sessions that were transferred to a human agent, had no answer, had unclicked recommendations, or received negative feedback. This metric indicates the chatbot's service capability. A higher number of resolved receptions indicates a stronger service capability.
Unresolved rate: The number of sessions transferred to a human agent, with no answer, with unclicked recommendations, or with negative feedback, divided by the total number of sessions.
No-answer rate: The number of sessions that contain a conversation round with no answer, divided by the total number of sessions. Note: This rate is calculated at the session level. If any conversation round in a session results in no answer, the entire session is counted as having no answer.
Reviews: The number of chats that users reviewed. This data is displayed after the corresponding API is configured.
Likes: The number of chats that users liked. This data is displayed after the corresponding API is configured.
Dislikes: The number of chats that users disliked. This data is displayed after the corresponding API is configured.
Note: To collect data for the review rate, like rate, and dislike rate, you must add like and dislike buttons to the frontend when you develop the client-side dialog box. You also need to call the reserved feedback API to record this information. This process ensures that the field is recorded in the underlying logs.
The following list describes the metrics in the preceding figure:
Chatbot answer type distribution table: Shows the number of conversation rounds for different answer types during the selected time range.
Chatbot answer type distribution chart: Displays conversation data for various answer types during the selected time range. These types include Q&A pairs, recommended Q&A pairs, Dialog Studio, chit-chat, no answer, transfer to human agent, and others.
Reception details: Displays data for the selected time range, including receptions, conversation rounds, average rounds per session, resolved receptions, Q&A pair conversation rounds, Dialog Studio conversation rounds, recommendation conversation rounds, chit-chat conversation rounds, and no-answer conversation rounds. You can download and export this data.
Knowledge analysis
Knowledge analysis provides a statistical analysis of the Q&A data in the knowledge base. It includes the following metrics:
Total Q&A pairs: The total number of active Q&A pairs attached to the specified chatbot. This number is based on the last day of the selected date range. This setting applies to all subsequent sections.
Effective Q&A pair ratio: The number of effective Q&A pairs hit in the last 7 days divided by the total number of active Q&A pairs on the last day of the selected range. If you select a time period, the data from the last day of that period is used. This metric indicates how many Q&A pairs in the knowledge base correspond to actual user questions and measures the effectiveness of the knowledge base design.
Q&A pair hit ratio: The number of conversation rounds where a Q&A pair was hit during the selected date range, divided by the total number of conversation rounds. This metric indicates how well the knowledge base covers user questions.
The following list describes the metrics in the following figure:
Top questions: This section lists the most frequently asked questions during the specified time range, ranked by the number of times a Q&A pair was hit. Administrators can use this list to regularly optimize answers to top questions and ensure the quality of the chatbot.
Unpopular questions: This section lists questions that have not been asked by any user in the last 30 days for the specified chatbot. This list can reflect user query trends. Administrators can use this list to redesign or archive Q&A pairs. You can download the data for analysis. The list displays 2,000 entries, and you can download up to 20,000 entries. You must select a single date in the date filter.
The following list describes the metrics in the following figure:
Q&A pair details: Displays the hit details of the knowledge base based on user questions for the selected chatbot and time range. The list displays 2,000 entries, and you can download up to 20,000 entries.
Keyword: The keyword for a knowledge item.
Knowledge ID: The unique identifier for the knowledge.
Knowledge Name: The name of the knowledge item.
Category: The category to which the knowledge belongs.
Direct hits: The number of conversation rounds that directly hit this Q&A pair.
Recommendation hits: The number of conversation rounds that hit a recommended Q&A pair.
Recommendation click-through rate: The number of recommendation clicks divided by the number of recommendation hits.
Original sound
Knowledge ID
Title
User utterance: The original question asked by the user.
Answer types: Recommended knowledge points and knowledge points
Unclicked recommendation
Dialog Studio analysis
Dialog Studio analysis provides a statistical analysis of the performance of conversation flows in Dialog Studio. The following data is available on the dashboard:
Total conversation flows: The total number of conversation flows for the current chatbot. This number is based on the last day of the selected date range.
Effective conversation flow ratio: The number of effective conversation flows hit in the last 7 days divided by the total number of conversation flows on the last day of the selected range. If you select a time period, the data from the last day of that period is used.
Dialog Studio hit ratio: The number of conversation rounds where Dialog Studio was hit during the selected period, divided by the total number of conversation rounds.
The following list describes the metrics in the following figure:
Top questions: This section lists the most frequently asked questions during the specified time range, ranked by the number of times a conversation flow was hit. Administrators can use this list to regularly optimize answers to top questions and ensure the quality of the chatbot.
Unpopular questions: This section lists questions that have not been asked by any user in the last 30 days for the specified chatbot. This list can reflect user query trends. Administrators can use this list to redesign or archive conversation flows. You can download the data for analysis. The list displays 2,000 entries, and you can download up to 20,000 entries. You must select a single date in the date filter.
Conversation flow details: Displays the trigger details of conversation flows based on user sessions for the selected chatbot and time range. Note: When you view conversation flow node details and original utterances, conversation data is available only if the number of conversation rounds is greater than zero.
Conversation flow name.
Node details.
Conversation flow name.
Node name: The name of the node that was hit in this conversation flow.
Conversation rounds: The number of conversation rounds that hit the node.
Conversation rounds: The number of conversation rounds that hit this conversation flow.
Cases triggered: The number of sessions that hit this conversation flow.
Cases completed: The number of sessions that reached a leaf node or a business completion node.
Completion rate: The number of cases completed divided by the number of cases triggered.
Bounce rate: The number of sessions that did not reach a completion node divided by the number of cases triggered.
Like rate: The number of cases that hit this conversation flow and received a like, divided by the number of cases that hit this conversation flow and were reviewed.
Dislike rate: The number of cases that hit this conversation flow and received a dislike, divided by the number of cases that hit this conversation flow and were reviewed.
Original utterance
Date
Conversation flow name
Node name
User utterance: The original question asked by the user.
Conversation flow bounce: Indicates whether the session reached a leaf node or a business completion node.
Service capability
The following list describes the metrics in the following figure:
No-answer rate: The number of sessions with no answer divided by the total number of sessions. This metric measures the chatbot's ability to cover user questions. A lower no-answer rate indicates higher coverage. Administrators can improve coverage by maintaining the knowledge base or Dialog Studio to add answers for questions that the chatbot cannot yet handle.
Unclicked recommendation rate: The number of sessions where a recommended Q&A pair was not clicked in the final conversation round, divided by the total number of sessions with recommended Q&A pairs. This metric measures the accuracy of the chatbot's recommendations. When a user's question is not directly answered, the chatbot typically recommends the three most relevant Q&A pairs. If the user does not click any recommendation, it suggests that the recommendations did not meet the user's needs.
Improvement plan: This feature uses key words and detailed user utterances to group no-answer questions and unclicked recommendations. This helps administrators analyze coverage gaps in online Q&A. Administrators can then improve the overall quality of the chatbot by enhancing the knowledge base or Dialog Studio.
No-answer clustering: This feature uses a clustering algorithm to group no-answer questions based on user utterances. This helps administrators add and optimize specific Q&A pairs.
Unclicked recommendation clustering: This feature groups user questions that triggered recommendations based on user utterances. This helps administrators update and optimize specific Q&A pairs.
Session history
The following list describes the metrics in the following figure:
Session history details: Displays the conversations between users and the selected chatbot during the specified time range.
Query time: The time when each user query was triggered.
Query content: The original user query.
Answer type: The type of the chatbot's answer, such as Q&A pair, Dialog Studio, recommended Q&A pair, or chit-chat.
session_id: The unique identifier for the session.
chat_id: The unique identifier for the user's question.
Answer content: The content of the answer. For a Q&A pair, this is the Q&A pair title. For Dialog Studio, this is the answer returned by Dialog Studio. For chit-chat, this is the answer returned to the user.
Knowledge ID: The ID of the knowledge.
Knowledge Name: The name of a knowledge entry.
Knowledge Category: The name of the knowledge category.
Conversation flow name: This field has a value if a conversation flow was hit.
Node name: The name of the conversation flow node that was hit in this Q&A round.
Business completion node: The value of 'sessionFinish' in Dialog Studio. This indicates whether the node is a designated business completion node.
Intent type: The type of intent that was hit when a conversation flow was triggered. This is the source of the intent hit.
Intent details: The specific utterance that triggered the intent when a conversation flow was hit.
Intent name: The name of the intent that was hit when a conversation flow was triggered.
Note:
A maximum of 5,000 records are displayed for single-day and multi-day queries.
For multi-day queries, you can view and download the corpus in batches.
The corpus is available for up to 90 days from the log on date.