OpenClaw log analysis with the Skill plugin
The alibabacloud-adb-openclaw-insight Skill connects to an AnalyticDB for MySQL database and performs three-tiered automated analysis on OpenClaw session logs, covering operational metrics, user behavior, and organizational insights.
How it works
L1 - Operational analysis (no LLM required)
L1 uses SQL-based statistics to analyze quantitative operational metrics without an LLM.
|
Use case |
Description |
Example |
|
L1-1 token efficiency analysis |
Calculates token consumption (input and output), cache hit rates, and average cost per session by user and model. |
Identify user token consumption patterns. |
|
L1-2 session depth analysis |
Analyzes average turns per task chain to identify deep-interaction sessions. |
Identifies users with the most complex tasks. A high turn count indicates greater complexity. |
|
L1-3 toolchain analysis |
Tracks tool frequency and combination patterns to discover common tool sequences. |
Discover that |
|
L1-4 high-cost session identification |
Pinpoints sessions with unusually high token consumption or costs to identify cost hotspots. |
Identify that a single session from User A accounts for over 50% of total token consumption. |
|
L1-5 Anomaly detection |
Uses the Z-score algorithm to flag statistical anomalies in token consumption and response time. |
Detect a task stuck in a retry loop, consuming tokens without productive work. |
L2 - Behavioral analysis (LLM required)
L2 uses an LLM for in-depth qualitative and quantitative analysis of user behavior and task processes.
|
Use case |
Description |
Example |
|
L2-1 Intent classification |
Categorizes user queries into intents such as coding, debugging, Q&A, and data analysis. |
If "coding" is the top intent and "chat" the least common, most users are focused on development tasks. |
|
L2-2 task complexity assessment |
Assigns a complexity score (1–5) to each task chain based on conversation turns, tool usage, and token count. |
View task chains ranked by execution complexity. |
|
L2-3 task success rate estimation |
Infers whether each task chain achieved its goal based on the final conversation outcome. |
Determine that most task chains succeed and identify root causes for failures. |
|
L2-4 prompt quality scoring |
Scores user-submitted prompts to identify low-quality input patterns for optimization. |
Identify users with high-quality prompts and promote their examples as best practices. |
|
L2-5 topic clustering |
Clusters conversation sessions by topic to discover popular areas of interest. |
Discover trending AI-related topics that users frequently discuss. |
|
L2-6 retry behavior detection |
Detects patterns of repeated retries or follow-up questions within the same session. |
Identify task types where agent performance is weakest, providing feedback to the model provider for optimization. |
|
L2-7 reasoning depth analysis |
Analyzes the number and depth of the assistant's "Thinking" turns to assess reasoning complexity. |
Identifies task types that require the most model reasoning, indicating higher complexity. |
|
L2-8 user proficiency tracking |
Classifies users into Beginner, Intermediate, or Advanced levels based on historical interaction patterns. |
Gauge overall AI proficiency across your user base. |
L3 - Organizational analysis (LLM required)
L3 examines collective behavior patterns across users and sessions to identify organizational knowledge gaps and improvement opportunities.
|
Use case |
Description |
Example |
|
L3-1 technology stack heatmap |
Generates a technology heatmap by tracking mention frequency of technology stacks, languages, and frameworks. |
Discover trending technologies and topics that your users focus on. |
|
L3-2 duplicate question detection |
Identifies frequently repeated questions across users and sessions, often indicating documentation or knowledge base gaps. |
Pinpoint common user issues and identify processes that are candidates for automation. |
|
L3-3 skill candidate analysis |
Recommends scenarios that can be standardized into standalone Skills based on frequent task patterns. |
Convert high-frequency tasks into dedicated Skills to reduce token consumption and simplify agent interactions. |
|
L3-4 User profile report |
Generates a summary report for each user, covering behavioral characteristics, technology preferences, and usage habits. |
Understand each user's AI usage preferences, work complexity, and problem-solving efficiency. |
|
L3-5 final narrative report |
Consolidates all L1, L2, and L3 results into a comprehensive Markdown analysis report. |
Provides a high-level overview for management on AI usage costs and adoption across the organization. |
After each analysis, the Skill generates an insight_logic_explanation.md file that explains the calculation logic for each use case's metrics. The language matches the report's primary language (Chinese for Chinese data, English for English data).
Usage notes
-
Data volume requirements
-
L1 analysis: No minimum data requirement, but at least 1 session record is recommended for meaningful statistics.
-
L2 analysis: At least 50 task chains recommended for valuable intent classification and topic clustering results.
-
L3 analysis: At least 100 task chains recommended for accurate cross-session duplicate question detection.
-
-
LLM dependencies
-
L1 analysis does not depend on an LLM.
-
L2 and L3 use cases require a configured LLM API. Use cases are skipped if the API is not configured.
-
By default, a maximum of 500 sessions are sent to the LLM for a single analysis. You can adjust this limit by modifying
maxSessionsForLlmin theconfig.jsonfile. -
A model with a long context window is recommended (32K minimum, 128K+ preferred).
-
-
Database schema dependencies
The Skill depends on the following three tables. Theinit_dbscript automatically creates or updates these tables and their fields.Table name
Purpose
openclaw_sessionsStores each message in a session (role, content, tokens, etc.).
openclaw_logsStores runtime logs (optional).
openclaw_analysis_resultsStores the results of each analysis.
-
By default, analysis covers the last seven days. Specify a custom time window with the
--fromand--toflags. Results reflect only data within that window and are not cumulative. TheretentionDaysparameter (default: 7 days) controls data retention, and older data is automatically deleted. -
Real-time stream analysis is not supported. All analyses run as batch jobs on historical data.
-
For L2-5 topic clustering, highly dispersed user intents may produce coarse-grained clusters.
-
L3-2 duplicate question detection relies on LLM-inferred semantic similarity. It works best for questions that are identical or very close in wording.
Install and configure
Automatic installation
Install the Skill with a single command from the OpenClaw dialog box. Enter the following command and provide the required configuration when prompted. For detailed instructions, see the Manual deployment tab.
Use the following command to install the alibabacloud-adb-openclaw-insight Skill to your Skill directory:
git clone https://github.com/aliyun/alibabacloud-adb-mysql-mcp-server
cd alibabacloud-adb-mysql-mcp-server/skill/alibabacloud-adb-openclaw-insight
After the download is complete, read SKILL.md, install the dependencies, update the configuration file, and initialize the database.
Manual deployment
Step 1: Clone and prepare environment
git clone https://github.com/aliyun/alibabacloud-adb-mysql-mcp-server
cd alibabacloud-adb-mysql-mcp-server/skill/alibabacloud-adb-openclaw-insight
Step 2: Install project dependencies
-
Option 1: Use uv (recommended)
# Install the uv package manager curl -LsSf https://astral.sh/uv/install.sh | sh # Install project dependencies uv pip install -r requirements.txt -
Option 2: Use pip
pip install -r requirements.txt # If "pip" is not found, try: pip3 install -r requirements.txt
Step 3: Configure connection information
This is the only step that requires manual intervention. Gather the following configuration details before proceeding.
-
AnalyticDB for MySQL connection information
Parameter
Description
Example
adb.hostThe endpoint for your AnalyticDB for MySQL cluster.
am-xxxxxxxx.ads.aliyuncs.comadb.portThe database port number.
3306adb.databaseThe name of the target database.
openclaw_adbadb.usernameThe database username.
your_useradb.passwordThe database password.
your_password -
LLM configuration
NoteIf you only need L1 operational analysis, you can skip this section.
Parameter
Description
Example
llm.endpointThe service endpoint for the LLM API.
https://dashscope.aliyuncs.com/compatible-mode/v1llm.apiKeyYour API key.
sk-xxxxxxxxxxllm.modelThe model name.
qwen-plusllm.apiTypeThe API protocol type (
openaioranthropic).openai -
(Optional) Analysis module toggles
Parameter
Description
Default
analysis.enableL1Enable L1 analysis.
trueanalysis.enableL2Enable L2 analysis (requires LLM).
trueanalysis.enableL3Enable L3 analysis (requires LLM).
trueanalysis.analysisWindowDaysThe analysis time window in days.
7
Step 4: Write the configuration file
Generate the config.json file from the config.example.json template.
cp config.example.json config.json
Populate config.json with your configuration details. The final file structure should look like this:
{
"adb": {
"host": "<your-provided-host>",
"port": 3306,
"database": "<your-provided-database>",
"username": "<your-provided-username>",
"password": "<your-provided-password>",
"sessionTable": "openclaw_sessions",
"logsTable": "openclaw_logs",
"connectionPoolSize": 5
},
"collection": {
"intervalMinutes": 5,
"batchSize": 100,
"retentionDays": 7,
"enableLogCollection": true,
"enableTokenCollection": true
},
"filters": {
"minLevel": "info",
"includeSubsystems": [ ],
"excludeSubsystems": ["verbose", "trace"]
},
"llm": {
"endpoint": "<your-llm-endpoint-or-leave-empty>",
"apiKey": "<your-llm-apikey-or-leave-empty>",
"model": "<your-llm-model>",
"apiType": "<your-llm-api-type>",
"maxConcurrency": 5,
"temperature": 0.1,
"maxTokens": 128000
},
"analysis": {
"enableL1": true,
"enableL2": true,
"enableL3": true,
"analysisWindowDays": 7,
"maxSessionsForLlm": 500
}
}
Step 5: Initialize the database
This command creates or updates the required table schema in your AnalyticDB for MySQL cluster.
python -m scripts.init_db
# If "python" is not found, try:
python3 -m scripts.init_db
Run and verify
Start the service
-
Option 1: Start in the foreground
Start the service in the foreground for quick testing.python -m scripts.main # If "python" is not found, try: python3 -m scripts.mainThe service automatically collects OpenClaw session logs and runs scheduled analyses.
-
Option 2: Register as a cron job (Recommended for production)
Register the collection script as an OpenClaw cron job for continuous automatic data collection.{ "cron": "*/30 * * * * *", "command": "python -m scripts.main collect", "cwd": "/path/to/alibabacloud-adb-mysql-mcp-server/skill/alibabacloud-adb-openclaw-insight" }
Manually trigger analysis
After data collection runs, manually trigger analysis at any time.
# Analyze the past 1 day
python -m scripts.analyze_usage --from "$(date -d '1 day ago' '+%Y-%m-%d') 00:00:00" --to "$(date '+%Y-%m-%d') 23:59:59"
# Analyze the past 2 days
python -m scripts.analyze_usage --from "$(date -d '2 days ago' '+%Y-%m-%d') 00:00:00" --to "$(date '+%Y-%m-%d') 23:59:59"
Verify the installation
Run the following SQL queries to verify that data collection works correctly.
SELECT COUNT(*) FROM openclaw_sessions;
SELECT COUNT(*) FROM openclaw_logs;
If both queries return a count greater than 0, the Skill is installed correctly and collecting data.
Usage example
After mounting the Skill, enter a command. The system recognizes it and returns structured analysis results.
-
Enter the following in the command-line interface:
/alibabacloud-adb-openclaw-insight Analyze user OpenClaw usage for the last month. -
Analysis results
