OpenClaw log analysis with the Skill plugin

更新时间:
复制 MD 格式

The alibabacloud-adb-openclaw-insight Skill connects to an AnalyticDB for MySQL database and performs three-tiered automated analysis on OpenClaw session logs, covering operational metrics, user behavior, and organizational insights.

How it works

L1 - Operational analysis (no LLM required)

L1 uses SQL-based statistics to analyze quantitative operational metrics without an LLM.

Use case

Description

Example

L1-1 token efficiency analysis

Calculates token consumption (input and output), cache hit rates, and average cost per session by user and model.

Identify user token consumption patterns.

L1-2 session depth analysis

Analyzes average turns per task chain to identify deep-interaction sessions.

Identifies users with the most complex tasks. A high turn count indicates greater complexity.

L1-3 toolchain analysis

Tracks tool frequency and combination patterns to discover common tool sequences.

Discover that exec is the most-used tool, indicating frequent script executions.

L1-4 high-cost session identification

Pinpoints sessions with unusually high token consumption or costs to identify cost hotspots.

Identify that a single session from User A accounts for over 50% of total token consumption.

L1-5 Anomaly detection

Uses the Z-score algorithm to flag statistical anomalies in token consumption and response time.

Detect a task stuck in a retry loop, consuming tokens without productive work.

L2 - Behavioral analysis (LLM required)

L2 uses an LLM for in-depth qualitative and quantitative analysis of user behavior and task processes.

Use case

Description

Example

L2-1 Intent classification

Categorizes user queries into intents such as coding, debugging, Q&A, and data analysis.

If "coding" is the top intent and "chat" the least common, most users are focused on development tasks.

L2-2 task complexity assessment

Assigns a complexity score (1–5) to each task chain based on conversation turns, tool usage, and token count.

View task chains ranked by execution complexity.

L2-3 task success rate estimation

Infers whether each task chain achieved its goal based on the final conversation outcome.

Determine that most task chains succeed and identify root causes for failures.

L2-4 prompt quality scoring

Scores user-submitted prompts to identify low-quality input patterns for optimization.

Identify users with high-quality prompts and promote their examples as best practices.

L2-5 topic clustering

Clusters conversation sessions by topic to discover popular areas of interest.

Discover trending AI-related topics that users frequently discuss.

L2-6 retry behavior detection

Detects patterns of repeated retries or follow-up questions within the same session.

Identify task types where agent performance is weakest, providing feedback to the model provider for optimization.

L2-7 reasoning depth analysis

Analyzes the number and depth of the assistant's "Thinking" turns to assess reasoning complexity.

Identifies task types that require the most model reasoning, indicating higher complexity.

L2-8 user proficiency tracking

Classifies users into Beginner, Intermediate, or Advanced levels based on historical interaction patterns.

Gauge overall AI proficiency across your user base.

L3 - Organizational analysis (LLM required)

L3 examines collective behavior patterns across users and sessions to identify organizational knowledge gaps and improvement opportunities.

Use case

Description

Example

L3-1 technology stack heatmap

Generates a technology heatmap by tracking mention frequency of technology stacks, languages, and frameworks.

Discover trending technologies and topics that your users focus on.

L3-2 duplicate question detection

Identifies frequently repeated questions across users and sessions, often indicating documentation or knowledge base gaps.

Pinpoint common user issues and identify processes that are candidates for automation.

L3-3 skill candidate analysis

Recommends scenarios that can be standardized into standalone Skills based on frequent task patterns.

Convert high-frequency tasks into dedicated Skills to reduce token consumption and simplify agent interactions.

L3-4 User profile report

Generates a summary report for each user, covering behavioral characteristics, technology preferences, and usage habits.

Understand each user's AI usage preferences, work complexity, and problem-solving efficiency.

L3-5 final narrative report

Consolidates all L1, L2, and L3 results into a comprehensive Markdown analysis report.

Provides a high-level overview for management on AI usage costs and adoption across the organization.

Note

After each analysis, the Skill generates an insight_logic_explanation.md file that explains the calculation logic for each use case's metrics. The language matches the report's primary language (Chinese for Chinese data, English for English data).

Usage notes

  • Data volume requirements

    • L1 analysis: No minimum data requirement, but at least 1 session record is recommended for meaningful statistics.

    • L2 analysis: At least 50 task chains recommended for valuable intent classification and topic clustering results.

    • L3 analysis: At least 100 task chains recommended for accurate cross-session duplicate question detection.

  • LLM dependencies

    • L1 analysis does not depend on an LLM.

    • L2 and L3 use cases require a configured LLM API. Use cases are skipped if the API is not configured.

    • By default, a maximum of 500 sessions are sent to the LLM for a single analysis. You can adjust this limit by modifying maxSessionsForLlm in the config.json file.

    • A model with a long context window is recommended (32K minimum, 128K+ preferred).

  • Database schema dependencies
    The Skill depends on the following three tables. The init_db script automatically creates or updates these tables and their fields.

    Table name

    Purpose

    openclaw_sessions

    Stores each message in a session (role, content, tokens, etc.).

    openclaw_logs

    Stores runtime logs (optional).

    openclaw_analysis_results

    Stores the results of each analysis.

  • By default, analysis covers the last seven days. Specify a custom time window with the --from and --to flags. Results reflect only data within that window and are not cumulative. The retentionDays parameter (default: 7 days) controls data retention, and older data is automatically deleted.

  • Real-time stream analysis is not supported. All analyses run as batch jobs on historical data.

  • For L2-5 topic clustering, highly dispersed user intents may produce coarse-grained clusters.

  • L3-2 duplicate question detection relies on LLM-inferred semantic similarity. It works best for questions that are identical or very close in wording.

Install and configure

Automatic installation

Install the Skill with a single command from the OpenClaw dialog box. Enter the following command and provide the required configuration when prompted. For detailed instructions, see the Manual deployment tab.

Use the following command to install the alibabacloud-adb-openclaw-insight Skill to your Skill directory:
git clone https://github.com/aliyun/alibabacloud-adb-mysql-mcp-server
cd alibabacloud-adb-mysql-mcp-server/skill/alibabacloud-adb-openclaw-insight
After the download is complete, read SKILL.md, install the dependencies, update the configuration file, and initialize the database.

Manual deployment

Step 1: Clone and prepare environment

git clone https://github.com/aliyun/alibabacloud-adb-mysql-mcp-server
cd alibabacloud-adb-mysql-mcp-server/skill/alibabacloud-adb-openclaw-insight

Step 2: Install project dependencies

  • Option 1: Use uv (recommended)

    # Install the uv package manager
    curl -LsSf https://astral.sh/uv/install.sh | sh
    
    # Install project dependencies
    uv pip install -r requirements.txt
  • Option 2: Use pip

    pip install -r requirements.txt
    # If "pip" is not found, try:
    pip3 install -r requirements.txt

Step 3: Configure connection information

This is the only step that requires manual intervention. Gather the following configuration details before proceeding.

  • AnalyticDB for MySQL connection information

    Parameter

    Description

    Example

    adb.host

    The endpoint for your AnalyticDB for MySQL cluster.

    am-xxxxxxxx.ads.aliyuncs.com

    adb.port

    The database port number.

    3306

    adb.database

    The name of the target database.

    openclaw_adb

    adb.username

    The database username.

    your_user

    adb.password

    The database password.

    your_password

  • LLM configuration

    Note

    If you only need L1 operational analysis, you can skip this section.

    Parameter

    Description

    Example

    llm.endpoint

    The service endpoint for the LLM API.

    https://dashscope.aliyuncs.com/compatible-mode/v1

    llm.apiKey

    Your API key.

    sk-xxxxxxxxxx

    llm.model

    The model name.

    qwen-plus

    llm.apiType

    The API protocol type (openai or anthropic).

    openai

  • (Optional) Analysis module toggles

    Parameter

    Description

    Default

    analysis.enableL1

    Enable L1 analysis.

    true

    analysis.enableL2

    Enable L2 analysis (requires LLM).

    true

    analysis.enableL3

    Enable L3 analysis (requires LLM).

    true

    analysis.analysisWindowDays

    The analysis time window in days.

    7

Step 4: Write the configuration file

Generate the config.json file from the config.example.json template.

cp config.example.json config.json

Populate config.json with your configuration details. The final file structure should look like this:

{
  "adb": {
    "host": "<your-provided-host>",
    "port": 3306,
    "database": "<your-provided-database>",
    "username": "<your-provided-username>",
    "password": "<your-provided-password>",
    "sessionTable": "openclaw_sessions",
    "logsTable": "openclaw_logs",
    "connectionPoolSize": 5
  },
  "collection": {
    "intervalMinutes": 5,
    "batchSize": 100,
    "retentionDays": 7,
    "enableLogCollection": true,
    "enableTokenCollection": true
  },
  "filters": {
    "minLevel": "info",

    "includeSubsystems": [ ],

    "excludeSubsystems": ["verbose", "trace"]
  },
  "llm": {
    "endpoint": "<your-llm-endpoint-or-leave-empty>",
    "apiKey": "<your-llm-apikey-or-leave-empty>",
    "model": "<your-llm-model>",
    "apiType": "<your-llm-api-type>",
    "maxConcurrency": 5,
    "temperature": 0.1,
    "maxTokens": 128000
  },
  "analysis": {
    "enableL1": true,
    "enableL2": true,
    "enableL3": true,
    "analysisWindowDays": 7,
    "maxSessionsForLlm": 500
  }
}

Step 5: Initialize the database

This command creates or updates the required table schema in your AnalyticDB for MySQL cluster.

python -m scripts.init_db
# If "python" is not found, try:
python3 -m scripts.init_db

Run and verify

Start the service

  • Option 1: Start in the foreground
    Start the service in the foreground for quick testing.

    python -m scripts.main
    # If "python" is not found, try:
    python3 -m scripts.main

    The service automatically collects OpenClaw session logs and runs scheduled analyses.

  • Option 2: Register as a cron job (Recommended for production)
    Register the collection script as an OpenClaw cron job for continuous automatic data collection.

    {
      "cron": "*/30 * * * * *",
      "command": "python -m scripts.main collect",
      "cwd": "/path/to/alibabacloud-adb-mysql-mcp-server/skill/alibabacloud-adb-openclaw-insight"
    }

Manually trigger analysis

After data collection runs, manually trigger analysis at any time.

# Analyze the past 1 day
python -m scripts.analyze_usage --from "$(date -d '1 day ago' '+%Y-%m-%d') 00:00:00" --to "$(date '+%Y-%m-%d') 23:59:59"

# Analyze the past 2 days
python -m scripts.analyze_usage --from "$(date -d '2 days ago' '+%Y-%m-%d') 00:00:00" --to "$(date '+%Y-%m-%d') 23:59:59"

Verify the installation

Run the following SQL queries to verify that data collection works correctly.

SELECT COUNT(*) FROM openclaw_sessions;
SELECT COUNT(*) FROM openclaw_logs;

If both queries return a count greater than 0, the Skill is installed correctly and collecting data.

Usage example

After mounting the Skill, enter a command. The system recognizes it and returns structured analysis results.

  • Enter the following in the command-line interface:

    /alibabacloud-adb-openclaw-insight Analyze user OpenClaw usage for the last month.
  • Analysis results

    image