Mobile Use

开放能力概览

工具

名称

描述

支持说明

OSS

oss_env_init

创建并初始化 OSS 环境变量,指定 Endpoint、Access Key ID、Access Key Secret、Security Token 和 Region。临时安全凭证通过 STS(Security Token Service)获取,更多信息请参见使用STS临时访问凭证访问OSS

支持

oss_upload

将本地文件或目录上传到指定的 OSS Bucket。若上传目录,会先压缩为ZIP文件再上传。可指定 OSS 中的对象名称,若未指定,则默认使用文件或 ZIP 名称。

说明

使用此工具前必须调用oss_env_init工具初始化 OSS 环境变量。

支持

oss_download

从指定的 OSS Bucket 下载对象到本地路径。若父目录不存在,会自动创建。若目标文件已存在,会被覆盖。

说明

使用此工具前必须调用oss_env_init工具初始化 OSS 环境变量。

支持

oss_upload_annon

使用 HTTP PUT 将本地文件或目录上传到指定 URL。若上传目录,会先压缩为 ZIP 文件再上传。若目标已存在,会被覆盖。

支持

oss_download_annon

从指定 URL 下载文件到本地路径。若父目录不存在,会自动创建。若目标文件已存在,会被覆盖。

支持

UI

click

在屏幕的特定坐标处点击。

支持

input_text

输入文本。

支持

send_key

发送按键。支持 Android 平台按键:

  • 3:HOME

  • 4:BACK

  • 24:VOLUME_UP

  • 25:VOLUME_DOWN

  • 26:POWER

  • 82:MENU

支持

swipe

在屏幕上执行滑动手势。

支持

get_all_ui_elements

在超时时间内获取设备上的所有 UI 元素(包括非交互元素)。

支持

get_clickable_ui_elements

在超时时间内获取所有可点击的 UI 元素。

支持

App

get_installed_apps

检索系统中安装的应用程序列表。支持通过开始菜单条目和桌面快捷方式筛选,并可选择排除系统应用。返回应用程序详细信息,包括名称、启动命令、可选的停止命令和工作目录。

支持

start_app

使用提供的命令和可选工作目录启动指定应用程序。返回与启动应用程序关联的进程列表,包括进程名称、PID 和启动命令。

支持

stop_app_by_cmd

使用提供的停止命令终止应用程序。请谨慎使用,因为这将强制终止指定进程。

支持

Shell

shell

在 Android 平台上执行 shell 命令并返回输出或错误。

支持

OSS

名称

描述

参数

oss_env_init

创建并初始化 OSS 环境变量,指定 Endpoint、Access Key ID、Access Key Secret、Security Token 和 Region。临时安全凭证通过 STS(Security Token Service)获取,更多信息请参见使用STS临时访问凭证访问OSS

{
  "inputSchema": {
    "properties": {
      "access_key_id": {
        "description": "OSS 身份验证的访问密钥 ID",
        "type": "string"
      },
      "access_key_secret": {
        "description": "OSS 身份验证的访问密钥 Secret",
        "type": "string"
      },
      "endpoint": {
        "description": "OSS 服务端点,例如:若未指定,默认为 https://oss-cn-hangzhou.aliyuncs.com",
        "type": "string"
      },
      "region": {
        "description": "OSS 区域,例如:cn-hangzhou。若未指定,默认为 cn-hangzhou",
        "type": "string"
      },
      "security_token": {
        "description": "OSS 身份验证的安全令牌",
        "type": "string"
      }
    },
    "required": ["access_key_id", "access_key_secret", "security_token"],
    "type": "object"
  }
}

oss_upload

将本地文件或目录上传到指定的 OSS Bucket。若上传目录,会先压缩为ZIP文件再上传。可指定 OSS 中的对象名称,若未指定,则默认使用文件或 ZIP 名称。

说明

使用此工具前必须调用oss_env_init工具初始化 OSS 环境变量。

{
  "inputSchema": {
    "properties": {
      "bucket": {
        "description": "OSS Bucket 名称",
        "type": "string"
      },
      "object": {
        "description": "OSS Bucket 中的对象路径,例如:test/test.txt",
        "type": "string"
      },
      "path": {
        "description": "要上传的本地文件或非空目录的完整路径,例如:/tmp/test.txt /tmp(Linux)或 C:/tmp/test.txt C:/tmp(Windows)",
        "type": "string"
      }
    },
    "required": ["bucket", "object", "path"],
    "type": "object"
  }
}

oss_download

从指定的 OSS Bucket 下载对象到本地路径。若父目录不存在,会自动创建。若目标文件已存在,会被覆盖。

说明

使用此工具前必须调用oss_env_init工具初始化 OSS 环境变量。

{
  "inputSchema": {
    "properties": {
      "bucket": {
        "description": "OSS Bucket 名称",
        "type": "string"
      },
      "object": {
        "description": "OSS Bucket 中的对象路径,例如:test/test.txt",
        "type": "string"
      },
      "path": {
        "description": "保存下载文件的本地完整路径,例如:/tmp/test.txt(Linux)或 C:/tmp/test.txt(Windows)",
        "type": "string"
      }
    },
    "required": ["bucket", "object", "path"],
    "type": "object"
  }
}

oss_upload_annon

使用 HTTP PUT 将本地文件或目录上传到指定 URL。若上传目录,会先压缩为 ZIP 文件再上传。若目标已存在,会被覆盖。

{
  "inputSchema": {
    "properties": {
      "path": {
        "description": "要上传的本地文件或非空目录的完整路径,例如:/tmp/test.txt /tmp(Linux)或 C:/tmp/test.txt  C:/tmp(Windows)",
        "type": "string"
      },
      "url": {
        "description": "上传文件的 HTTP/HTTPS URL",
        "type": "string"
      }
    },
    "required": ["url", "path"],
    "type": "object"
  }
}

oss_download_annon

从指定 URL 下载文件到本地路径。若父目录不存在,会自动创建。若目标文件已存在,会被覆盖。

{
  "inputSchema": {
    "properties": {
      "path": {
        "description": "保存下载文件的本地完整路径,例如:/tmp/test.txt(Linux)或 C:/tmp/test.txt(Windows)",
        "type": "string"
      },
      "url": {
        "description": "下载文件的 HTTP/HTTPS URL",
        "type": "string"
      }
    },
    "required": ["url", "path"],
    "type": "object"
  }
}

UI

名称

描述

参数

click

在屏幕的特定坐标处点击。

{
  "inputSchema": {
    "properties": {
      "button": {
        "description": "按钮类型。可选值:left, middle, right,默认:left",
        "type": "string"
      },
      "x": {
        "description": "X 坐标",
        "type": "integer"
      },
      "y": {
        "description": "Y 坐标",
        "type": "integer"
      }
    },
    "required": ["x", "y", "button"],
    "type": "object"
  }
}

input_text

输入文本。

{
  "inputSchema": {
    "properties": {
      "text": {
        "description": "客户端输入的文本",
        "type": "string"
      }
    },
    "required": ["text"],
    "type": "object"
  }
}

send_key

发送按键。支持 Android 平台按键:

  • 3:HOME

  • 4:BACK

  • 24:VOLUME_UP

  • 25:VOLUME_DOWN

  • 26:POWER

  • 82:MENU

{
  "inputSchema": {
    "properties": {
      "key": {
        "description": "客户端发送的按键",
        "type": "integer"
      }
    },
    "required": ["key"],
    "type": "object"
  }
}

swipe

在屏幕上执行滑动手势。

{
  "inputSchema": {
    "properties": {
      "duration_ms": {
        "default": 300,
        "description": "滑动持续时间(毫秒),默认:300",
        "type": "integer"
      },
      "end_x": {
        "description": "结束 X 坐标",
        "type": "integer"
      },
      "end_y": {
        "description": "结束 Y 坐标",
        "type": "integer"
      },
      "start_x": {
        "description": "起始 X 坐标",
        "type": "integer"
      },
      "start_y": {
        "description": "起始 Y 坐标",
        "type": "integer"
      }
    },
    "required": ["start_x", "start_y", "end_x", "end_y", "duration_ms"],
    "type": "object"
  }
}

get_all_ui_elements

在超时时间内获取设备上的所有 UI 元素(包括非交互元素)。

{
  "inputSchema": {
    "properties": {
      "timeout_ms": {
        "default": 1000,
        "description": "命令执行超时时间(单位:毫秒)。若未指定,默认值(例如 1000 毫秒)",
        "type": "integer"
      }
    },
    "required": ["timeout_ms"],
    "type": "object"
  }
}

get_clickable_ui_elements

在超时时间内获取所有可点击的 UI 元素。

{
  "inputSchema": {
    "properties": {
      "timeout_ms": {
        "default": 1000,
        "description": "命令执行超时时间(单位:毫秒)。若未指定,默认值(例如 1000 毫秒)",
        "type": "integer"
      }
    },
    "required": ["timeout_ms"],
    "type": "object"
  }
}

App

名称

描述

参数

get_installed_apps

检索系统中安装的应用程序列表。支持通过开始菜单条目和桌面快捷方式筛选,并可选择排除系统应用。返回应用程序详细信息,包括名称、启动命令、可选的停止命令和工作目录。

{
  "inputSchema": {
    "properties": {
      "desktop": {
        "default": false,
        "description": "包含桌面快捷方式(默认:false)",
        "type": "boolean"
      },
      "ignore_system_app": {
        "default": true,
        "description": "排除系统应用(默认:true)",
        "type": "boolean"
      },
      "start_menu": {
        "default": true,
        "description": "包含开始菜单应用(默认:true)",
        "type": "boolean"
      }
    },
    "required": [],
    "type": "object"
  }
}

start_app

使用提供的命令和可选工作目录启动指定应用程序。返回与启动应用程序关联的进程列表,包括进程名称、PID 和启动命令。

{
  "inputSchema": {
    "properties": {
      "activity": {
        "description": "Android 上的活动名称,例如 .xxActivity 或 com.xx/.xxActivity。",
        "type": "string"
      },
      "start_cmd": {
        "description": "启动应用程序的命令。在 Android 上,使用 'monkey -p ' 语法(例如:'monkey -p <package_name> -c android.intent.category.LAUNCHER 1')。",
        "type": "string"
      },
      "work_directory": {
        "default": "",
        "description": "启动应用程序的工作目录。若省略,将使用默认目录。",
        "type": "string"
      }
    },
    "required": ["start_cmd"],
    "type": "object"
  }
}

stop_app_by_cmd

使用提供的停止命令终止应用程序。请谨慎使用,因为这将强制终止指定进程。

{
  "inputSchema": {
    "properties": {
      "stop_cmd": {
        "description": "用于终止应用程序的命令。",
        "type": "string"
      }
    },
    "required": ["stop_cmd"],
    "type": "object"
  }
}

Shell

名称

描述

参数

shell

在 Android 平台上执行 shell 命令并返回输出或错误。

{
  "inputSchema": {
    "properties": {
      "command": {
        "description": "客户端输入的命令",
        "type": "string"
      },
      "timeout_ms": {
        "default": 1000,
        "description": "命令执行超时时间(单位:毫秒)。若未指定,默认值(例如 1000 毫秒)",
        "type": "integer"
      }
    },
    "required": ["command", "timeout_ms"],
    "type": "object"
  }
}

MCP Tool List

{
  "wuying_oss": {
    "tools": [
      {
        "name": "oss_env_init",
        "description": "Create and initialize OSS environment variables with the specified endpoint, access key ID, access key secret, security token, and region. The temporary security credentials obtained from the STS (Security Token Service). For more information, see: https://help.aliyun.com/zh/oss/developer-reference/use-temporary-access-credentials-provided-by-sts-to-access-oss?spm=a2c4g.11186623.help-menu-search-31815.d_1#9ab17afd7cs4t .",
        "inputSchema": {
          "properties": {
            "access_key_id": {
              "description": "The Access Key ID for OSS authentication",
              "type": "string"
            },
            "access_key_secret": {
              "description": "The Access Key Secret for OSS authentication",
              "type": "string"
            },
            "endpoint": {
              "description": "The OSS service endpoint, e.g.,  If not specified, the default is https://oss-cn-hangzhou.aliyuncs.com",
              "type": "string"
            },
            "region": {
              "description": "The OSS region, e.g., cn-hangzhou. If not specified, the default is cn-hangzhou",
              "type": "string"
            },
            "security_token": {
              "description": "The Security Token for OSS authentication",
              "type": "string"
            }
          },
          "required": [
            "access_key_id",
            "access_key_secret",
            "security_token"
          ],
          "type": "object"
        }
      },
      {
        "name": "oss_upload",
        "description": "Upload a local file or directory to the specified OSS bucket. If a directory is specified, it will be compressed into a ZIP file before uploading. The object name in OSS can be specified; if not, the file or ZIP name will be used by default. Note: You must call the oss_env_init tool to initialize OSS environment variables before using this tool.",
        "inputSchema": {
          "properties": {
            "bucket": {
              "description": "OSS bucket name",
              "type": "string"
            },
            "object": {
              "description": "Object path in OSS bucket, e.g., test/test.txt",
              "type": "string"
            },
            "path": {
              "description": "Local file or not empty directory full path to upload, e.g., /tmp/test.txt /tmp on Linux or C:/tmp/test.txt C:/tmp on Windows",
                            "type": "string"
                        }
                    },
                    "required": [
                        "bucket",
                        "object",
                        "path"
                    ],
                    "type": "object"
                }
            },
            {
                "name": "oss_download",
                "description": "Download an object from the specified OSS bucket to the given local path. If the parent directory does not exist, it will be created automatically. If the target file already exists, it will be overwritten. Note: You must call the oss_env_init tool to initialize OSS environment variables before using this tool.",
                "inputSchema": {
                    "properties": {
                        "bucket": {
                            "description": "OSS bucket name",
                            "type": "string"
                        },
                        "object": {
                            "description": "Object path in OSS bucket, e.g., test/test.txt",
                            "type": "string"
                        },
                        "path": {
                            "description": "Local full path to save the downloaded file, e.g., /tmp/test.txt on Linux or C:/tmp/test.txt on Windows",
                            "type": "string"
                        }
                    },
                    "required": [
                        "bucket",
                        "object",
                        "path"
                    ],
                    "type": "object"
                }
            },
            {
                "name": "oss_upload_annon",
                "description": "Upload a local file or directory to the specified URL using HTTP PUT. If a directory is specified, it will be compressed into a ZIP file before uploading. If the upload target already exists, it will be overwritten.",
                "inputSchema": {
                    "properties": {
                        "path": {
                            "description": "Local file or not empty directory full path to upload, e.g., /tmp/test.txt /tmp on Linux or C:/tmp/test.txt  C:/tmp on Windows",
                            "type": "string"
                        },
                        "url": {
                            "description": "The HTTP/HTTPS URL to upload the file to",
                            "type": "string"
                        }
                    },
                    "required": [
                        "url",
                        "path"
                    ],
                    "type": "object"
                }
            },
            {
                "name": "oss_download_annon",
                "description": "Download a file from the specified URL to the given local path. If the parent directory does not exist, it will be created automatically. If the target file already exists, it will be overwritten.",
                "inputSchema": {
                    "properties": {
                        "path": {
                            "description": "The full local file path to save the downloaded file, e.g., /tmp/test.txt on Linux or C:/tmp/test.txt on Windows",
                            "type": "string"
                        },
                        "url": {
                            "description": "The HTTP/HTTPS URL to download the file from",
                            "type": "string"
                        }
                    },
                    "required": [
                        "url",
                        "path"
                    ],
                    "type": "object"
                }
            }
        ]
    },
    "wuying_ui": {
        "tools": [
            {
                "name": "click",
                "description": "Click on the screen at specific coordinates.",
                "inputSchema": {
                    "properties": {
                        "button": {
                            "description": "button type,Available values:left,middle,right,default:left",
                            "type": "string"
                        },
                        "x": {
                            "description": "X coordinate",
                            "type": "integer"
                        },
                        "y": {
                            "description": "Y coordinate",
                            "type": "integer"
                        }
                    },
                    "required": [
                        "x",
                        "y",
                        "button"
                    ],
                    "type": "object"
                }
            },
            {
                "name": "input_text",
                "description": "Input text",
                "inputSchema": {
                    "properties": {
                        "text": {
                            "description": "client input text",
                            "type": "string"
                        }
                    },
                    "required": [
                        "text"
                    ],
                    "type": "object"
                }
            },
            {
                "name": "send_key",
                "description": "Send a key.Supported Keys on Android Platform: 3:HOME,4:BACK,24:VOLUME_UP,25:VOLUME_DOWN,26:POWER,82:MENU",
                "inputSchema": {
                    "properties": {
                        "key": {
                            "description": "client send key",
                            "type": "integer"
                        }
                    },
                    "required": [
                        "key"
                    ],
                    "type": "object"
                }
            },
            {
                "name": "swipe",
                "description": "Perform a swipe gesture on the screen.",
                "inputSchema": {
                    "properties": {
                        "duration_ms": {
                            "default": 300,
                            "description": "Duration of swipe in milliseconds,default:300",
                            "type": "integer"
                        },
                        "end_x": {
                            "description": "Ending X coordinate",
                            "type": "integer"
                        },
                        "end_y": {
                            "description": "Ending Y coordinate",
                            "type": "integer"
                        },
                        "start_x": {
                            "description": "Starting X coordinate",
                            "type": "integer"
                        },
                        "start_y": {
                            "description": "Starting Y coordinate",
                            "type": "integer"
                        }
                    },
                    "required": [
                        "start_x",
                        "start_y",
                        "end_x",
                        "end_y",
                        "duration_ms"
                    ],
                    "type": "object"
                }
            },
            {
                "name": "get_all_ui_elements",
                "description": "Get all UI elements from the device with timeout, including non-interactive elements.",
                "inputSchema": {
                    "properties": {
                        "timeout_ms": {
                            "default": 1000,
                            "description": "Command execution timeout (unit: milliseconds). If not specified, the default value (such as 1000 milliseconds) is used",
                            "type": "integer"
                        }
                    },
                    "required": [
                        "timeout_ms"
                    ],
                    "type": "object"
                }
            },
            {
                "name": "get_clickable_ui_elements",
                "description": "Get all clickable UI elements within timeout.",
                "inputSchema": {
                    "properties": {
                        "timeout_ms": {
                            "default": 1000,
                            "description": "Command execution timeout (unit: milliseconds). If not specified, the default value (such as 1000 milliseconds) is used",
                            "type": "integer"
                        }
                    },
                    "required": [
                        "timeout_ms"
                    ],
                    "type": "object"
                }
            }
        ]
    },
    "wuying_app": {
        "tools": [
            {
                "name": "get_installed_apps",
                "description": "Retrieve a list of installed applications on the system. Supports filtering by Start Menu entries and Desktop shortcuts, with an option to exclude system applications. Returns application details including name, start command, optional stop command, and working directory.",
                "inputSchema": {
                    "properties": {
                        "desktop": {
                            "default": false,
                            "description": "Include Desktop shortcuts (default: false)",
                            "type": "boolean"
                        },
                        "ignore_system_app": {
                            "default": true,
                            "description": "Exclude system applications (default: true)",
                            "type": "boolean"
                        },
                        "start_menu": {
                            "default": true,
                            "description": "Include Start Menu applications (default: true)",
                            "type": "boolean"
                        }
                    },
                    "required": [],
                    "type": "object"
                }
            },
            {
                "name": "start_app",
                "description": "Start a specified application using the provided command and optional working directory. Returns a list of processes associated with the launched application, including their process names, PIDs, and startup commands.\n\n",
                "inputSchema": {
                    "properties": {
                        "activity": {
                            "description": "The activity name on android,like .xxActivity or com.xx/.xxActivity.",
                            "type": "string"
                        },
                        "start_cmd": {
                            "description": "The command to start the application. On Android, use the 'monkey -p ' syntax (e.g., 'monkey -p <package_name> -c android.intent.category.LAUNCHER 1').",
                            "type": "string"
                        },
                        "work_directory": {
                            "default": "",
                            "description": "The directory from which the application should be launched. If omitted, the default directory is used.",
                            "type": "string"
                        }
                    },
                    "required": [
                        "start_cmd"
                    ],
                    "type": "object"
                }
            },
            {
                "name": "stop_app_by_cmd",
                "description": "Terminate an application using the provided stop command. Use with caution as this will forcefully terminate the specified process.",
                "inputSchema": {
                    "properties": {
                        "stop_cmd": {
                            "description": "The command used to terminate the application.",
                            "type": "string"
                        }
                    },
                    "required": [
                        "stop_cmd"
                    ],
                    "type": "object"
                }
            }
        ]
    },
    "wuying_shell": {
        "tools": [
            {
                "name": "shell",
                "description": "Executes an shell command with timeout and returns the output or an error on android platform.",
                "inputSchema": {
                    "properties": {
                        "command": {
                            "description": "client input command",
                            "type": "string"
                        },
                        "timeout_ms": {
                            "default": 1000,
                            "description": "Command execution timeout (unit: milliseconds). If not specified, the default value (such as 1000 milliseconds) is used",
                            "type": "integer"
                        }
                    },
                    "required": [
                        "command",
                        "timeout_ms"
                    ],
                    "type": "object"
                }
            }
        ]
    }
}