开放能力概览
工具 | 名称 | 描述 | 支持说明 |
| 创建一个新的AgentBay沙箱并返回其ID。 | 支持 | |
| 当用户需要访问无影MCP运行时，用于获取其URL的命令。每次获取的URL仅单次有效，使用后即失效。 | 支持 | |
| 捕获当前显示屏幕的全屏截图，并返回一个可共享的URL。截图会自动经过处理并安全存储。为保障安全，生成的URL将在64分钟后过期。 | 支持 | |
| 任务完成后释放资源。 | 支持 | |
| 在Android平台上执行shell命令，支持设置超时，执行完成后返回命令输出或错误信息。 | 支持 | |
| 在屏幕的特定坐标处点击。 | 支持 | |
| 输入文本。 | 支持 | |
| 发送按键。支持 Android 平台按键：
| 支持 | |
| 在屏幕上执行滑动手势。 | 支持 | |
| 在超时时间内获取设备上的所有 UI 元素（包括非交互元素）。 | 支持 | |
| 在超时时间内获取所有可点击的 UI 元素。 | 支持 | |
| 检索系统中安装的应用程序列表。支持通过开始菜单条目和桌面快捷方式筛选，并可选择排除系统应用。返回应用程序详细信息，包括名称、启动命令、可选的停止命令和工作目录。 | 支持 | |
| 使用提供的命令和可选工作目录启动指定应用程序。返回与启动应用程序关联的进程列表，包括进程名称、PID 和启动命令。 | 支持 | |
| 使用提供的停止命令终止应用程序。请谨慎使用，因为这将强制终止指定进程。 | 支持 | |
| 在 Android 平台上执行 shell 命令并返回输出或错误。 | 支持 |
System
名称 | 描述 | 参数 |
| 创建一个新的AgentBay沙箱并返回其ID。 | |
| 当用户需要访问无影MCP运行时，用于获取其URL的命令。每次获取的URL仅单次有效，使用后即失效。 | |
| 捕获当前显示屏幕的全屏截图，并返回一个可共享的URL。截图会自动经过处理并安全存储。为保障安全，生成的URL将在64分钟后过期。 | |
| 任务完成后释放资源。 | |
| 在Android平台上执行shell命令，支持设置超时，执行完成后返回命令输出或错误信息。 | |
UI
名称 | 描述 | 参数 |
| 在屏幕的特定坐标处点击。 | |
| 输入文本。 | |
| 发送按键。支持 Android 平台按键：
| |
| 在屏幕上执行滑动手势。 | |
| 在超时时间内获取设备上的所有 UI 元素（包括非交互元素）。 | |
| 在超时时间内获取所有可点击的 UI 元素。 | |
App
名称 | 描述 | 参数 |
| 检索系统中安装的应用程序列表。支持通过开始菜单条目和桌面快捷方式筛选，并可选择排除系统应用。返回应用程序详细信息，包括名称、启动命令、可选的停止命令和工作目录。 | |
| 使用提供的命令和可选工作目录启动指定应用程序。返回与启动应用程序关联的进程列表，包括进程名称、PID 和启动命令。 | |
| 使用提供的停止命令终止应用程序。请谨慎使用，因为这将强制终止指定进程。 | |
Shell
名称 | 描述 | 参数 |
| 在 Android 平台上执行 shell 命令并返回输出或错误。 | |
MCP Tool List
{
"wuying_oss": {
"tools": [
{
"name": "oss_env_init",
"description": "Create and initialize OSS environment variables with the specified endpoint, access key ID, access key secret, security token, and region. The temporary security credentials obtained from the STS (Security Token Service). For more information, see: https://help.aliyun.com/zh/oss/developer-reference/use-temporary-access-credentials-provided-by-sts-to-access-oss?spm=a2c4g.11186623.help-menu-search-31815.d_1#9ab17afd7cs4t .",
"inputSchema": {
"properties": {
"access_key_id": {
"description": "The Access Key ID for OSS authentication",
"type": "string"
},
"access_key_secret": {
"description": "The Access Key Secret for OSS authentication",
"type": "string"
},
"endpoint": {
"description": "The OSS service endpoint, e.g., If not specified, the default is https://oss-cn-hangzhou.aliyuncs.com",
"type": "string"
},
"region": {
"description": "The OSS region, e.g., cn-hangzhou. If not specified, the default is cn-hangzhou",
"type": "string"
},
"security_token": {
"description": "The Security Token for OSS authentication",
"type": "string"
}
},
"required": [
"access_key_id",
"access_key_secret",
"security_token"
],
"type": "object"
}
},
{
"name": "oss_upload",
"description": "Upload a local file or directory to the specified OSS bucket. If a directory is specified, it will be compressed into a ZIP file before uploading. The object name in OSS can be specified; if not, the file or ZIP name will be used by default. Note: You must call the oss_env_init tool to initialize OSS environment variables before using this tool.",
"inputSchema": {
"properties": {
"bucket": {
"description": "OSS bucket name",
"type": "string"
},
"object": {
"description": "Object path in OSS bucket, e.g., test/test.txt",
"type": "string"
},
"path": {
"description": "Local file or not empty directory full path to upload, e.g., /tmp/test.txt /tmp on Linux or C:/tmp/test.txt C:/tmp on Windows",
"type": "string"
}
},
"required": [
"bucket",
"object",
"path"
],
"type": "object"
}
},
{
"name": "oss_download",
"description": "Download an object from the specified OSS bucket to the given local path. If the parent directory does not exist, it will be created automatically. If the target file already exists, it will be overwritten. Note: You must call the oss_env_init tool to initialize OSS environment variables before using this tool.",
"inputSchema": {
"properties": {
"bucket": {
"description": "OSS bucket name",
"type": "string"
},
"object": {
"description": "Object path in OSS bucket, e.g., test/test.txt",
"type": "string"
},
"path": {
"description": "Local full path to save the downloaded file, e.g., /tmp/test.txt on Linux or C:/tmp/test.txt on Windows",
"type": "string"
}
},
"required": [
"bucket",
"object",
"path"
],
"type": "object"
}
},
{
"name": "oss_upload_annon",
"description": "Upload a local file or directory to the specified URL using HTTP PUT. If a directory is specified, it will be compressed into a ZIP file before uploading. If the upload target already exists, it will be overwritten.",
"inputSchema": {
"properties": {
"path": {
"description": "Local file or not empty directory full path to upload, e.g., /tmp/test.txt /tmp on Linux or C:/tmp/test.txt C:/tmp on Windows",
"type": "string"
},
"url": {
"description": "The HTTP/HTTPS URL to upload the file to",
"type": "string"
}
},
"required": [
"url",
"path"
],
"type": "object"
}
},
{
"name": "oss_download_annon",
"description": "Download a file from the specified URL to the given local path. If the parent directory does not exist, it will be created automatically. If the target file already exists, it will be overwritten.",
"inputSchema": {
"properties": {
"path": {
"description": "The full local file path to save the downloaded file, e.g., /tmp/test.txt on Linux or C:/tmp/test.txt on Windows",
"type": "string"
},
"url": {
"description": "The HTTP/HTTPS URL to download the file from",
"type": "string"
}
},
"required": [
"url",
"path"
],
"type": "object"
}
}
]
},
"wuying_ui": {
"tools": [
{
"name": "click",
"description": "Click on the screen at specific coordinates.",
"inputSchema": {
"properties": {
"button": {
"description": "button type,Available values:left,middle,right,default:left",
"type": "string"
},
"x": {
"description": "X coordinate",
"type": "integer"
},
"y": {
"description": "Y coordinate",
"type": "integer"
}
},
"required": [
"x",
"y",
"button"
],
"type": "object"
}
},
{
"name": "input_text",
"description": "Input text",
"inputSchema": {
"properties": {
"text": {
"description": "client input text",
"type": "string"
}
},
"required": [
"text"
],
"type": "object"
}
},
{
"name": "send_key",
"description": "Send a key.Supported Keys on Android Platform: 3:HOME,4:BACK,24:VOLUME_UP,25:VOLUME_DOWN,26:POWER,82:MENU",
"inputSchema": {
"properties": {
"key": {
"description": "client send key",
"type": "integer"
}
},
"required": [
"key"
],
"type": "object"
}
},
{
"name": "swipe",
"description": "Perform a swipe gesture on the screen.",
"inputSchema": {
"properties": {
"duration_ms": {
"default": 300,
"description": "Duration of swipe in milliseconds,default:300",
"type": "integer"
},
"end_x": {
"description": "Ending X coordinate",
"type": "integer"
},
"end_y": {
"description": "Ending Y coordinate",
"type": "integer"
},
"start_x": {
"description": "Starting X coordinate",
"type": "integer"
},
"start_y": {
"description": "Starting Y coordinate",
"type": "integer"
}
},
"required": [
"start_x",
"start_y",
"end_x",
"end_y",
"duration_ms"
],
"type": "object"
}
},
{
"name": "get_all_ui_elements",
"description": "Get all UI elements from the device with timeout, including non-interactive elements.",
"inputSchema": {
"properties": {
"timeout_ms": {
"default": 1000,
"description": "Command execution timeout (unit: milliseconds). If not specified, the default value (such as 1000 milliseconds) is used",
"type": "integer"
}
},
"required": [
"timeout_ms"
],
"type": "object"
}
},
{
"name": "get_clickable_ui_elements",
"description": "Get all clickable UI elements within timeout.",
"inputSchema": {
"properties": {
"timeout_ms": {
"default": 1000,
"description": "Command execution timeout (unit: milliseconds). If not specified, the default value (such as 1000 milliseconds) is used",
"type": "integer"
}
},
"required": [
"timeout_ms"
],
"type": "object"
}
}
]
},
"wuying_app": {
"tools": [
{
"name": "get_installed_apps",
"description": "Retrieve a list of installed applications on the system. Supports filtering by Start Menu entries and Desktop shortcuts, with an option to exclude system applications. Returns application details including name, start command, optional stop command, and working directory.",
"inputSchema": {
"properties": {
"desktop": {
"default": false,
"description": "Include Desktop shortcuts (default: false)",
"type": "boolean"
},
"ignore_system_app": {
"default": true,
"description": "Exclude system applications (default: true)",
"type": "boolean"
},
"start_menu": {
"default": true,
"description": "Include Start Menu applications (default: true)",
"type": "boolean"
}
},
"required": [],
"type": "object"
}
},
{
"name": "start_app",
"description": "Start a specified application using the provided command and optional working directory. Returns a list of processes associated with the launched application, including their process names, PIDs, and startup commands.\n\n",
"inputSchema": {
"properties": {
"activity": {
"description": "The activity name on android,like .xxActivity or com.xx/.xxActivity.",
"type": "string"
},
"start_cmd": {
"description": "The command to start the application. On Android, use the 'monkey -p ' syntax (e.g., 'monkey -p <package_name> -c android.intent.category.LAUNCHER 1').",
"type": "string"
},
"work_directory": {
"default": "",
"description": "The directory from which the application should be launched. If omitted, the default directory is used.",
"type": "string"
}
},
"required": [
"start_cmd"
],
"type": "object"
}
},
{
"name": "stop_app_by_cmd",
"description": "Terminate an application using the provided stop command. Use with caution as this will forcefully terminate the specified process.",
"inputSchema": {
"properties": {
"stop_cmd": {
"description": "The command used to terminate the application.",
"type": "string"
}
},
"required": [
"stop_cmd"
],
"type": "object"
}
}
]
},
"wuying_shell": {
"tools": [
{
"name": "shell",
"description": "Executes an shell command with timeout and returns the output or an error on android platform.",
"inputSchema": {
"properties": {
"command": {
"description": "client input command",
"type": "string"
},
"timeout_ms": {
"default": 1000,
"description": "Command execution timeout (unit: milliseconds). If not specified, the default value (such as 1000 milliseconds) is used",
"type": "integer"
}
},
"required": [
"command",
"timeout_ms"
],
"type": "object"
}
}
]
}
}该文章对您有帮助吗？