智能问答API

基于丰富的全模态搜索与理解能力，通过AI搜问平台上搭建的问答应用，当前功能可以支持针对数据集内图片、文档、视频进行深度问答能力。可以通过文本、图片或图文作为问题输入，同时支持定制问答助手的回复方式，获得针对数据集全集内容的回答，快速适配大数据下的各类问答场景。其中视频相关素材另有模板化输出，可定制化总结提炼视频内容。

接口说明

接入说明：

智能问答流式接口，支持多模态输入（文本、图片、结构化数据）和流式输出（文本、图片、视频、信源等）。接口采用 SSE（Server-Sent Events）协议推送响应数据，每个数据流为一个 JSON 对象，输出时会通过 type 字段标记不同数据类型。

数据源：

支持对视频进行问答，可通过CleverSee AI 搜问平台的产品界面内或通过数据集数据新增/更新 API完成数据上传及更新。

认证方式：

通过 阿里云 SDK 调用 CleverSee - 智能问答 服务，具体调用指南请参考：AI问答引擎接口

请求参数

请求示例

正常请求示例

JSON格式

图文问答请求

  {
    "id": "chat-002",
    "message": {
      "role": "user",
      "parts": [
        { "type": "text", "text": "找下类似这双鞋的款式" },
        {
          "type": "file",
          "url": "https://example.com/image.png",
          "mediaType": "image/png"
        }
      ]
    },
    "options": { "appId": "app_001" }
  }

视频模板请求

{
  "id": "chat-002",
  "message": {
    "role": "user",
    "parts": [
      {
        "type": "text",
        "text": "总结这个视频"
      },
      {
        "type": "data",
        "data": {
          "type": "template",
          "videoId": "vid_001"
        }
      }
    ]
  },
  "options": {
    "appId": "app_001"
  }
}

返回参数

响应以 SSE 流形式逐帧推送，每帧为一个 JSON 对象，必须包含 type 字段标识当前数据块类型。

数据块类型总览

消息开始和结束

完整的消息生命周期遵循以下顺序：

start → 内容块（可多个，可交替) → finish

步骤说明：

1. start：消息开始，携带 messageId。

2. 内容块（text-*、data-* 等，可多个、可交替）。

3. finish：消息结束，携带 finishReason。

文本流式回复

文本内容输出采用流式分块输出，一个完整的文本段落由 text-start → 若干 text-delta → text-end 组成。多个文本块可与媒体块交替出现，实现图文混排或视频文字混排。

1. text-start：标记文本块开始，携带该文本块的唯一 id。

2. text-delta：推送增量文本内容，同一 id 可多次出现，客户端按顺序拼接。

3. text-end：标记文本块结束。

图文视频混排

文本块与媒体块可交替输出，形成混排效果。典型顺序为：

text-start → text-delta → text-end → data-image-info → text-start → text-delta → text-end → data-video-info → ...

视频问答应用（模板输出型）回复

回复信源引用

返回示例

正常返回示例

JSON格式

图片问答响应

以下为图片问答应用内流式输出，且图片文本混排的示例

  data: {"type":"start","messageId":"msg-001"}

  data: {"type":"text-start","id":"text-1"}

  data: {"type":"text-delta","id":"text-1","delta":"按风格选：轻运动 VS 日常休闲\n\n"}

  data: {"type":"text-delta","id":"text-1","delta":"**轻运动款** —— 活力感与舒适感兼顾"}

  data: {"type":"text-end","id":"text-1"}
  
  data: {"type":"data-image-info","data":{"id":"img-001","imageUrl":"https://cdn.example.com/i1.jpg","name":"轻运动款 - 拼接运动鞋"}}

  data: {"type":"text-start","id":"text-2"}
  
  data: {"type":"text-delta","id":"text-2","delta":"**日常休闲款** —— 简洁耐看更百搭"}

  data: {"type":"text-end","id":"text-2"}

  data: {"type":"data-image-info","data":{"id":"img-002","imageUrl":"https://cdn.example.com/i2.jpg","name":"日常休闲款 - 纯色休闲鞋"}}

  data: {"type":"data-reference","data":{"items":[
    {"sourceType":"document","fileName":"运动鞋舒适度评测报告.docx","fileType":"docx","chunkContent":"在中端价位段中，轻量透气材质逐渐成为主流...","score":0.87},
    {"sourceType":"image","datasetName":"商品图片数据集","thumbnailUrl":"https://cdn.example.com/i2-thumb.jpg","imageUrl":"https://cdn.example.com/i2.jpg"}
  ]}}

  data: {"type":"finish","finishReason":"stop"}

视频问答响应（视频文字混排）

以下为视频问答应用（对话型）内流式输出，且视频文本混排的示例

data:{"id":"mock-session-001","type":"start","requestId":"mock-request-001"}

data:{"id":"text-block-001","type":"text-start","requestId":"mock-request-001"}

data:{"id":"text-block-001","type":"text-delta","requestId":"mock-request-001","delta":"视频中"}

data:{"id":"text-block-001","type":"text-delta","requestId":"mock-request-001","delta":"展示了"}

data:{"id":"text-block-001","type":"text-delta","requestId":"mock-request-001","delta":"悬挂"}

data:{"id":"text-block-001","type":"text-delta","requestId":"mock-request-001","delta":"的"}

data:{"id":"text-block-001","type":"text-delta","requestId":"mock-request-001","delta":"装饰"}

data:{"id":"text-block-001","type":"text-delta","requestId":"mock-request-001","delta":"摆件"}

data:{"id":"text-block-001","type":"text-delta","requestId":"mock-request-001","delta":"，并通过"}

data:{"id":"text-block-001","type":"text-delta","requestId":"mock-request-001","delta":"特写"}

data:{"id":"text-block-001","type":"text-delta","requestId":"mock-request-001","delta":"镜头"}

data:{"id":"text-block-001","type":"text-delta","requestId":"mock-request-001","delta":"展示"}

data:{"id":"text-block-001","type":"text-delta","requestId":"mock-request-001","delta":"其"}

data:{"id":"text-block-001","type":"text-delta","requestId":"mock-request-001","delta":"纹理"}

data:{"id":"text-block-001","type":"text-delta","requestId":"mock-request-001","delta":"细节"}

data:{"id":"text-block-001","type":"text-end","requestId":"mock-request-001"}

data:{"id":"video-ref-001","type":"data-video-ref","requestId":"mock-request-001","data":{"coverUrl":"https://.../cover_frame.jpg?Expires=...","videoUrl":"https://.../demo_video.mp4?Expires=...","sourceType":"video","name":"sample_video.mp4","start":"0.00","datasetId":"DATASET_001","end":"10.00","id":"video-001"}}

data:{"id":"text-block-002","type":"text-start","requestId":"mock-request-001"}

data:{"id":"text-block-002","type":"text-delta","requestId":"mock-request-001","delta":"。为了"}

data:{"id":"text-block-002","type":"text-delta","requestId":"mock-request-001","delta":"展示"}

data:{"id":"text-block-002","type":"text-delta","requestId":"mock-request-001","delta":"使用"}

data:{"id":"text-block-002","type":"text-delta","requestId":"mock-request-001","delta":"效果"}

data:{"id":"text-block-002","type":"text-delta","requestId":"mock-request-001","delta":"，画面"}

data:{"id":"text-block-002","type":"text-delta","requestId":"mock-request-001","delta":"演示了"}

data:{"id":"text-block-002","type":"text-delta","requestId":"mock-request-001","delta":"该"}

data:{"id":"text-block-002","type":"text-delta","requestId":"mock-request-001","delta":"摆件"}

data:{"id":"text-block-002","type":"text-delta","requestId":"mock-request-001","delta":"在"}

data:{"id":"text-block-002","type":"text-delta","requestId":"mock-request-001","delta":"室内"}

data:{"id":"text-block-002","type":"text-delta","requestId":"mock-request-001","delta":"与"}

data:{"id":"text-block-002","type":"text-delta","requestId":"mock-request-001","delta":"桌面"}

data:{"id":"text-block-002","type":"text-delta","requestId":"mock-request-001","delta":"场景"}

data:{"id":"text-block-002","type":"text-delta","requestId":"mock-request-001","delta":"中的"}

data:{"id":"text-block-002","type":"text-delta","requestId":"mock-request-001","delta":"摆放"}

data:{"id":"text-block-002","type":"text-end","requestId":"mock-request-001"}

data:{"id":"video-ref-002","type":"data-video-ref","requestId":"mock-request-001","data":{"coverUrl":"https://.../cover_frame_02.jpg?Expires=...","videoUrl":"https://.../demo_video.mp4?Expires=...","sourceType":"video","name":"sample_video.mp4","start":"10.00","datasetId":"DATASET_001","end":"20.00","id":"video-001"}}

data:{"id":"text-block-003","type":"text-start","requestId":"mock-request-001"}

data:{"id":"text-block-003","type":"text-delta","requestId":"mock-request-001","delta":"。"}

data:{"id":"text-block-003","type":"text-end","requestId":"mock-request-001"}

data:{"id":"reference-001","type":"data-reference","requestId":"mock-request-001","data":{"items":[{"coverUrl":"https://.../cover_frame.jpg?Expires=...","videoUrl":"https://.../demo_video.mp4?Expires=...","sourceType":"video","name":"sample_video.mp4","start":"10.00","datasetId":"DATASET_001","end":"20.00","id":"video-001"}]}}

data:{"id":"mock-session-001","type":"finish","requestId":"mock-request-001","finishReason":"stop"}

视频模板响应

以下为问答应用（模板输出型）内不同类型模板的输出示例

流式Json类（动作表达、赛事高光、知识要点）

id:mock-msg-default-001
event:start
data:{"id":"mock-msg-default-001","type":"start","requestId":"mock-request-001"}

id:mock-video-info-001
event:data-video-info
data:{"id":"mock-video-info-001","type":"data-video-info","requestId":"mock-request-001","data":{"coverUrl":"https://example.com/mock/cover.jpg","keywords":"团队协作,产品演示,用户体验,功能亮点","videoUrl":"https://example.com/mock/video.mp4","sourceType":"video","videoType":"raw","name":"demo-video.mp4","id":"mock-video-id-001"}}

id:mock-template-info-001
event:data-template-info
data:{"id":"mock-template-info-001","type":"data-template-info","requestId":"mock-request-001","data":{"templateName":"动作表达","templateId":"tpl-demo-001"}}

id:mock-json-stream-001
event:json-start
data:{"id":"mock-json-stream-001","type":"json-start","requestId":"mock-request-001"}

id:mock-json-stream-001
event:json-delta
data:{"id":"mock-json-stream-001","type":"json-delta","requestId":"mock-request-001","delta":"{"}

id:mock-json-stream-001
event:json-delta
data:{"id":"mock-json-stream-001","type":"json-delta","requestId":"mock-request-001","delta":"\n  \"applicable\": true,"}

id:mock-json-stream-001
event:json-delta
data:{"id":"mock-json-stream-001","type":"json-delta","requestId":"mock-request-001","delta":"\n  \"content_type\": \"产品演示片段\","}

id:mock-json-stream-001
event:json-delta
data:{"id":"mock-json-stream-001","type":"json-delta","requestId":"mock-request-001","delta":"\n  \"actions\": ["}

id:mock-json-stream-001
event:json-delta
data:{"id":"mock-json-stream-001","type":"json-delta","requestId":"mock-request-001","delta":"\n    {"}

id:mock-json-stream-001
event:json-delta
data:{"id":"mock-json-stream-001","type":"json-delta","requestId":"mock-request-001","delta":"\n      \"start\": 0.0,"}

id:mock-json-stream-001
event:json-delta
data:{"id":"mock-json-stream-001","type":"json-delta","requestId":"mock-request-001","delta":"\n      \"end\": 10.0,"}

id:mock-json-stream-001
event:json-delta
data:{"id":"mock-json-stream-001","type":"json-delta","requestId":"mock-request-001","delta":"\n      \"subject\": \"演示人员\","}

id:mock-json-stream-001
event:json-delta
data:{"id":"mock-json-stream-001","type":"json-delta","requestId":"mock-request-001","delta":"\n      \"action\": \"走向展示区并介绍核心功能\","}

id:mock-json-stream-001
event:json-delta
data:{"id":"mock-json-stream-001","type":"json-delta","requestId":"mock-request-001","delta":"\n      \"intensity\": \"自然、清晰\","}

id:mock-json-stream-001
event:json-delta
data:{"id":"mock-json-stream-001","type":"json-delta","requestId":"mock-request-001","delta":"\n      \"context\": \"通过稳定讲解帮助观众理解产品亮点\""}

id:mock-json-stream-001
event:json-delta
data:{"id":"mock-json-stream-001","type":"json-delta","requestId":"mock-request-001","delta":"\n    }"}

id:mock-json-stream-001
event:json-delta
data:{"id":"mock-json-stream-001","type":"json-delta","requestId":"mock-request-001","delta":"\n  ]"}

id:mock-json-stream-001
event:json-delta
data:{"id":"mock-json-stream-001","type":"json-delta","requestId":"mock-request-001","delta":"\n}"}

id:mock-json-stream-001
event:json-end
data:{"id":"mock-json-stream-001","type":"json-end","requestId":"mock-request-001"}

结构化Json输出（视频纲要、视频转脚本、语音转录）

id:mock-start-001
event:start
data:{"id":"mock-start-001","type":"start","requestId":"mock-request-001"}

id:video-info-001
event:data-video-info
data:{"id":"video-info-001","type":"data-video-info","requestId":"mock-request-001","data":{"coverUrl":"https://.../cover_frame.jpg?Expires=...","keywords":"人物交流,活动现场,生活分享,轻松互动","videoUrl":"https://.../sample_video.mp4?Expires=...","sourceType":"video","videoType":"raw","name":"sample_video.mp4","id":"VIDEO_ID_001"}}

id:template-info-001
event:data-template-info
data:{"id":"template-info-001","type":"data-template-info","requestId":"mock-request-001","data":{"templateName":"视频转脚本","templateId":"tpl-demo-001"}}

id:template-content-001
event:data-template-video-content
data:{"id":"template-content-001","type":"data-template-video-content","requestId":"mock-request-001","data":{"header":{"shot_size":"景别","director_prompt":"导演提示","content_summary":"内容概要","transcription":"转录","remarks":"备注","start":"开始时间","end":"结束时间"},"list":[

  {"content_summary":"几位年轻人在户外活动现场边走边交流。","transcription":"男声介绍活动安排，语气轻松自然，背景伴有现场环境声。","director_prompt":"跟随人物移动后切换正面镜头","shot_size":"中景","start":0.0,"end":10.0,"remarks":"场景光线自然，周围人群氛围轻松愉快。"},

  {"content_summary":"主持人在室内活动现场向观众介绍流程。","transcription":"男声说明接下来的互动环节，语气平稳且具有节奏感。","director_prompt":"缓慢推进","shot_size":"中景","start":10.0,"end":20.0,"remarks":"画面色调温暖，观众神情专注，现场布置整洁明亮。"},

  {"content_summary":"一位长者在室内与年轻人进行交流。","transcription":"长者语气温和，分享自己的生活经验与建议。","director_prompt":"缓慢推进","shot_size":"中景","start":20.0,"end":30.0,"remarks":"场景色调偏暖，人物互动自然。"},

]}}

id:mock-start-001
event:finish
data:{"id":"mock-start-001","type":"finish","requestId":"mock-request-001","finishReason":"stop"}

Markdown格式（剧情理解、会议纪要、自定义模板）

data: {"type":"start","messageId":"mock-msg-default-001"}

data: {"type":"start-step"}

data: {"type":"data-template-info","data":{"items":[{"title":"MacBook Neo 深度测评：全面超越，最轻盈的生产力","videoUrl":"https://video.example.com/macbook-neo-review.mp4","coverUrl":"https://images.unsplash.com/photo-1529139574466-a303027c1d8b?auto=format&fit=crop&w=800&q=80","tags":["数码产品","评测","MacBook","生产力"]}]}}

data: {"id":"b43b71c8-dcf0-4597-b1e8-3b7b7599f73e","type":"data-template-info","requestId":"1e78148a17782391368705400d119a","data":{"templateName":"视频转脚本","templateId":"tpl-3"}}

data: {"type":"text-start","id":"text-video-summary-001"}

data: {"type":"text-delta","id":"text-video-summary-001","delta":"## 视频内容总结\n\n### 1. MacBook Neo 深度测评\n\n- **核心亮点**：全新 M4 Ultra 芯片，单核性能提升 32%，整机重量仅 1.24kg\n- **续航表现**：视频播放续航达 22 小时，日常办公可撑满两个工作日\n- **不足之处**：接口仍为 3 个 USB-C，缺少 SD 卡槽\n\n> 适合人群：追求轻薄便携的创意工作者"}

data: {"type":"text-end","id":"text-video-summary-001"}

data: {"type":"finish-step"}

data: {"type":"finish","finishReason":"stop"}

拒答返回示例

JSON格式

data: {"type":"start","messageId":"mock-msg-default-001"}

data: {"type":"text-start","id":"text-default-part1"}

data: {"type":"text-delta","id":"text-default-part1","delta":"抱歉，这个话题超出了我的回答范围，有其他问题请随时问我。"}

data: {"type":"text-end","id":"text-default-part1"}

data: {"type":"finish","finishReason":"stop"}

错误码

搜索结果中多媒体资源（图片、视频）URL获取预签名 API

如需获取数据集内数据url链接并进行下载，请参考搜索结果中多媒体资源（图片、视频）URL获取预签名 API