相似度

更新时间:
复制 MD 格式

AI_SIMILARITY 函数用于比较两段文本的语义相似度并返回 0~1 的分数,适用于重复工单识别、问答去重和相似商品匹配等场景。

命令格式

REST 接口

POST /v2/vectordb/ai/similarity
Content-Type: application/json

{
  "model_name": "<模型名>",
  "texts": ["<文本A>", "<文本B>"],
  "params": {}
}

Python

schema = MilvusClient.create_schema(auto_id=True, enable_dynamic_field=False)
schema.add_field("id", DataType.INT64, is_primary=True)
schema.add_field("left_text", DataType.VARCHAR, max_length=4096)
schema.add_field("right_text", DataType.VARCHAR, max_length=4096)
schema.add_field("similarity", DataType.DOUBLE)
schema.add_field("dummy_vector", DataType.FLOAT_VECTOR, dim=2)
schema.add_function(
    Function(
        name="compare_texts",
        function_type=texttransform_function_type(),
        input_field_names=["left_text", "right_text"],
        output_field_names=["similarity"],
        params={
            "provider": "aliyun_milvus",
            "model_name": "<模型名>",
            "task": "ai_similarity",
            "temperature": "0",
        },
    )
)

参数说明

参数

说明

model_name

必填。已配置的文本模型名称;REST 也兼容 model。

texts

REST 必填。长度必须为偶数,[A,B,C,D] 表示比较 (A,B) 和 (C,D)。

temperature

选填。建议设为 0,让评分更稳定。

max_concurrency / timeout_sec

选填。控制单条调用的并发数和超时时间。

provider / task

仅 Collection Function 必填,固定为 aliyun_milvus 与 ai_similarity。

返回值说明

data.output.outputs 的元素数等于文本对数,每项为分数字符串。Collection 写入时会解析为 FLOAT 或 DOUBLE;无法解析或不在 0~1 内的值会写为 0。

使用示例:识别重复客服问题

客服系统需要判断两个问题是否可合并处理。下面用 REST 比较两对文本,并用 Collection Function 在写入时计算相似度。

REST 接口

#!/usr/bin/env bash
set -euo pipefail

MILVUS_REST_BASE_URL="http://c-xxxx.milvus.aliyuncs.com:19530"
MILVUS_AUTH_TOKEN="<yourUsername>:<yourPassword>"

post_json() {
  local path="$1"
  local body="$2"
  curl -X POST \
    "$MILVUS_REST_BASE_URL$path" \
    -H "Authorization: Bearer $MILVUS_AUTH_TOKEN" \
    -H "Content-Type: application/json" \
    -d "$body"
}

MODEL_NAME="qwen3.7-max"

BODY=$(cat <<JSON
{
  "model_name": "$MODEL_NAME",
  "texts": [
    "A cat is a mammal.", "Cats belong to mammals.",
    "An apple is a fruit.", "Python is a programming language."
  ],
  "params": {"max_concurrency": 2, "temperature": 0}
}
JSON
)

RESPONSE_BODY="$(post_json "/v2/vectordb/ai/similarity" "$BODY")"
if command -v jq >/dev/null 2>&1; then
  echo "$RESPONSE_BODY" | jq .
  [ "$(echo "$RESPONSE_BODY" | jq -r '.code // -1')" = "0" ] || exit 1
else
  echo "$RESPONSE_BODY"
fi
# 预期:outputs 元素数=文本对数,如 ["1","0.0"](相似对高分、不相似对接近0)

Python

安装 PyMilvus 后,将 MILVUS_URI、MILVUS_TOKEN 占位值替换为实际集群地址和 token 后运行;示例已内联公共辅助函数。

from __future__ import annotations

from typing import Any

from pymilvus import DataType, Function, FunctionType, MilvusClient

MILVUS_URI = "http://c-xxxx.milvus.aliyuncs.com:19530"
MILVUS_TOKEN = "<yourUsername>:<yourPassword>"

DUMMY_VECTOR_DIM = 2
TEXTTRANSFORM_FUNCTION_TYPE = 9

def texttransform_function_type() -> Any:
    for type_name in ("TEXTTRANSFORM", "TEXT_TRANSFORM", "TextTransform"):
        function_type = getattr(FunctionType, type_name, None)
        if function_type is not None:
            return function_type
    # 阿里云 Milvus 将 TEXTTRANSFORM 作为托管扩展(函数类型值 9)提供;
    # 部分 pymilvus 版本尚未内置该枚举成员,而 Function(...) 通过 FunctionType(...) 校验。
    existing = getattr(FunctionType, "_value2member_map_", {}).get(TEXTTRANSFORM_FUNCTION_TYPE)
    if existing is not None:
        return existing
    extension = int.__new__(FunctionType, TEXTTRANSFORM_FUNCTION_TYPE)
    extension._name_ = "TEXTTRANSFORM"
    extension._value_ = TEXTTRANSFORM_FUNCTION_TYPE
    FunctionType._value2member_map_[TEXTTRANSFORM_FUNCTION_TYPE] = extension
    FunctionType._member_map_["TEXTTRANSFORM"] = extension
    return extension

def add_id(schema: Any) -> None:
    schema.add_field("id", DataType.INT64, is_primary=True)

def add_dummy_vector(schema: Any) -> None:
    schema.add_field("dummy_vector", DataType.FLOAT_VECTOR, dim=DUMMY_VECTOR_DIM)

def run_texttransform_example(*, client, collection_name, input_fields, output_field, function_name, function_params, rows) -> None:
    if client.has_collection(collection_name):
        client.drop_collection(collection_name)
    schema = MilvusClient.create_schema(auto_id=True, enable_dynamic_field=False)
    add_id(schema)
    for name, data_type, max_length in input_fields:
        field_params = {"max_length": max_length} if max_length is not None else {}
        schema.add_field(name, data_type, **field_params)
    output_name, output_data_type, output_max_length = output_field
    output_params = {"max_length": output_max_length} if output_max_length is not None else {}
    schema.add_field(output_name, output_data_type, **output_params)
    add_dummy_vector(schema)
    schema.add_function(
        Function(
            name=function_name,
            function_type=texttransform_function_type(),
            input_field_names=[name for name, _, _ in input_fields],
            output_field_names=[output_name],
            params=function_params,
        )
    )
    index_params = client.prepare_index_params()
    index_params.add_index(field_name="dummy_vector", index_type="AUTOINDEX", metric_type="COSINE")
    client.create_collection(collection_name=collection_name, schema=schema, index_params=index_params)
    client.insert(collection_name, rows)
    client.flush(collection_name)
    fields = [name for name, _, _ in input_fields] + [output_name]
    for row in client.query(collection_name, filter="", output_fields=fields, limit=len(rows)):
        print(row)

MODEL_NAME = "qwen3.7-max"
client = MilvusClient(uri=MILVUS_URI, token=MILVUS_TOKEN)

run_texttransform_example(
    client=client,
    collection_name="simple_ai_similarity",
    input_fields=[("left_text", DataType.VARCHAR, 4096), ("right_text", DataType.VARCHAR, 4096)],
    output_field=("similarity", DataType.DOUBLE, None),
    function_name="compare_texts",
    function_params={"provider": "aliyun_milvus", "model_name": MODEL_NAME, "task": "ai_similarity", "temperature": "0"},
    rows=[{"left_text": "A cat is a mammal.", "right_text": "Cats belong to mammals.", "dummy_vector": [0.1, 0.2]}],
)
# 预期:similarity 得到接近 1 的分数(高度相似)

预期结果:相似对(如猫/哺乳动物)得到接近 1 的分数,不相似对接近 0;业务可将高于 0.85 的问题分到同一处理队列。