Miao-Search: Connect a data source via API

Overview

• If you have a proprietary search service with an API, you can integrate it with Miao-Search as a custom data source.

• By combining your third-party search API with the Miao-Search large model, you can deliver intelligent, generative search results.

• Miao-Search currently supports API definitions in JSON format.

Configuration

Configuration example

{
  "searchSourceRequestConfig": {
    "headers": [
      {
        "name": "Content-Type",
        "value": "application/json"
      }
    ],
    "method": "GET",
    "connectTimeout": 3000,
    "socketTimeout": 3000,
    "params": [
      {
        "name": "querySelf",
        "value": "${query}"
      },
      {
        "name": "current",
        "value": "${current}"
      },
      {
        "name": "size",
        "value": "${size}"
      },
      {
        "valueType": "boolean",
        "name": "includeContent",
        "value": "${includeContent}"
      },
      {
        "valueType": "time",
        "name": "startTime",
        "valueFormat": "yyyy-MM-dd HH:mm:ss",
        "value": "${startTime}"
      },
      {
        "valueType": "time",
        "name": "endTime",
        "valueFormat": "yyyy-MM-dd HH:mm:ss",
        "value": "${endTime}"
      }
    ],
    "pathParamsEnable": false,
    "body": "{\"query\":\"${querySelf}\",\"size\":\"${size}\",\"current\":\"${current}\"}",
    "url": "http://xxxx/api/search"
  },
  "searchSourceResponseConfig": {
    "jqNodes": [
      {
        "path": "totalSelf",
        "type": "number",
        "key": "total"
      },
      {
        "path": "dataSelf",
        "type": "list",
        "jqNodes": [
          {
            "path": "summary",
            "type": "string",
            "key": "summary"
          },
          {
            "path": "score",
            "type": "string",
            "key": "score"
          },
          {
            "path": "docUuid",
            "type": "string",
            "key": "docUuid"
          },
          {
            "path": "pubTime",
            "type": "string",
            "key": "pubTime"
          },
          {
            "path": "source",
            "type": "string",
            "key": "source"
          },
          {
            "path": "tag",
            "type": "string",
            "key": "tag"
          },
          {
            "path": "title",
            "type": "string",
            "key": "title"
          },
          {
            "path": ".",
            "type": "object",
            "jqNodes": [
              {
                "path": "docUuid",
                "type": "string",
                "key": "docUuid"
              }
            ],
            "key": "extendInfo"
          },
          {
            "path": "url",
            "type": "string",
            "key": "url"
          },
          {
            "path": "content",
            "type": "string",
            "key": "content"
          }
        ],
        "key": "data"
      }
    ]
  }
}

Request configuration: searchSourceRequestConfig

• URL (url)

  The URL of your API endpoint. The endpoint must be publicly accessible, such as http://xxxx.

• Request header (headers)

  You can add request headers, such as Content-Type and authentication information.

• HTTP method (method): The HTTP method for the request. Supported methods are GET and POST.

• Connection timeout (connectTimeout)

  The connection timeout in milliseconds. A value of 6000 or less is recommended.

• Request timeout (socketTimeout)

  The request timeout in milliseconds. A value of 6000 or less is recommended.

• Enable path parameters (pathParamsEnable)

  Specifies whether to send parameters in the URL path. The default value is false, which sends parameters in the JSON request body. If set to true, parameters are sent in the URL path.

• Request parameters (params)

  Defines the parameters to build the API request.

  Parameter	Description
  name	The name of the request parameter.
  value	The value of the parameter. This can be a fixed value or a variable in the format ${variableName}. The following built-in variables are supported: query: The search query. current: The current page number. size: The page size. includeContent: Your API should support this variable and return the content by default. startTime: The start time of the time range for the search. endTime: The end time of the time range for the search.
  valueType	The field type of the parameter, used for format conversion. Default: string. Supported types include: string number boolean time
  valueFormat	The format for the time value. This parameter is valid only when valueType is time. Supported formats include: yyyy-MM-dd yyyy-MM-dd HH:mm:ss longTime (Unix timestamp in seconds) longTimeMillis (Unix timestamp in milliseconds)

• Request body (body)

  The content of the request body. You can reference variables by using the ${variableName} syntax. Variables can be built-in or defined in params.

Response parsing configuration: searchSourceResponseConfig

Response parsing transforms the raw response from your API into the following standard response format.

{
  "total": 100,
  "data": [{
    "url": "http://xxx",
    "summary": "xxx",
    "score": 1,
    "docUuid": "xxx",
    "pubTime": "2024-12-10 15:43:44",
    "source": "xxx",
    "tag": "xxx",
    "title": "xxx",
    "content": "xxx"
  }]
}

jqNodes

This object defines the parsing configuration and supports up to three levels of nesting.

FAQ