Model inference

Model inference input (input)

The input format varies based on the task, as described below:

• FEATURE_EXTRACTION (Text Embedding): The input format is a string or an array of strings.

• FEATURE_EXTRACTION (Multimodal Embedding): The following two input formats are supported.

  Array of JSON objects

  Each JSON object must contain the following parameters:

  Parameter	Type	Description
  image	String	The Base64-encoded string or OSS URI of the input image for model inference.
  text	String	The input text string for model inference.

  JSON object

  Parameter name	Type	Description
  images	Array of strings	The Base64-encoded strings or OSS URIs of the input images for model inference.
  texts	Array of strings	The input text strings for model inference.

  You can use this input format to send requests for inference scenarios that involve only images or only plain text.

• QUESTION_ANSWERING: The input format is a string.

• SEMANTIC_SIMILARITY: The input format is a JSON object that contains the following parameters:

  Parameter	Type	Description
  query	String	The query for which you want to compare similarity (rerank).
  chunks	Array of strings	A list of document chunks for which you want to compare similarity.

Model inference parameters (params)

Example 1: Feature extraction

Example 2: Q&A

Request:

POST v1/ai/models/qa_model/infer HTTP/1.1
Content-Type: application/json

{
    "input": "Write an essay of about 200 words"
}

Response:

HTTP/1.1 200 OK
Date: Tue, 28 Nov 2023 03:18:55 GMT
Transfer-encoding: chunked
Content-type: application/json

{"code":0,"msg":"SUCCESS","success":true,"data":{"outputs":["","Life is like ","a journey, full of ","unknowns and challenges.","On this journey, ","we will meet various people and ","events. Some make us ","feel excited and curious",", while others make us ","feel frustrated and disappointed",". But no matter what ","situations we encounter, ","we must maintain a positive and optimistic attitude",", believe in ourselves, believe ","in the future, and keep moving forward.","\n\nOn the journey of life",", we will encounter ","various people and events. ","Some people make us feel excited ","and curious, while others ","make us feel frustrated ","and disappointed. But ","no matter what situations ","we encounter, we must maintain ","a positive and optimistic attitude, believe in ourselves",", believe in the future, ","and keep moving forward.","\n\nSometimes, we may ","feel lost and unsure of what to do",". At these times, we need to give ourselves ","some time and space to think ","about what we want and how to achieve ","our goals. At the same time, ","we also need to communicate with others",", listen to their opinions ","and ideas, and from ","them, gain inspiration and help.","\n\nOn the journey ","of life, we need to ","constantly learn and grow ","to better face the challenges ahead",". No matter what ","difficulties and challenges we face, ","we must maintain a positive and optimistic attitude",", believe in ourselves, believe ","in the future, and keep moving forward.","\n\nOn the journey of life",", we need to constantly ","learn and grow to better ","face the challenges ahead.","No matter what ","difficulties and challenges we face, we must ","maintain a positive and optimistic attitude,"]}}%

Example 3: Semantic similarity

Request:

POST v1/ai/models/bge_rerank_model/infer HTTP/1.1
Content-Type: application/json

{
    "input": {"query": "a dog", "chunks": ["a slide", "a small yellow dog"]}
}

Response:

HTTP/1.1 200 OK
Date: Tue, 28 Nov 2023 03:18:55 GMT
Content-type: application/json
Content-length: 419

{
  "code" : 0,
  "msg" : "SUCCESS",
  "data" : [ {
    "chunk" : "a small yellow dog",
    "score" : "0.6785464882850647",
    "index" : 1
  }, {
    "chunk" : "a slide",
    "score" : "0.5992767214775085",
    "index" : 0
  }],
  "success" : true
}