Conversation content extraction

Updated at:

This topic describes the conversation content extraction feature, including its AI capabilities and implementation. This feature extracts specific topics from conversations in various scenarios, such as sales, speeches, interviews, and customer service. You can customize multiple topics for extraction to quickly identify key points in a conversation. This information helps you iterate on products and develop marketing strategies.

Request parameters

Parameter

Type

Required

Description

ContentExtractionEnabled

boolean

No

Enables or disables the conversation content extraction feature. The default value is false.

ContentExtraction

object

No

The parameter object for conversation content extraction.

ContentExtraction.SceneIntroduction

string

Yes

The scenario description for conversation content extraction.

ContentExtraction.ExtractionContents

list[]

Yes

A list of dimensions for content extraction. This list includes the name and definition of each extraction item. The number of items cannot exceed 100.

ContentExtraction.ExtractionContents[i].Title

string

Yes

The name of the extraction dimension.

ContentExtraction.ExtractionContents[i].Content

string

Yes

Defining dimensions for conversation content extraction.

ContentExtraction.ExtractionContents[i].Identity

string

No

The speaker identity for this extraction dimension. This parameter must be used with Speaker Diarization.

For descriptions of other request parameters, see the document for your scenario:

Note

The Speaker Diarization feature helps optimize the results of conversation content extraction. If you enable both Speaker Diarization and conversation content extraction, Speaker Diarization automatically runs first. The diarization result is then used as identity information in the conversation content. You can use the ContentExtraction.ExtractionContents.Identity parameter to specify the speaker for a topic and improve the extraction results.

Example settings

The conversation content extraction feature requires a clear conversation scenario, such as online education phone sales, offline car outlet sales, or real estate sales.

Use the `Title` field to specify the name of the extraction item and the `Content` field to provide its description. Use clear and direct descriptions. Do not add role definitions, regular expressions, or complex list structures, because this can cause conflicts with the service's built-in prompts.

{
    "Input":{
        ...
    },
    "Parameters": {
        "ContentExtractionEnabled": true,
        "ContentExtraction": {
            "SceneIntroduction": "Offline car outlet sales scenario",
            "ExtractionContents": [
                {
                    "Title": "Customer price objections",
                    "Content": "Customer mentions car model prices, promotional offers, or concerns about future price drops",
                    "Identity": "Customer"
                },
                {
                    "Title": "Competitor model feedback",
                    "Content": "Summarize all competitor brands or models that the customer explicitly mentioned, and summarize the customer's thoughts on them"
                },
                {
                    "Title": "Sales script for guiding purchase",
                    "Content": "Extract the sales script used to guide the customer to purchase a car from the conversation",
                    "Identity": "Sales"
                }
            ]
        }
    }
}

Recommendations:

  • Ensure that `Title` values are distinct. Avoid using repetitive or unclear descriptions for different titles, such as "Customer questions", "Conversation topics", or "Customer needs".

  • Do not add conditional logic to the `Content` field.

Code examples

#!/usr/bin/env python
# coding=utf-8

import os
import json
import datetime
from aliyunsdkcore.client import AcsClient
from aliyunsdkcore.request import CommonRequest
from aliyunsdkcore.auth.credentials import AccessKeyCredential


def create_common_request(domain, version, protocolType, method, uri):
    request = CommonRequest()
    request.set_accept_format('json')
    request.set_domain(domain)
    request.set_version(version)
    request.set_protocol_type(protocolType)
    request.set_method(method)
    request.set_uri_pattern(uri)
    request.add_header('Content-Type', 'application/json')
    return request


def init_parameters():
    root = dict()
    root['AppKey'] = 'Enter the AppKey that you created in the Tingwu console'

    # Basic request parameters
    input = dict()
    input['SourceLanguage'] = 'cn'
    input['TaskKey'] = 'task' + datetime.datetime.now().strftime('%Y%m%d%H%M%S')
    input['FileUrl'] = 'Enter the URL of the audio file to test'
    root['Input'] = input

    # AI-related parameters. Set them as needed.
    parameters = dict()
    # Conversation content extraction
    parameters['ContentExtractionEnabled'] = True
    content_extraction = {
        "SceneIntroduction": "Offline car outlet sales scenario",
        "ExtractionContents": [
            {
                "Title": "Customer price objections",
                "Content": "Customer mentions car model prices, promotional offers, or concerns about future price drops",
                "Identity": "Customer"
            },
            {
                "Title": "Competitor model feedback",
                "Content": "Summarize all competitor brands or models that the customer explicitly mentioned, and summarize the customer's thoughts on them",
                "Identity": "Customer"
            },
            {
                "Title": "Sales script for guiding purchase",
                "Content": "Extract the sales script used to guide the customer to purchase a car from the conversation",
                "Identity": "Sales"
            }
        ]
    }
    parameters['ContentExtraction'] = content_extraction
    root['Parameters'] = parameters
    return root


body = init_parameters()
print(body)

# TODO: Set your AccessKeyId and AccessKeySecret as environment variables.
credentials = AccessKeyCredential(os.environ['ALIBABA_CLOUD_ACCESS_KEY_ID'], os.environ['ALIBABA_CLOUD_ACCESS_KEY_SECRET'])
client = AcsClient(region_id='cn-beijing', credential=credentials)

request = create_common_request('tingwu.cn-beijing.aliyuncs.com', '2023-09-30', 'https', 'PUT', '/openapi/tingwu/v2/tasks')
request.add_query_param('type', 'offline')

request.set_content(json.dumps(body).encode('utf-8'))
response = client.do_action_with_exception(request)
print("response: \n" + json.dumps(json.loads(response), indent=4, ensure_ascii=False))
package com.alibaba.tingwu.client.demo.aitest;

import com.alibaba.fastjson.JSONObject;
import com.aliyuncs.CommonRequest;
import com.aliyuncs.CommonResponse;
import com.aliyuncs.DefaultAcsClient;
import com.aliyuncs.IAcsClient;
import com.aliyuncs.exceptions.ClientException;
import com.aliyuncs.http.FormatType;
import com.aliyuncs.http.MethodType;
import com.aliyuncs.http.ProtocolType;
import com.aliyuncs.profile.DefaultProfile;
import org.junit.Test;

/**
 * @author tingwu2023
 */
public class ContentExtractionTest {

    @Test
    public void testContentExtraction() throws ClientException {
        CommonRequest request = createCommonRequest("tingwu.cn-beijing.aliyuncs.com", "2023-09-30", ProtocolType.HTTPS, MethodType.PUT, "/openapi/tingwu/v2/tasks");
        request.putQueryParameter("type", "offline");

        JSONObject root = new JSONObject();
        root.put("AppKey", "Enter the AppKey that you created in the Tingwu console");

        JSONObject input = new JSONObject();
        input.fluentPut("FileUrl", "Enter the URL of the audio file to test")
                .fluentPut("SourceLanguage", "cn")
                .fluentPut("TaskKey", "task" + System.currentTimeMillis());
        root.put("Input", input);

        JSONObject parameters = new JSONObject();
        parameters.put("ContentExtractionEnabled", true);
        JSONObject contentExtraction = new JSONObject();
        contentExtraction.fluentPut("SceneIntroduction", "Offline car outlet sales scenario")
                .fluentPut("ExtractionContents", new JSONArray()
                        .fluentAdd(new JSONObject().fluentPut("Title", "Customer price objections").fluentPut("Content", "Customer mentions car model prices, promotional offers, or concerns about future price drops").fluentPut("Identity", "Customer"))
                        .fluentAdd(new JSONObject().fluentPut("Title", "Competitor model feedback").fluentPut("Content", "Summarize all competitor brands or models that the customer explicitly mentioned, and summarize the customer's thoughts on them"))
                        .fluentAdd(new JSONObject().fluentPut("Title", "Sales script for guiding purchase").fluentPut("Content", "Extract the sales script used to guide the customer to purchase a car from the conversation").fluentPut("Identity", "Sales"))
                );
        parameters.put("ContentExtraction", contentExtraction);
        root.put("Parameters", parameters);
        System.out.println(root.toJSONString());
        request.setHttpContent(root.toJSONString().getBytes(), "utf-8", FormatType.JSON);

        // TODO: Set your AccessKeyId and AccessKeySecret as environment variables.
        DefaultProfile profile = DefaultProfile.getProfile("cn-beijing", System.getenv("ALIBABA_CLOUD_ACCESS_KEY_ID"), System.getenv("ALIBABA_CLOUD_ACCESS_KEY_SECRET"));
        IAcsClient client = new DefaultAcsClient(profile);
        CommonResponse response = client.getCommonResponse(request);
        System.out.println(response.getData());
    }

    public static CommonRequest createCommonRequest(String domain, String version, ProtocolType protocolType, MethodType method, String uri) {
        // Create an API request and set its parameters.
        CommonRequest request = new CommonRequest();
        request.setSysDomain(domain);
        request.setSysVersion(version);
        request.setSysProtocol(protocolType);
        request.setSysMethod(method);
        request.setSysUriPattern(uri);
        request.setHttpContentType(FormatType.JSON);
        return request;
    }
}

Example output

{
    "Message": "success", 
    "Code": "0", 
    "Data": {
        "Result": {
            "Transcription": "https://speech-swap-hangzhou.oss-cn-hangzhou.aliyuncs.com/tingwu/output/1503864348104017/05c45066fc6d496dae9b583426fdaae8/05c45066fc6d496dae9b583426fdaae8_Transcription_20231028230430.json", 
            "ContentExtraction": "https://speech-swap-hangzhou.oss-cn-hangzhou.aliyuncs.com/tingwu/output/1503864348104017/05c45066fc6d496dae9b583426fdaae8/05c45066fc6d496dae9b583426fdaae8_ContentExtraction_20231028230459.json"
        }, 
        "TaskId": "05c45066fc6df96dg09bf8z4*********", 
        "TaskStatus": "COMPLETED"
    }, 
    "RequestId": "7AE5CB5C-7287-16D1-BA93-G43********"
}

The `ContentExtraction` field contains the HTTPS URL for downloading the conversation content extraction result.

Protocol parsing

The conversation content extraction result is a JSON message. The following is an example.

{
    "TaskId": "4ee872e72fd0490694f1cd6*********",
    "ContentExtraction": [
        {
            "Title": "Customer price objections",
            "Result": "The customer expressed interest in the car model's price and promotional offers. They asked about the specific discount amount, trade-in subsidy for scrapped cars, loan interest, total cost, and discount variations for different models.",
            "Remarks": "The customer repeatedly mentioned price and discount-related issues during the conversation. This shows their sensitivity to the purchase cost and consideration of value for money.",
            "MatchedSentenceIds": [2, 4, 10, 21, 34, 54, 89, 123, 129, 130, 139, 147, 154, 150, 160]
        },
        {
            "Title": "Competitor model feedback",
            "Result": "Not mentioned",
            "Remarks": "The customer did not mention any competitor brands or models, nor did they express any thoughts on competitors during the conversation.",
            "MatchedSentenceIds": []
        },
        {
            "Title": "Estimated purchase time",
            "Result": "Before the end of the year",
            "Remarks": "The customer stated they plan to buy a new car during the year-end promotion when the price is most favorable.",
            "MatchedSentenceIds": [3, 5, 6, 7]
        }
    ]
}

The fields are described as follows:

Parameter

Type

Description

TaskId

string

The `TaskId` generated when the task was created.

ContentExtraction

list[]

A collection of conversation content extraction results. It can contain zero, one, or more result items.

ContentExtraction[i].Title

string

The name of the conversation content extraction result. It corresponds to the `ContentExtraction.ExtractionContents[i].Title` input parameter.

ContentExtraction[i].Result

string

The content extraction result.

ContentExtraction[i].Remarks

string

The analysis of this extraction item by the Large Language Model (LLM).

ContentExtraction[i].MatchedSentenceIds

list[]

The sentence IDs from the original conversation that match this content.