Abuse detection service

更新时间:
复制 MD 格式

The abuse detection service is used for scenarios such as quality inspection of customer service calls and monitoring of live streams.

Note

This service is provided by the NLP Self-Learning Platform. You can use this service by calling the API directly.

Service activation and resource plan purchase

Before you start, ensure that the service is activated. After you activate the service, you must purchase a resource plan.

Service invocation and testing

For more information about how to invoke the model, see Model Invocation.

For SDK examples, see SDK Examples.

Test

You can run this API operation directly in the OpenAPI Developer Portal to avoid calculating a signature. After a successful call, the OpenAPI Developer Portal automatically generates an SDK code example.

Configure access credentials using environment variables

  1. Notes:

    1. The AccessKey of an Alibaba Cloud account has permissions for all APIs. This poses a high security risk. We recommend that you create and use a Resource Access Management (RAM) user for API access or daily O&M. To create a RAM user, log on to the RAM console.

    2. Do not hardcode your AccessKey ID and AccessKey secret into your code. This practice can lead to security risks if the keys are leaked. Instead, use environment variables to store and access your AccessKey pair.

  2. Configuration for Linux and macOS

    export NLP_AK_ENV=<access_key_id>
    export NLP_SK_ENV=<access_key_secret>

    Replace <access_key_id> with your AccessKey ID and <access_key_secret> with your AccessKey secret. For more information about how to obtain an AccessKey ID and an AccessKey secret, see Step 2: Obtain an AccessKey for an account.

  3. Configuration for Windows

    1. Create a file to configure the environment variables. Add the environment variables NLP_AK_ENV and NLP_SK_ENV, and set their values to your AccessKey ID and AccessKey secret.

    2. Restart Windows.

Java code example

/**
 * An AccessKey for an Alibaba Cloud account has permissions for all APIs. This poses a high security threat. 
 * Create and use a RAM user for API access or daily O&M. Log on to the RAM console to create a RAM user.
 * This example shows how to store the AccessKey ID and AccessKey secret in environment variables. 
 * You can also store them in a configuration file as needed.
 * Do not hardcode your AccessKey ID and AccessKey secret into your code. This can cause security threats if the keys are leaked.
 */
String accessKeyId = System.getenv("NLP_AK_ENV");
String accessKeySecret = System.getenv("NLP_SK_ENV");
DefaultProfile defaultProfile = DefaultProfile.getProfile("cn-hangzhou",accessKeyId,accessKeySecret);
IAcsClient client = new DefaultAcsClient(defaultProfile);
String content = "{\"session_id\": 0, \"text\": \"No, no, that sentence of yours, ** is provocative again. Don't say everything I say is provocative, **, why should I be afraid of you? **, I said it, so what, ** I'm not afraid of a road with no delay, *** what's wrong with saying that, ** nothing, nothing, nothing, I'm just saying.\"}";
RunPreTrainServiceRequest request = new RunPreTrainServiceRequest();
request.setServiceName("NLP-Dialog-Risk");
request.setPredictContent(content);
RunPreTrainServiceResponse response = client.getAcsResponse(request);
System.out.println(response.getPredictResult());

Python code example

# Install dependencies
pip install aliyun-python-sdk-core
pip install aliyun-python-sdk-nlp-automl
# -*- coding: utf8 -*-
import json
import os

from aliyunsdkcore.client import AcsClient
from aliyunsdkcore.acs_exception.exceptions import ClientException
from aliyunsdkcore.acs_exception.exceptions import ServerException
from aliyunsdknlp_automl.request.v20191111 import RunPreTrainServiceRequest

"""
An AccessKey for an Alibaba Cloud account has permissions for all APIs. This poses a high security threat. 
Create and use a RAM user for API access or daily O&M. Log on to the RAM console to create a RAM user.
This example shows how to store the AccessKey ID and AccessKey secret in environment variables. 
You can also store them in a configuration file as needed.
Do not hardcode your AccessKey ID and AccessKey secret into your code. This can cause security threats if the keys are leaked.
"""
access_key_id = os.environ['NLP_AK_ENV']
access_key_secret = os.environ['NLP_SK_ENV']

# Initialize AcsClient instance
client = AcsClient(
  access_key_id,
  access_key_secret,
  "cn-hangzhou"
);
content = {"session_id": 0, "text": "No, no, that sentence of yours, ** is provocative again. Don't say everything I say is provocative, **, why should I be afraid of you? **, I said it, so what, ** I'm not afraid of a road with no delay, *** what's wrong with saying that, ** nothing, nothing, nothing, I'm just saying."}
# Initialize a request and set parameters
request = RunPreTrainServiceRequest.RunPreTrainServiceRequest()
request.set_ServiceName('NLP-Dialog-Risk')
request.set_PredictContent(json.dumps(content))
# Print response
response = client.do_action_with_exception(request)
resp_obj = json.loads(response)
predict_result = json.loads(resp_obj['PredictResult'])
print(predict_result['result'])

Example of PredictContent

{
  "session_id": 0, 
  "text": "No, no, that sentence of yours, ** is provocative again. Don't say everything I say is provocative, **, why should I be afraid of you? **, I said it, so what, ** I'm not afraid of a road with no delay, *** what's wrong with saying that, ** nothing, nothing, nothing, I'm just saying."
}

Example of PredictResult

{
  "label": "abuse", 
  "keywords": [
    {"end": "64", "begin": "61", "value": "****"}, 
    {"end": "11", "begin": "9", "value": "****"}, 
    {"end": "38", "begin": "35", "value": "****"}
  ], 
  "session_id": 0, 
  "prob": "0.542"
}

Request parameters

Parameter

Description

session_id

The ID used to track the test sample.

text

The text sample to detect.

Response parameters

Parameter

Description

label

The detected label category.

abuse: Indicates that profanity was detected.

normal: Indicates that the text is normal and no profanity was detected.

keywords

A list of keywords related to abuse in the text. Each element includes the start position, end position, and the value of the keyword.

session_id

The ID of the detected sample.

prob

The probability of abuse. The value is between 0 and 1. A larger value indicates a higher confidence level for abuse.