Create an annotation task

Prerequisites

• A dataset that contains the data to be annotated has been created. For more information, see Create a dataset.

• Your Alibaba Cloud account has administrative or annotation administrator permissions. For more information about how to grant permissions, see Manage a workspace > Member role configuration.

Create a task

1. Go to iTAG.

   1. Log on to the PAI console.

   2. In the navigation pane on the left, click Workspaces. Click the name of the workspace that you want to manage to open it.

   3. In the navigation pane on the left, choose Data Preparation > iTAG.

2. Create an annotation task.

   By default, PAI uses the classic version of iTAG to create annotation tasks. To annotate data for a Large Language Model (LLM) or use more templates, you can use the new version of iTAG.

   Classic version

   1. Click Create Task to open the Create Annotation Task page.

   2. On the classic annotation task page, configure the following parameters.

      Parameter	Description
      Task Name	The name must be 1 to 100 characters in length. It must start with a letter or a digit and can contain underscores (_) or hyphens (-).
      Dataset Type	The default value is General Dataset.
      Input Dataset	Select a dataset that is created in PAI Dataset Management. Note iTAG does not support selecting a dataset version. Version v1 is used by default.
      Template Type	The following models are supported: General Template: a common template that is preset on the platform. Custom Template: For more information, see Annotation templates.
      Label Configuration	When Template Type is set to General Template, you can select a specific general template type. The following types are supported: Image For more information about the application scenarios and the input and output data formats of this template type, see Image. Image OCR: Performs OCR on the text in the selected area of an image. Object Detection: Locates specific objects in an image. Image Classification: Classifies and tags images based on preset labels. PDF: Performs OCR and tag categorization on PDF files. Review and Matting: Reviews images and performs image matting tasks. Table Recognition: Uses an algorithm to pre-detect core table elements. You can edit the elements as needed. Text For more information about the application scenarios and the input and output data formats of this template type, see Text. Named Entity Recognition: Establishes relationships between text entities. Text Classification: Classifies and tags text based on preset labels. Single-tag and multi-tag classification are supported. Entity Relationship: Defines relationships between text entities for knowledge graph scenarios. Video For more information about the application scenarios and the input and output data formats of this template type, see Video. Video Classification: Classifies and tags videos based on preset labels. Single-tag and multi-tag classification are supported. Audio For more information about the application scenarios and the input and output data formats of this template type, see Audio. Audio Classification: Classifies and tags audio based on preset labels. Single-tag and multi-tag classification are supported. Audio Segmentation: Segments the content of an audio dataset and adds tags to the segments. Audio Recognition: Recognizes text from audio content.
      Select Field for Image Content	Select the field name in the dataset that corresponds to the data content (image, text, video, or audio) of the annotation task.
      OCR Result Configuration	This parameter is available only when you select Image OCR from the Image type for the template. By default, OCR Result is selected, which indicates that OCR is performed on the text in the selected area of an image.
      Tag Configuration	Enter the tag names that annotators need to identify, select, and tag in this annotation task. Press the Enter key to add each tag. For example, when detecting cats in an image, you can add tags such as "cat", "American Shorthair", and "British Shorthair". This helps annotators tag the samples. You can also specify whether an annotator can apply one or multiple tags to a selected object in a sample. If only one tag can be applied to a selected object, set the tag to Single-select. If multiple tags can be applied to a selected object, set the tag to Multi-select. For example, when detecting cats in an image, if the tag is set to Multi-select, the annotator can select a cat in the sample and apply both the "cat" and "American Shorthair" tags. Note Single-select and Multi-select here refer to whether one or multiple tags can be added to a single selected object in a sample, not whether multiple objects can be selected and tagged in a sample.
      Enable Smart Annotation	For more information about the configuration, see Use smart annotation to pre-annotate data.
      Task Description	A brief description of the annotation task. You can link to a document, upload a document or image, or manually enter a description.
      Sub-task Package Allocation	iTAG groups all annotation tasks from the dataset into sub-task packages based on specified rules. During the annotation process, annotators can claim these packages to work on the tasks within them. Currently, sub-task packages can be allocated based on the following rules: Fixed Size: Each sub-task package contains a fixed number of annotation tasks. When you set a fixed size, the relationship between the data volume of the dataset and the number of tasks in a sub-task package is as follows: If the dataset size is 0 to 20,000 data entries, the sub-task package size can be 1 to 200 tasks. If the dataset size is 20,000 to 100,000 data entries, the sub-task package size can be 5 to 200 tasks. If the dataset size is 100,000 to 500,000 data entries, the sub-task package size can be 25 to 200 tasks. If the dataset size is 500,000 to 1,000,000 data entries, the sub-task package size can be 50 to 200 tasks. Import by Field: Sub-task packages are divided based on a selected field in the dataset. Data with the same value in this field is placed in the same sub-task package. Directed Assignment: Sub-task packages are created for specific annotators or teams.
      Distribution Mode	You can select Single-person Annotation or Multi-person Voting. For more information about the differences between the modes, see the description on the console.
      Task Flow	When you select Single-person Annotation, you can choose Annotation, Annotation-Review, Annotation-Acceptance, or Annotation-Review-Acceptance. When you select Multi-person Voting, you can choose Annotation-Review or Annotation-Review-Acceptance.
      Personnel Configuration	Based on the selected Task Flow, configure annotators/groups, reviewers/groups, acceptance personnel/groups, or task administrators. You can collaborate with multiple members in the workspace to complete the annotation task. For more information about role permissions in iTAG, see Assign member roles.
      Sampling Method	You can select Sample Data from Sub-task Package, Sample Sub-task Package, or Custom. For more information about the differences between the sampling methods, see the description on the console.
      Review Ratio	If you select Single-person Annotation and the Task Flow includes a Review step, you must configure the sampling ratio for the annotation tasks in the review step. The default value is 100%.

   3. After you complete the configuration, click Create.

   New version

   1. Click Create Task to open the Create Annotation Task page.

   2. Click to open the Management Center > Task Management page.

   3. Click , select New Version, and then configure the parameters.

   4. Select data

      1. Select Raw Dataset or Labeled Dataset, and then click Next.

         Note

         iTAG does not support selecting a dataset version. By default, version v1 is used.

      2. Preview the data, and then click Next.

   5. Select a template

      You can use an existing template, an official template, a custom template, or create a new template. Configure the template and click Next.

      For more information, see Annotation templates.

   6. Configure the task

      Configure the following parameters, and then click Create Task.

      Parameter		Description
      Basic Information	Task Name	A custom name for the annotation task. The name must be 1 to 100 characters in length, start with a letter or a digit, and can contain underscores (_) or hyphens (-).
      	Distribution Mode	Single-person Annotation: Claim mode. One task package is claimed at a time for annotation. Multi-person Voting: Each task package is repeatedly annotated by multiple people in sequence.
      	Task Flow	The flow includes three steps: annotation, review, and acceptance. Annotation is required. Review and acceptance are optional.
      	Personnel Configuration	Based on the selected Task Flow, configure annotators/groups, reviewers/groups, acceptance personnel/groups, or task administrators. You can collaborate with multiple members in the workspace to complete the annotation task. For more information about role permissions in iTAG, see Assign member roles. You can configure this on the Tenant Configuration>Personnel Management page.
      	Task Description	A brief description of the annotation task. You can link to a document, upload a document or image, or manually enter a description.
      	Priority	The larger the value, the higher the priority.
      	Business Type	A tag used to classify and manage annotation tasks. You can click Business Type on the right to add a business type before setting it.
      	Task Tag	In an annotation task, you can select the tag names that annotators need to identify, select, and tag. You can configure this on the Tenant Configuration>Cloud Resource Configuration page by selecting Tag Configuration.
      Smart Annotation	Annotation Method	You can select Do Not Use, or Use Offline Pre-labeling Results.
      Advanced Configuration	Maximum Skips for Sub-task Package	Set the maximum number of times a sub-task package can be skipped. After this limit is reached, the task cannot be skipped.
      	Discard Sub-task Package on Timeout	If this is enabled, a sub-task package is discarded if it is not claimed within the specified time.
      	Release Sub-task Package on Timeout	If this is enabled, a sub-task package is released if it is not claimed within the specified time.
      	Allow Annotators to Release Task Packages	If this is enabled, annotators are allowed to release tasks they have claimed.
      	Append Data to Task	Appending data to an existing task can prevent frequent task creation and task fragmentation. Only administrators have the permission to perform this operation. For more information about the API call, see AppendAllDataToTask - Append data.

Manage tasks

After you create an annotation task, you can go to the Management Center > Task Management page to view the task status, view sub-task package details, or retrieve the annotation results.