Data preparation

Models learn patterns from labeled data. Before you can train a model, you must prepare a labeled dataset. In a labeled dataset, the entities and their relationships are identified in the source text. For example, to extract relationships between people from the sentence "Xiao Cheng and Xiao Yun joined Alibaba DAMO Academy on the same day.", you would annotate "Xiao Cheng" and "Xiao Yun" as entities and label their relationship as "colleague". You must provide at least 500 valid labeled entries for each entity.

If you already have a labeled dataset, organize it into the format supported by the platform and upload it. If you do not have a labeled dataset, you can upload raw files and annotate them on the annotation platform. For more information about using the annotation platform, see the next section. The labeled dataset that you upload must be in JSON format and use the following structure. The `content` field contains the text content. The `records` field contains the entity labels. In the `records` field, the key is the entity name and the value is the offset of the annotated content in the source text. The `relations` field contains the entity relationships.

{
    "id": "1", // This is the unique identifier for the document in your dataset.
    "content": "Alibaba's Jack Ma came to Suzhou for a visit.", 
    "records": {
        "LOC": [
            {
                "span": "Suzhou", 
                "index": "T3", 
                "offset": [ // The byte offset of the annotated field. The offset of the first byte is 0.
                    8, 
                    10
                ]
            }
        ], 
        "ORG": [
            {
                "span": "Alibaba", 
                "index": "T1", 
                "offset": [
                    0, 
                    4
                ]
            }
        ], 
        "PER": [
            {
                "span": "Jack Ma", 
                "index": "T2", 
                "offset": [
                    4, 
                    6
                ]
            }
        ], 
        "relations": {
            "Parent": [
                {
                    "index": "R1", 
                    "from": "T1", 
                    "to": "T2"
                }
            ]
        }
    }
}

To upload files, navigate to the project that you created and choose Data Center > Upload Dataset.