Conditional random field-Platform For AI(PAI)-阿里云帮助中心

The Conditional Random Field (CRF) component in Machine Learning Designer trains linear chain CRF models for sequence labeling tasks. A linear chain CRF is a conditional probability distribution model where output variables constitute a Markov random field (MRF). CRFs can be used in different prediction scenarios. The linear chain CRF is mostly used, especially in annotation scenarios. For background, see Conditional random field.

Configure the component

Configure the Conditional Random Field component using the pipeline page or PAI commands.

Method 1: Configure on the pipeline page

On the pipeline page of Machine Learning Designer (formerly Machine Learning Studio) in Platform for AI (PAI), set the following parameters.

Tab	Parameter	Description
Fields setting	ID columns	The column that contains the ID of each sample. Samples are stored in n-tuples.
	Feature columns	The word to be annotated and its features, if the word has features.
	Target columns	The column that you want to select.
Parameters setting	Feature generation template	The template that defines which context features are generated. Default: `[-2:0],[-1:0],[0:0],[1:0],[2:0],[-1:0]/[0:0],[0:0]/[1:0],[-2:1],[-1:1],[0:1],[1:1],[2:1],[-2:1]/[-1:1],[-1:1]/[0:1],[0:1]/[1:1],[1:1]/[2:1],[-2:1]/[-1:1]/[0:1],[-1:1]/[0:1]/[1:1],[0:1]/[1:1]/[2:1]`
	Infrequently used word filtering threshold	Minimum feature frequency. Features with a count below this value are discarded. Default: `1`.
	L1 regularization coefficient	Weight for L1 regularization. Higher values increase sparsity. Default: `1`.
	L2 regularization coefficient	Weight for L2 regularization. Default: `0`.
	Maximum iterations	Maximum number of L-BFGS optimization iterations. Default: `100`.
	Convergence threshold	Stops training when the change in log-likelihood between iterations falls below this value. Default: `0.00001`.
Tuning	Cores	Number of cores. Determined by the system by default.
Tuning	Memory size per core	Memory per core. Determined by the system by default.

Understanding the feature generation template

Each item in the template follows the format [row_offset:col_index], where:

Field	Meaning	Examples
`row_offset`	Position relative to the current token. `0` = current token, `-1` = previous token, `1` = next token.	`-2`, `-1`, `0`, `1`, `2`
`col_index`	Column index in the feature columns. `0` = first feature column, `1` = second feature column.	`0`, `1`

Items joined by / form a conjunction feature that combines multiple positions into a single feature.

Template item	Meaning
`[0:0]`	Value of feature column 0 at the current token
`[-1:0]`	Value of feature column 0 at the previous token
`[-1:0]/[0:0]`	Conjunction of feature column 0 at the previous and current tokens
`[0:1]`	Value of feature column 1 at the current token

Method 2: Use PAI commands

Use the SQL Script component to call PAI commands. For more information, see SQL Script.

Train a model

PAI -name=linearcrf
    -project=algo_public
    -DinputTableName=crf_input_table
    -DidColName=sentence_id
    -DfeatureColNames=word,f1
    -DlabelColName=label
    -DoutputTableName=crf_model
    -Dlifecycle=28
    -DcoreNum=10

Parameter	Required	Default value	Description
`inputTableName`	Yes	—	Table containing the input features.
`inputTablePartitions`	No	All partitions	Partitions to read from the input table.
`featureColNames`	No	All columns except the label column	Feature columns from the input table.
`labelColName`	Yes	—	The label column to predict.
`idColName`	Yes	—	The column that contains sample labels.
`outputTableName`	Yes	—	Table to write the trained model to.
`outputTablePartitions`	No	All partitions	Partitions to write in the output model table.
`template`	No	See Feature generation template	Template that controls feature generation. Format: `<template_item>,<template_item>,...` where each item is `[row_offset:col_index]/[row_offset:col_index]/...`
`freq`	No	`1`	Minimum frequency threshold for features. Features with a count below this value are discarded.
`iterations`	No	`100`	Maximum number of optimization iterations.
`l1Weight`	No	`1.0`	L1 regularization weight. Applies to the L-BFGS algorithm.
`l2Weight`	No	`1.0`	L2 regularization weight. Applies to the L-BFGS algorithm.
`epsilon`	No	`0.0001`	Convergence threshold. Training stops when the log-likelihood change between two consecutive iterations falls below this value. Applies to the L-BFGS algorithm.
`lbfgsStep`	No	`10`	Number of historical steps used by the L-BFGS optimizer. Applies to the L-BFGS algorithm only.
`threadNum`	No	`3`	Number of parallel threads for model training.
`lifecycle`	No	—	Retention period of the output table.
`coreNum`	No	System determines	Number of cores.
`memSizePerCore`	No	System determines	Memory per core.

Run prediction

After training, use the crf_predict command to label new sequences.

PAI -name=crf_predict
    -project=algo_public
    -DinputTableName=crf_test_input_table
    -DmodelTableName=crf_model
    -DidColName=sentence_id
    -DfeatureColNames=word,f1
    -DlabelColName=label
    -DoutputTableName=crf_predict_result
    -DdetailColName=prediction_detail
    -Dlifecycle=28
    -DcoreNum=10

Parameter	Required	Default value	Description
`inputTableName`	Yes	—	Table containing the input features.
`inputTablePartitions`	No	All partitions	Partitions to read from the input table.
`featureColNames`	No	All columns except the label column	Feature columns from the input table.
`labelColName`	No	—	Label column. Optional for prediction.
`IdColName`	Yes	—	The column that contains sample labels.
`resultColName`	No	`prediction_result`	Column name for predicted labels in the output table.
`scoreColName`	No	`prediction_score`	Column name for prediction confidence scores.
`detailColName`	No	—	Column name for per-label score details.
`outputTableName`	Yes	—	Table to write prediction results to.
`outputTablePartitions`	No	All partitions	Partitions to write in the output table.
`modelTableName`	Yes	—	Table containing the trained model.
`modelTablePartitions`	No	All partitions	Partitions to read from the model table.
`lifecycle`	No	—	Retention period of the output table.
`coreNum`	No	System determines	Number of cores.
`memSizePerCore`	No	System determines	Memory per core.

Example

This example trains a CRF model on the CoNLL chunking dataset, then runs prediction on new data.

Input data format

The input table contains one row per token. Each sentence is identified by a shared sentence_id. The label column uses IOB2 notation to mark chunk boundaries:

Label	Meaning
`B-<TYPE>`	First token of a chunk of type `<TYPE>`
`I-<TYPE>`	Continuation token inside a chunk
`O`	Token outside any chunk

sentence_id	word	f1	label
1	Rockwell	NNP	B-NP
1	International	NNP	I-NP
1	Corp	NNP	I-NP
1	's	POS	B-NP
...	...	...	...
823	Ohio	NNP	B-NP
823	grew	VBD	B-VP
823	3.8	CD	B-NP
823	%	NN	I-NP
823	.	.	O

Output data

Note

The label column is optional in the output.

sentence_id	word	f1	label
1	Confidence	NN	B-NP
1	in	IN	B-PP
1	the	DT	B-NP
1	pound	NN	I-NP
...	...	...	...
77	have	VBP	B-VP
77	announced	VBN	I-VP
77	similar	JJ	B-NP
77	increases	NNS	I-NP
77	.	.	O