Covariance-Platform For AI(PAI)-阿里云帮助中心

Covariance is a statistical method that measures the linear relationship between two random variables. It shows how two variables change in relation to each other by calculating the expected value of the product of their deviations. Covariance is an important concept in probability theory and statistics. It is widely used in machine learning for tasks such as feature selection and data pre-processing.

Algorithm description

Definition of covariance

Covariance is the expected value of the product of the deviations of two random variables. The formula is as follows:

$co v (X, Y) = E [(X - μ) (Y - ν)]$ In this formula:

X and Y are two random variables.
μ and ν are the expected values of X and Y.
E is the expectation operation.

Properties of covariance

Positive covariance: The two variables are positively correlated. When one variable increases, the other also tends to increase.
Negative covariance: The two variables are negatively correlated. When one variable increases, the other tends to decrease.
Zero covariance: The two variables have no linear relationship.

Component configuration

Method 1: Use the visual interface

You can add the Covariance component in the Designer workflow and configure its parameters in the pane on the right:

Parameter type	Parameter	Description
Fields setting	Input columns	The input columns. Only columns of the BIGINT or DOUBLE data type are supported.
Tuning	Number of cores	The number of cores for the computation. If you do not specify this parameter, the system automatically allocates the cores.
Tuning	Memory size	The memory size per core, in MB. If you do not specify this parameter, the system automatically allocates the memory.

Method 2: Use PAI commands

You can use PAI commands to configure the parameters for the Covariance component. You can use the SQL script component to call PAI commands. For more information, see Scenario 4: Execute PAI commands in an SQL script component.

PAI -name cov
    -project algo_public
    -DinputTableName=maple_test_cov_basic12x10_input
    -DoutputTableName=maple_test_cov_basic12x10_output
    -DcoreNum=6
    -DmemSizePerCore=110;

Parameter name	Required	Default value	Description
inputTableName	Yes	None	The name of the input table.
inputTablePartitions	No	All partitions of the input table	The partitions in the input table to use for training. The following formats are supported: partition_name=value name1=value1/name2=value2: multi-level partitions Note To specify multiple partitions, separate them with commas (,), for example, name1=value1,value2.
outputTableName	Yes	None	The name of the output table.
selectedColNames	No	All columns	The names of the columns to select from the input table.
lifecycle	No	None	The lifecycle of the output table.
coreNum	No	Automatically calculated	The number of cores for the computation. The value must be a positive integer from 1 to 9999.
memSizePerCore	No	Automatically calculated	The memory size per core. The value must be an integer from 1 to 65536. Unit: MB.