回归

GBDT回归

梯度渐进回归树GBDT（Gradient Boosting Decesion Tree）是一种迭代决策树算法，适用于线性及非线性回归场景。

GBDT回归示例

1. 使用SQL语句，生成测试数据。

   drop table if exists gbdt_ls_test_input;
   create table gbdt_ls_test_input
   as
   select
       *
   from
   (
       select
           cast(1 as double) as f0,
           cast(0 as double) as f1,
           cast(0 as double) as f2,
           cast(0 as double) as f3,
           cast(0 as bigint) as label
       from dual
       union all
           select
               cast(0 as double) as f0,
               cast(1 as double) as f1,
               cast(0 as double) as f2,
               cast(0 as double) as f3,
               cast(0 as bigint) as label
       from dual
       union all
           select
               cast(0 as double) as f0,
               cast(0 as double) as f1,
               cast(1 as double) as f2,
               cast(0 as double) as f3,
               cast(1 as bigint) as label
       from dual
       union all
           select
               cast(0 as double) as f0,
               cast(0 as double) as f1,
               cast(0 as double) as f2,
               cast(1 as double) as f3,
               cast(1 as bigint) as label
       from dual
       union all
           select
               cast(1 as double) as f0,
               cast(0 as double) as f1,
               cast(0 as double) as f2,
               cast(0 as double) as f3,
               cast(0 as bigint) as label
       from dual
       union all
           select
               cast(0 as double) as f0,
               cast(1 as double) as f1,
               cast(0 as double) as f2,
               cast(0 as double) as f3,
               cast(0 as bigint) as label
       from dual
   ) a;

   生成的测试数据表gbdt_ls_test_input如下。

   f0	f1	f2	f3	label
   1.0	0.0	0.0	0.0	0
   0.0	0.0	1.0	0.0	1
   0.0	0.0	0.0	1.0	1
   0.0	1.0	0.0	0.0	0
   1.0	0.0	0.0	0.0	0
   0.0	1.0	0.0	0.0	0

2. 使用PAI命令，提交GBDT回归组件的训练参数。

   drop offlinemodel if exists gbdt_ls_test_model;
   PAI -name gbdt
       -project algo_public
       -DfeatureSplitValueMaxSize="500"
       -DlossType="3"
       -DrandSeed="0"
       -DnewtonStep="1"
       -Dshrinkage="0.5"
       -DmaxLeafCount="32"
       -DlabelColName="label"
       -DinputTableName="gbdt_ls_test_input"
       -DminLeafSampleCount="1"
       -DsampleRatio="1"
       -DmaxDepth="10"
       -DmetricType="0"
       -DmodelName="gbdt_ls_test_model"
       -DfeatureRatio="1"
       -Dp="1"
       -Dtau="0.6"
       -DtestRatio="0"
       -DfeatureColNames="f0,f1,f2,f3"
       -DtreeCount="10"

3. 使用PAI命令，提交预测组件参数。

   drop table if exists gbdt_ls_test_prediction_result;
   PAI -name prediction
       -project algo_public
       -DdetailColName="prediction_detail"
       -DmodelName="gbdt_ls_test_model"
       -DitemDelimiter=","
       -DresultColName="prediction_result"
       -Dlifecycle="28"
       -DoutputTableName="gbdt_ls_test_prediction_result"
       -DscoreColName="prediction_score"
       -DkvDelimiter=":"
       -DinputTableName="gbdt_ls_test_input"
       -DenableSparse="false"
       -DappendColNames="label"

4. 查看预测结果表gbdt_ls_test_prediction_result。

   label	prediction_result	prediction_score	prediction_detail
   0	NULL	0.0	{“label”: 0}
   0	NULL	0.0	{“label”: 0}
   1	NULL	0.9990234375	{“label”: 0.9990234375}
   1	NULL	0.9990234375	{“label”: 0.9990234375}
   0	NULL	0.0	{“label”: 0}
   0	NULL	0.0	{“label”: 0}

线性回归

线性回归示例

1. 使用SQL语句，生成测试数据。

    drop table if exists lm_test_input;
     create table lm_test_input as
     select
       *
     from
     (
       select 10 as y, 1.84 as x1, 1 as x2, '0:1.84 1:1' as sparsecol1 from dual
         union all
       select 20 as y, 2.13 as x1, 0 as x2, '0:2.13' as sparsecol1 from dual
         union all
       select 30 as y, 3.89 as x1, 0 as x2, '0:3.89' as sparsecol1 from dual
         union all
       select 40 as y, 4.19 as x1, 0 as x2, '0:4.19' as sparsecol1 from dual
         union all
       select 50 as y, 5.76 as x1, 0 as x2, '0:5.76' as sparsecol1 from dual
         union all
       select 60 as y, 6.68 as x1, 2 as x2, '0:6.68 1:2' as sparsecol1 from dual
         union all
       select 70 as y, 7.58 as x1, 0 as x2, '0:7.58' as sparsecol1 from dual
         union all
       select 80 as y, 8.01 as x1, 0 as x2, '0:8.01' as sparsecol1 from dual
         union all
       select 90 as y, 9.02 as x1, 3 as x2, '0:9.02 1:3' as sparsecol1 from dual
         union all
       select 100 as y, 10.56 as x1, 0 as x2, '0:10.56' as sparsecol1 from dual
     ) tmp;

2. 使用PAI命令，提交线性回归组件参数。

   PAI -name linearregression
       -project algo_public
       -DinputTableName=lm_test_input
       -DlabelColName=y
       -DfeatureColNames=x1,x2
       -DmodelName=lm_test_input_model_out
       -DoutputTableName=lm_test_input_conf_out
       -DenableCoefficientEstimate=true
       -DenableFitGoodness=true
       -Dlifecycle=1;

3. 使用PAI命令，提交预测组件参数。

   pai -name prediction
       -project algo_public
       -DmodelName=lm_test_input_model_out
       -DinputTableName=lm_test_input
       -DoutputTableName=lm_test_input_predict_out
       -DappendColNames=y;

4. 查看输出的模型评估表lm_test_input_conf_out。

   +------------+------------+------------+------------+--------------------+------------+
   | colname    | value      | tscore     | pvalue     | confidenceinterval | p          |
   +------------+------------+------------+------------+--------------------+------------+
   | Intercept  | -6.42378496687763 | -2.2725755951390028 | 0.06       | {"2.5%": -11.964027, "97.5%": -0.883543} | coefficient |
   | x1         | 10.260063429838898 | 23.270944360826963 | 0.0        | {"2.5%": 9.395908, "97.5%": 11.124219} | coefficient |
   | x2         | 0.35374498323846265 | 0.2949247320997519 | 0.81       | {"2.5%": -1.997160, "97.5%": 2.704650} | coefficient |
   | rsquared   | 0.9879675667384592 | NULL       | NULL       | NULL               | goodness   |
   | adjusted_rsquared | 0.9845297286637332 | NULL       | NULL       | NULL               | goodness   |
   | aic        | 59.331109494251805 | NULL       | NULL       | NULL               | goodness   |
   | degree_of_freedom | 7.0        | NULL       | NULL       | NULL               | goodness   |
   | standardErr_residual | 3.765777749448906 | NULL       | NULL       | NULL               | goodness   |
   | deviance   | 99.26757440771128 | NULL       | NULL       | NULL               | goodness   |
   +------------+------------+------------+------------+--------------------+------------+

5. 查看预测结果表lm_test_input_predict_out。

   +------------+-------------------+------------------+-------------------+
   | y          | prediction_result | prediction_score | prediction_detail |
   +------------+-------------------+------------------+-------------------+
   | 10         | NULL              | 12.808476727264404 | {"y": 12.8084767272644} |
   | 20         | NULL              | 15.43015013867922 | {"y": 15.43015013867922} |
   | 30         | NULL              | 33.48786177519568 | {"y": 33.48786177519568} |
   | 40         | NULL              | 36.565880804147355 | {"y": 36.56588080414735} |
   | 50         | NULL              | 52.674180388994415 | {"y": 52.67418038899442} |
   | 60         | NULL              | 62.82092871092313 | {"y": 62.82092871092313} |
   | 70         | NULL              | 71.34749583130122 | {"y": 71.34749583130122} |
   | 80         | NULL              | 75.75932310613193 | {"y": 75.75932310613193} |
   | 90         | NULL              | 87.1832221199846 | {"y": 87.18322211998461} |
   | 100        | NULL              | 101.92248485222113 | {"y": 101.9224848522211} |
   +------------+-------------------+------------------+-------------------+

PS-SMART回归

参数服务器PS（Parameter Server）致力于解决大规模的离线及在线训练任务，SMART（Scalable Multiple Additive Regression Tree）是GBDT（Gradient Boosting Decesion Tree）基于PS实现的迭代算法。PS-SMART支持百亿样本及几十万特征的训练任务，可以在上千节点中运行。同时，PS-SMART支持多种数据格式及直方图近似等优化技术。

PS-SMART回归示例

1. 使用如下SQL语句，生成输入数据（以KV格式数据为例）。

   drop table if exists smart_regression_input;
   create table smart_regression_input as
   select
   *
   from
   (
   select 2.0 as label, '1:0.55 2:-0.15 3:0.82 4:-0.99 5:0.17' as features from dual
       union all
   select 1.0 as label, '1:-1.26 2:1.36 3:-0.13 4:-2.82 5:-0.41' as features from dual
       union all
   select 1.0 as label, '1:-0.77 2:0.91 3:-0.23 4:-4.46 5:0.91' as features from dual
       union all
   select 2.0 as label, '1:0.86 2:-0.22 3:-0.46 4:0.08 5:-0.60' as features from dual
       union all
   select 1.0 as label, '1:-0.76 2:0.89 3:1.02 4:-0.78 5:-0.86' as features from dual
       union all
   select 1.0 as label, '1:2.22 2:-0.46 3:0.49 4:0.31 5:-1.84' as features from dual
       union all
   select 0.0 as label, '1:-1.21 2:0.09 3:0.23 4:2.04 5:0.30' as features from dual
       union all
   select 1.0 as label, '1:2.17 2:-0.45 3:-1.22 4:-0.48 5:-1.41' as features from dual
       union all
   select 0.0 as label, '1:-0.40 2:0.63 3:0.56 4:0.74 5:-1.44' as features from dual
       union all
   select 1.0 as label, '1:0.17 2:0.49 3:-1.50 4:-2.20 5:-0.35' as features from dual
   ) tmp;

   生成的数据如下。
2. 构建实验，详情请参见算法建模。
3. 配置PS-SMART回归组件的参数（配置如下表格中的参数，其余参数使用默认值）。

   页签	参数	描述
   字段设置	是否稀疏格式	选中是否稀疏格式复选框。
   	特征列	选择features列。
   	标签列	选择label列。
   参数设置	目标函数类别	配置为Linear regression。
   	评估指标类型	选择rooted mean square error。
   	树数量	配置为5。

4. 配置统一预测组件的参数（配置如下表格中的参数，其余参数使用默认值）。

   页签	参数	描述
   字段设置	特征列	默认全选，多余列不影响预测结果。
   	原样输出列	选择为label列。
   	稀疏矩阵	选中稀疏矩阵复选框。
   	key与value分隔符	配置为英文冒号（:）。
   	kv对间的分隔符	配置为空格或\u0020。

5. 配置PS-SMART预测组件的参数（配置如下表格中的参数，其余参数使用默认值）。

   页签	参数	描述
   字段设置	特征列	默认全选，多余列不影响预测结果。
   	原样输出列	选择label列。
   	稀疏矩阵	选中稀疏矩阵复选框。
   	key与value分隔符	配置为英文冒号（:）。
   	kv对间的分隔符	配置为空格或\u0020。

6. 运行实验，查看统一预测组件的预测结果。
7. 查看PS-Smart预测组件的预测结果。其中prediction_score列表示预测值，leaf_index列表示预测的叶子节点编号。
8. 右键单击PS-SMART回归组件，在快捷菜单，选择查看数据 > 查看输出桩3，查看特征重要性。

   其中id列表示传入的特征序号。因为该示例的输入数据是KV格式，所以id列表示KV对中的key。该特性重要性表中仅有2个特性，表示树在分裂过程中仅使用了这两个特性，可以认为其他特性的特征重要性为0。value列表示特征重要性类型，默认为gain，即该特征对模型带来的信息增益之和。

PS线性回归

线性回归（Linear Regression）是分析因变量和多个自变量之间的线性关系模型，参数服务器PS（Parameter Server）致力于解决大规模的离线及在线训练任务。PS线性回归支持千亿样本、十亿特征的大规模线性训练任务。

PS线性回归示例

1. 使用如下SQL语句，生成输入数据（以KV格式数据为例）。

   drop table if exists lm_test_input;
   create table lm_test_input as
   select
   *
   from
   (
   select 2 as label, '1:0.55 2:-0.15 3:0.82 4:-0.99 5:0.17' as features from dual
       union all
   select 1 as label, '1:-1.26 2:1.36 3:-0.13 4:-2.82 5:-0.41' as features from dual
       union all
   select 1 as label, '1:-0.77 2:0.91 3:-0.23 4:-4.46 5:0.91' as features from dual
       union all
   select 2 as label, '1:0.86 2:-0.22 3:-0.46 4:0.08 5:-0.60' as features from dual
       union all
   select 1 as label, '1:-0.76 2:0.89 3:1.02 4:-0.78 5:-0.86' as features from dual
       union all
   select 1 as label, '1:2.22 2:-0.46 3:0.49 4:0.31 5:-1.84' as features from dual
       union all
   select 0 as label, '1:-1.21 2:0.09 3:0.23 4:2.04 5:0.30' as features from dual
       union all
   select 1 as label, '1:2.17 2:-0.45 3:-1.22 4:-0.48 5:-1.41' as features from dual
       union all
   select 0 as label, '1:-0.40 2:0.63 3:0.56 4:0.74 5:-1.44' as features from dual
       union all
   select 1 as label, '1:0.17 2:0.49 3:-1.50 4:-2.20 5:-0.35' as features from dual
   ) tmp;

   生成的数据如下。
   说明 KV格式数据的特征ID必须为正整数，特征值必须为实数。如果特征ID为字符串，则需要进行序列化操作。如果特征值为类别型字符串，则需要进行特征离散化操作。
2. 构建实验，详情请参见算法建模。
3. 配置PS线性回归组件的参数（配置如下表格中的参数，其余参数使用默认值）。

   页签	参数	描述
   字段设置	是否稀疏格式	选择true。
   	选择特征列	选择features列。
   	选择标签列	选择label列。
   执行调优	核心数	配置为3。
   	每个核的内存大小	配置为1024 MB。

4. 配置预测组件的参数（配置如下表格中的参数，其余参数使用默认值）。

   页签	参数	描述
   字段设置	特征列	默认全选，多余列不影响预测结果。
   	原样输出列	选择label列。
   	稀疏矩阵	选中稀疏矩阵复选框。
   	key与value分隔符	配置为英文冒号（:）。
   	kv对间的分隔符	配置为\u0020。

5. 运行实验，查看预测结果。