Gradient boosting decision tree (GBDT) algorithm
This topic describes the gradient boosting decision tree (GBDT) algorithm.
Introduction
GBDT is a decision tree algorithm based on gradient boosting. It provides high interpretability and high prediction speed. Compared to other algorithms, GBDT requires less feature engineering. You do not need to standardize features or worry about dependencies between them. The algorithm is also robust and handles missing data well.
Scenarios
GBDT is commonly used in scenarios such as binary classification, multiclass classification, and sorting.
For example, personalized product recommendations often require click prediction models. These models are created using data from past user behaviors, such as clicks, impressions without clicks, and purchases. The model then predicts the probability that a user will click a product. This can be treated as a binary classification problem. GBDT is used for modeling, where the numbers 0 and 1 represent whether a product is clicked. Features are extracted based on user behavior and user properties. These features can include user popularity, product price, and product purchase rate. Such features are typically integers or floating-point numbers.
Parameter description
The following table describes the values for the model_parameter parameter in the CREATE MODEL syntax. You can select the parameters as needed.
Parameter | Description |
loss | The type of weak learning. Valid values are:
|
n_estimators | The number of trees. The value must be a positive integer. The default value is 100. |
learning_rate | The learning rate. The value must be a floating-point number. The default value is 0.06. |
max_leaf_nodes | The maximum number of leaf nodes in a tree. The value can be null or an integer. The default value is null, which means there is no limit on the number of leaf nodes. |
min_samples_leaf | The minimum number of samples required for a leaf node. The value must be a positive integer. The default value is 1. Note If the number of samples in a leaf node is less than the minimum required, the node and its sibling are pruned. |
subsample | The proportion of the total samples used for model creation. The value ranges from 0 to 1. The default value is 1. Note If this value is less than 1, only this proportion of samples is used for model creation. |
max_features | The proportion of the total features used for model creation. The value must be a floating-point number. The value ranges from 0 to 1. The default value is 1. |
max_depth | The maximum depth of the tree. The default value is 1. The value must be an integer. A larger value results in higher accuracy, but a value that is too high can lead to overfitting. |
random_state | The random number seed. The value must be an integer. The default value is 1. |
model_type | The storage class of the model. Valid values are:
|
Examples
Model creation and offline training
/*polar4ai*/CREATE MODEL airline_gbdt WITH
(model_class = 'gbdt',
x_cols = 'Time,Length',
y_cols='Delay',model_parameter=(n_estimators=200))
AS (SELECT * FROM db4ai.airlines);Model evaluation
/*polar4ai*/SELECT Airline FROM EVALUATE(MODEL airline_gbdt,
SELECT * FROM db4ai.airlines LIMIT 20) WITH
(x_cols = 'Time,Length',y_cols='Delay',metrics='acc');Model prediction
/*polar4ai*/SELECT Airline FROM PREDICT(MODEL airline_gbdt,
SELECT * FROM db4ai.airlines limit 20) WITH
(x_cols = 'Time,Length');