Get started with cold start ranking-Platform For AI(PAI)-阿里云帮助中心

Step 1: Contextual Bandit algorithm applicability

The Contextual Bandit algorithm can solve the cold start problem. However, the algorithm may not converge effectively if certain conditions are not met.

Note:

Ensure sufficient exploration traffic. In a scenario with a large and continuous influx of new items but limited exploration traffic, the algorithm may perform poorly. For example, this can happen if only one position in each result list is available for new items and the overall traffic is low. On average, each new item should receive several dozen impressions. If this requirement cannot be met, consider not using the item itself as the arm in your model.
Consider your modeling strategy. Typically, you can use the item itself as the arm for the Contextual Bandit algorithm. However, if traffic is insufficient, use coarser-grained item attributes, such as themes or categories, as arms. The algorithm then determines which arm to display, and a separate policy can be used to select a specific item from that arm, such as displaying the newest one.
Determine if cold start ranking is necessary. If conditions are not ideal, you can still achieve good results with only a model-based cold start candidate generation method, such as a DropoutNet model. You can enhance this model with an exploration mechanism. For example, to display N new items, you can use a vector k-nearest neighbors (k-NN) algorithm to match M (where M > N) new items. Retain the K items with the highest similarity (shortest distance) as initial candidates. Then, sample an additional N-K items based on their Softmax similarity probabilities and add them to the candidate set.
Sampling candidate items based on similarity score probabilities ensures relevance while incorporating an exploration mechanism.

Step 2: Create Hologres tables

1. Create a feature table

BEGIN;
CREATE TABLE rec.contextual_bandit_features (
    request_id text NOT NULL,
    user_id bigint NOT NULL,
    arm_id bigint NOT NULL,
    recom_time timestamp with time zone NOT NULL,
    feature text NOT NULL
    ,PRIMARY KEY (user_id, arm_id)
);
CALL set_table_property('rec.contextual_bandit_features', 'orientation', 'row');
CALL set_table_property('rec.contextual_bandit_features', 'clustering_key', 'user_id:asc,arm_id:asc');
CALL set_table_property('rec.contextual_bandit_features', 'time_to_live_in_seconds', '259200');
CALL set_table_property('rec.contextual_bandit_features', 'distribution_key', 'user_id,arm_id');
CALL set_table_property('rec.contextual_bandit_features', 'storage_format', 'sst');
COMMENT ON TABLE rec.contextual_bandit_features IS 'Feature table for the cold start algorithm';
COMMENT ON COLUMN rec.contextual_bandit_features.request_id IS 'Unique ID for each request';
COMMENT ON COLUMN rec.contextual_bandit_features.user_id IS 'User ID';
COMMENT ON COLUMN rec.contextual_bandit_features.arm_id IS 'item_id in most scenarios';
COMMENT ON COLUMN rec.contextual_bandit_features.recom_time IS 'Request time';
COMMENT ON COLUMN rec.contextual_bandit_features.feature IS 'Serialized feature vector';
END;

2. Create a model table

BEGIN;
CREATE TABLE rec.contextual_bandit_models (
    arm_id bigint NOT NULL,
    version bigint NOT NULL,
    invert_matrix_a integer[] NOT NULL,
    vector_b integer[] NOT NULL,
    matrix_b integer[]
    ,PRIMARY KEY (arm_id)
);
CALL set_table_property('rec.contextual_bandit_models', 'orientation', 'column');
CALL set_table_property('rec.contextual_bandit_models', 'clustering_key', 'arm_id:asc');
CALL set_table_property('rec.contextual_bandit_models', 'bitmap_columns', 'arm_id');
CALL set_table_property('rec.contextual_bandit_models', 'time_to_live_in_seconds', '259200');
CALL set_table_property('rec.contextual_bandit_models', 'storage_format', 'orc');
CALL set_table_property('rec.contextual_bandit_models', 'distribution_key', 'arm_id');
CALL set_table_property('rec.contextual_bandit_models', 'segment_key', 'version');
COMMENT ON TABLE rec.contextual_bandit_models IS 'Parameter table for the cold start model';
COMMENT ON COLUMN rec.contextual_bandit_models.arm_id IS 'Arm ID';
COMMENT ON COLUMN rec.contextual_bandit_models.version IS 'Algorithm version: timestamp';
COMMENT ON COLUMN rec.contextual_bandit_models.invert_matrix_a IS 'LinUCB algorithm parameter: inverse of matrix A';
COMMENT ON COLUMN rec.contextual_bandit_models.vector_b IS 'LinUCB algorithm parameter: vector b';
COMMENT ON COLUMN rec.contextual_bandit_models.matrix_b IS 'LinUCB algorithm parameter: matrix B';
END;

Step 3: Deploy the model training job

Download the JAR package for the Flink training job: ContextualBanditFlink-1.0-SNAPSHOT-jar-with-dependencies.jar.
In the Realtime Compute for Apache Flink console, go to the resource upload page and upload the JAR package.

In the Realtime Compute for Apache Flink console, create a Stream/JAR job on the Job Development page.

In the dialog box, set File Name to cold_start_, Deployment Target to vvp-workload, and Storage Location to the Job Development folder. Then, click Confirm.

Configure the job information. For JAR URI, select the JAR you uploaded. In the Entry Point Class text box, enter com.alibaba.pairec.linucb.Driver. In the Entry Point Main Arguments text box, enter --datasource datahub if your data source is DataHub, or --datasource kafka if your data source is Kafka.

Set the Deployment Target to vvp-workload and the parallelism to 4. In the General Configuration pane on the right, set Engine Version to vvr-4.0.12-flink-1.13. In the Behavior Configuration pane, enable Stop with Drain. In the Flink Configuration pane, set Checkpoint Interval to 1h and select Fixed Delay as the Flink restart strategy.

Configure the training task. In the More Flink Configurations text box in the Advanced Configuration pane, enter the parameters for algorithm training.

Algorithm training parameters include input.event.time.format, input.event.reward.json (event reward weight JSON), input.exposure.event.name, input.event.equal.filter, and input.event.in.filter. In the Flink Restart Strategy Configuration, select Fixed Delay and set the Restart Interval to 60s. In the Resource Configuration, set JobManager CPU Cores to 1 and JobManager Memory to 2Gi.

After completing the configuration, click Save in the upper-left corner, and then click Publish in the upper-right corner to deploy the job.
Go to the Job Operations page to start the job and monitor its status. After deployment, confirm that the status of the cold_start job instance is RUNNING, the restart count is 0, and Used/Total Slots is 4/4. The DAG topology shows Source: Custom Source → Flat Map → Filter → Timestamps/Watermarks (Parallelism: 4) connected via HASH to KeyedProcess → Sink: Holo Sink (Parallelism: 4). If the Source task has sent 9.56 GB / 130,159,586 records, and the KeyedProcess task has received 9.56 GB / 130,111,690 records and output 1,441,004 records, this indicates that the model training job is running successfully.

Step 4: Deploy the model ranking service

Create the configuration file required for the model ranking service, upload it to OSS, and obtain its publicly accessible OSS URL.

Prepare the configuration file for the EAS service. The following is an example of a file named eas_config.json:

{
   "name": "cold_start",
   "containers": [
      {
         "image": "mybigpai-registry.cn-beijing.cr.aliyuncs.com/mybigpai/eas-contextual-bandit:0.0.1-efa1d85",
         "env": [
            {
               "name": "GIN_MODE",
               "value": "release"
            },
            {
               "name": "ALGO_CONFIG",
               "value": "https://${bucket}.oss-cn-beijing-internal.aliyuncs.com/cold_start.json?OSSAccessKeyId=xxxxx&Expires=1001639368243&Signature=XXXX"
            }
         ],
         "port": 8000,
         "command": "/linucb-server -l /log"
      }
   ],
   "metadata": {
      "cpu": 15,
      "instance": 2,
      "memory": 22000
   }
}

The parameters in the containers section are described as follows:

"image": "mybigpai-registry.cn-beijing.cr.aliyuncs.com/mybigpai/eas-contextual-bandit:0.0.1-efa1d85" is the fixed image repository address for the model ranking service.
ALGO_CONFIG: Specifies the configuration file for the model ranking service, which must be a publicly accessible URL.

Use the eascmd command-line tool to create or update the EAS service:

# Create the service
#eval "${eascmd} create ../conf/eas_config.json"
# Update the service
eval "${eascmd} modify cold_start -s ../conf/eas_config.json"
# View the service
eval "${eascmd} desc cold_start"

Step 5: Configure a cold start pipeline

Refer to the PaiRec documentation Customize pipelines to configure a dedicated cold start recommendation pipeline.

 {
   "PipelineConfs": {
    "video_feed": [ // Scenario name. You can configure multiple pipelines for a scenario.
      {
        "Name": "coldstart", // Custom pipeline name. Must be globally unique, including across different scenarios.
        "RecallNames": [], // Candidate generation list. Defined in RecallConfs.
        "FilterNames": [], // Filtering list. Defined in FilterConfs.
        "GeneralRankConf": { // Coarse ranking definition. Refer to coarse ranking configurations.
          "FeatureLoadConfs":[],
          "RankConf":{},
          "ActionConfs": [
            {
              "ActionType": "filter",
              "ActionName": ""
            }
          ]
        },
        "FeatureLoadConfs":[],
        "RankConf":{
          "RankAlgoList":[],
          "RankScore":"",
          "Processor":"",
          "BatchCount": 100
        },
        "ColdStartRankConf": {
          "AlgoName":"",
          "OnlyEmbeddingFeature": true
        },
        "SortNames":[] // Sorting methods. Defined in SortConfs.
      }
    ]
  }
 }

The service merges results from the cold start pipeline with those from the main, regular pipeline.

Step 6: Evaluate new-item cold start performance

The new-item cold start algorithm is item-based. Therefore, you cannot use traffic-splitting A/B testing to evaluate its effectiveness.

benchmark bucket: For new items with an odd-numbered item ID (or hash value), do not use the cold start algorithm. Instead, use regular recommendations or display new items randomly.
experiment bucket: For new items with an even-numbered item ID (or hash value), use a cold start algorithm, such as the LinUCB algorithm.

Choose metrics based on your business goals, such as impressions, click-through rate, retention rate, and playback duration.