Collaborative filtering (etrec)

How it works

1. Read user-item interaction records from the input table (user column + item column).

2. For each item, identify users who interacted with it and compute similarity scores against all other items using the selected similarity type.

3. Write the top N similar items per item to the output table as key-value pairs (item_id:similarity_score).

Choose a similarity type

Select the similarity algorithm based on your interaction data characteristics:

Configure the component

Two configuration methods are available: the PAI console (visual) and PAI commands (SQL-based). Both expose the same parameters.

PAI -name pai_etrec
    -project algo_public
    -DsimilarityType="wbcosine"
    -Dweight="1"
    -DminUserBehavior="2"
    -Dlifecycle="28"
    -DtopN="2000"
    -Dalpha="0.5"
    -DoutputTableName="etrec_test_result"
    -DmaxUserBehavior="500"
    -DinputTableName="etrec_test_input"
    -DuserColName="user"
    -DitemColName="item"

Example

This example creates a minimal input table, runs the collaborative filtering algorithm, and inspects the output.

Step 1: Create the training data table.

DROP TABLE IF EXISTS etrec_test_input;
CREATE TABLE etrec_test_input AS
SELECT *
FROM (
    SELECT CAST(0 AS STRING) AS user, CAST(0 AS STRING) AS item
    UNION ALL SELECT CAST(0 AS STRING), CAST(1 AS STRING)
    UNION ALL SELECT CAST(1 AS STRING), CAST(0 AS STRING)
    UNION ALL SELECT CAST(1 AS STRING), CAST(1 AS STRING)
) a;

The resulting etrec_test_input table:

Step 2: Run the PAI command.

DROP TABLE IF EXISTS etrec_test_result;
PAI -name pai_etrec
    -project algo_public
    -DsimilarityType="wbcosine"
    -Dweight="1"
    -DminUserBehavior="2"
    -Dlifecycle="28"
    -DtopN="2000"
    -Dalpha="0.5"
    -DoutputTableName="etrec_test_result"
    -DmaxUserBehavior="500"
    -DinputTableName="etrec_test_input"
    -DuserColName="user"
    -DitemColName="item";

Step 3: View the output table.

Query etrec_test_result. Each row represents an item and its most similar items, formatted as item_id:similarity_score key-value pairs.

Item 0 and item 1 are both interacted with by the same two users, so each has a similarity score of 1.0 with the other.