Create an experiment

Prerequisites

• An experiment project has been created. For more information, see Create an experiment project.

• A source table has been registered. For more information, see Register a source table.

• Experiment metrics have been created. For more information, see Create experiment metrics.

Create an experiment

1. Go to the ABTest page.

   1. Log on to the PAI console.

   2. In the left-side navigation pane, click Workspaces. On the Workspaces page, click the name of the workspace that you want to manage.

   3. In the left-side navigation pane, choose Model Deployment > PAI-ABTest.

2. In the left-side navigation pane of the PAI-ABTest page, choose Experiment Management > Experiments. On the Experiments page, click Create Experiment.

3. On the Create Experiment page, configure the following parameters.

   1. In the Basic Information step, configure the following parameters and then click Next.

      Parameter	Description
      Name	Enter a name for the experiment.
      Description	Enter a description to distinguish this experiment from others.
      Experiment project	Select an experiment project. If none are available, click Create Project.
      Experiment Layers	After you select an experiment project, select an experiment layer. If none are available, click Create Experiment Layer.
      Traffic Allocation	The percentage of traffic from the experiment layer allocated to this experiment. Valid values: 0% to 100%.
      Experiment period	Select the start and end dates for the experiment.
      User group	Configure the experiment whitelist: Enter UIDs for the whitelist and press Enter. You can enter a single UID or multiple UIDs separated by commas (,), for example, UID1,UID2. Alternatively, select an existing user group. To create one, click Create User Group.

   2. In the Experiment Configuration step, configure the following parameters and then click Next.

      In this step, configure up to five versions: one control version and up to four experiment versions. Set the traffic percentage and experiment parameters for each version as described in the following table:

      Parameter		Description
      Traffic Allocation		Select the Even Allocation checkbox to distribute experiment traffic equally among the configured versions.
      Version Configurations	Traffic Proportion	The percentage of experiment traffic allocated to this version.
      	Version name	The name of the version.
      	Version Description	Enter a description to distinguish this version from others.
      	User group	Configure the experiment whitelist: Enter UIDs for the whitelist and press Enter. You can enter a single UID or multiple UIDs separated by commas (,), for example, UID1,UID2. Alternatively, select an existing user group. To create one, click Create User Group.
      	Parameter Configuration	Add parameters and set their data types in the control version. These parameters and data types are automatically applied to all other versions. You must then set the parameter values for each version.

   3. In the Experiment metrics step, configure the Core Metrics and Followed metrics.

      Select one core metric and multiple followed metrics. The experiment report displays the data differences for these metrics across versions. Selecting metrics is optional.

4. Click Completed. The experiment you created appears in the experiment list with a Status of Preparing.

Manage an experiment

You can view and manage experiments in the experiment list.

View experiment details

Click an experiment's name to go to its details page, where you can view its basic information and experiment report:

• On the Basic Information tab, view the basic information and version-related parameters of the experiment. You can also click Edit to modify the configuration.

• On the Experiment report tab, view the experiment report.

Start an experiment

The initial status of an experiment is Preparing. To start the experiment:

1. In the Actions column of the target experiment, click Start.

2. In the Note dialog box, click OK. The experiment status automatically changes to Running, Pending, or Stopped based on the current time.

   • If the current time is within the experiment period, the status changes to Running.

   • If the current time is before the experiment's start date, the status changes to Pending. The system automatically changes the status to Running when the experiment period begins.

   • If the current time is after the experiment's end date, the status changes to Stopped.

Clone an experiment

Click Clone in the Actions column of an experiment to create a copy that reuses its configuration.

Delete an experiment

You can delete only experiments that are not in the Running status. To remove an experiment, click Delete in its Actions column.

Related documents

After starting an experiment, view its reports regularly. For more information, see View experiment reports.