A/B testing FAQ

Q1: What should I enter for the scene_tag parameter?

A: The name of the experiment group.

Q2: If I run an A/B testing experiment during the day and stop it in the evening, will the next day's report still contain data?

A: Yes, it will.

Q3: What is the purpose of the flow_divider parameter?

A: To direct specific queries to an experiment using an allowlist, enter the allowlist's content in the flow_divider parameter.

Q4: How can I improve the quality of A/B testing metrics?

A: We recommend starting a new experiment only after fully processing all data from previous user activity.

Q5: How do I URL-encode the scene_tag and flow_divider parameters?

Answer: Assume that the value of scene_tag is "test_1", and the value of flow_divider is "traffic bucketing",

This Python 3.x example shows how to URL-encode these parameters:

import urllib
from urllib import parse
print(parse.quote("scene_tag:" + parse.quote("test_1") + ",flow_divider:" + parse.quote("traffic bucket")))
#Result: scene_tag%3Atest_1%2Cflow_divider%3A%25E6%25B5%2581%25E9%2587%258F%2520%25E5%2588%2586%25E6%25A1%25B6

For search testing in the console, use the following string:

scene_tag:test_1,flow_divider:%E6%B5%81%E9%87%8F%20%E5%88%86%E6%A1%B6

On the Search Test page, in the Parameters section, enter the encoded string into the abtest field (i.e., scene_tag:test_1,flow_divider:%E6%B5%81%E9%87%8F%20%E5%88%86%E6%A1%B6). Click Search. In the request parsing area, confirm that the query and the abtest parameter were passed correctly. Note: You can also use an online URL encoding tool to generate the encoded values.