This glossary lists terms in alphabetical order.
Variable
For client-side experiments, variables are pushed to the app from the dynamic configurations of Mobile Delivery Service (MDS). For server-side and gateway experiments, you must define custom variables.
Rollout
After an experiment is complete, you can select a plan based on the results and apply it to all traffic. This process is called a rollout.
Experiment
An experiment is a core concept in A/B testing. It involves comparing at least two plans. Based on the metric results, you decide which plan to roll out.
Experiment plan
A plan can represent an algorithm, an interface style, or a business logic. Experiment plans use variables to control and create these differences. Different experiment versions are applied to different user groups, and the experiment is evaluated by comparing the results from these homogeneous groups.
Lab
A lab is used to manage a group of experiments. Labs are categorized as either client-side or server-side.
Debug
A test is the process of verifying that an experiment runs correctly after it is created but before it officially starts.
Metrics
Metrics are used to evaluate experiment results. The system provides three default metrics: Page View (PV), Unique Visitor (UV), and 7-day retention. You can also use custom events in Mobile Analysis Service (MAS) to create MAS metrics or compound indicators.
Custom event
In Mobile Analysis Service (MAS), an event records a user action within an app. You can create a custom event by embedding code that triggers on a specific action, such as a button click. Custom events can be used as metrics in A/B experiments.
Confidence level
A confidence interval is a range of plausible values for a parameter. The confidence level is the probability that the true value of the parameter is within this interval.
Confidence interval
A confidence interval is an estimated range for a population parameter that is constructed from sample statistics. It indicates the probable range in which the true value of a parameter lies and provides a measure of the reliability of the estimate.