A/B Testing Concepts and Applications

These flashcards summarize key concepts and vocabulary related to A/B testing and its applications in marketing.

A/B test

A method where users are randomly divided into two groups to compare different stimuli and measure outcomes.

Confidence Intervals

A statistical range that shrinks as more data is collected, indicating the reliability of an estimate.

Randomized Control Trial

An experiment where participants are randomly assigned to different groups to test the effects of various treatments.

Measurable Outcomes

Explicit metrics used to assess the effectiveness of different marketing strategies, such as sales, clicks, or engagement.

Email Marketing

A digital marketing strategy that tests different variables like subject lines, discounts, and timing to measure their impact on user behavior.

Multivariate Testing

A testing method that examines multiple factors simultaneously to determine their individual and combined effects.

Average Treatment Effect (ATE)

The difference in means across two groups, used to quantify the impact of a treatment.

Effect Size (Cohen’s D)

A measure indicating the magnitude of an effect or the strength of a relationship in data.

Sample Size Determination

The process of calculating the number of observations needed to achieve reliable results in an experiment.

Thompson Sampling

A Bayesian approach to A/B testing where users are assigned to treatment groups based on the probability of each treatment being the best.