Table of Content

Hypothesis Testing

A/B Testing

A/B Testing - Example

Hypothesis Testing: A Simple Explanation

What is Hypothesis Testing?

Hypothesis testing is a statistical method used to determine whether a claim or assumption about a population is true or not, based on sample data.

Think of it as a courtroom trial:

• The null hypothesis (H₀) is like the assumption that the accused person is innocent.
• The alternative hypothesis (H₁) is the claim that the accused person is guilty.
• The evidence (sample data) is collected to decide whether to reject the null hypothesis or fail to reject it.

Steps in Hypothesis Testing

1. Define Hypotheses
   • Null Hypothesis (H₀): No change or effect.
   • Alternative Hypothesis (H₁): There is a significant change or effect.
2. Collect Sample Data
   • Example: Testing if a new ad campaign increases sales.
3. Set a Significance Level (α)
   • Common choice: 0.05 (5%) → We accept a 5% chance of being wrong.
4. Perform a Statistical Test
   • T-Test: Comparing averages (e.g., revenue before & after).
   • Chi-Square Test: Comparing proportions (e.g., click rates).
5. Interpret the P-Value
   • If p-value < 0.05, reject H₀ → There is a significant effect.
   • If p-value ≥ 0.05, fail to reject H₀ → No significant effect.

Example: Does a New Website Design Increase Sales?