Hypothesis Testing Explained: Steps, P-Values and Examples

Hypothesis testing is a way of deciding whether data gives enough evidence to reject a default assumption. You state a null hypothesis (usually 'no effect' or 'no difference'), calculate a test statistic from your sample, and find the p-value - the probability of getting results at least this extreme if the null hypothesis were true. If the p-value is at or below your significance level (often 0.05), you reject the null hypothesis; otherwise you fail to reject it.

This guide walks through the five steps, two complete worked examples, what a p-value does and doesn't tell you, and the mistakes that cost students the most marks.

What are the five steps of a hypothesis test?

• State the hypotheses: the null hypothesis H₀ (no effect, no difference) and the alternative H₁ (what you suspect is true).
• Choose the significance level α - the risk of a false positive you are willing to accept, most often 0.05.
• Check the conditions and pick the right test for your data and design - our free statistical test chooser walks you through this.
• Calculate the test statistic and its p-value from your sample.
• Decide and conclude in context: reject H₀ if p ≤ α, otherwise fail to reject it - then say what that means for the real question.

Null and alternative hypotheses

The null hypothesis is the skeptical default: nothing is going on. The alternative is the claim you are looking for evidence of. The alternative can be two-sided (the value is different, in either direction) or one-sided (it is specifically bigger, or specifically smaller) - and that choice has to be made before you look at the data.

What does a p-value actually mean?

The p-value is the probability of getting a result at least as extreme as yours, assuming the null hypothesis is true. A small p-value means your data would be surprising if H₀ were true, and that surprise counts as evidence against it.

What a p-value is not matters just as much. The American Statistical Association's 2016 statement on p-values stresses that a p-value is not the probability that the null hypothesis is true, and that it does not measure the size or importance of an effect. With a big enough sample, a tiny and unimportant difference can be 'significant'; with a small one, a real effect can miss the 0.05 line.

Worked example 1: a one-sample t-test

A teacher wants to know whether her class's mean quiz score differs from the long-run average of 70. A random sample of 16 students has a mean of 74 and a standard deviation of 8, and she tests at the 5% level.

First, the hypotheses: H₀: μ = 70 and H₁: μ ≠ 70 - two-sided, because a difference in either direction matters. The population standard deviation is unknown, so the right test is a one-sample t-test. The test statistic measures how many standard errors the sample mean sits from the claimed value:

With n − 1 = 15 degrees of freedom, the two-sided critical value at the 5% level is 2.131. Our t of 2.0 falls short of it, and its two-sided p-value is about 0.064:

The conclusion, in context: at the 5% significance level, there is not enough evidence that the class's mean quiz score differs from 70. Notice what this does not say. It does not prove the mean is 70 - the sample is small, and a real 4-point difference could easily be hiding in the noise. That is why statisticians say 'fail to reject' the null hypothesis, never 'accept' it.

Worked example 2: a one-proportion z-test

You flip a coin 100 times and get 60 heads. Is it fair? Here H₀: p = 0.5 and H₁: p ≠ 0.5. With 50 expected heads and 50 expected tails, the sample is easily large enough for a z-test for a proportion. Work out the standard error under the null hypothesis, then the z statistic:

For z = 2.0, the two-sided p-value is about 0.046 - just under 0.05. So at the 5% level you reject H₀: there is evidence that the coin is not fair.

Both examples produced a test statistic of 2.0, yet one is significant and the other is not. The t-distribution with only 15 degrees of freedom has heavier tails than the normal distribution, so it takes a larger statistic to reach the same p-value. That is exactly why matching the test to the data matters.

Type I and Type II errors

Every decision from a hypothesis test can be wrong in one of two ways:

Power - the chance of detecting an effect that is really there - grows with larger samples, larger true effects and less variable data. The quiz example above is a good illustration: with only 16 students, a genuine 4-point improvement could easily go undetected.

Common mistakes that cost marks

• Writing 'accept H₀' instead of 'fail to reject H₀'.
• Reading the p-value as the probability that H₀ is true.
• Choosing a one-sided test after seeing which way the data points.
• Skipping the conditions check - random sampling, independence, and sample size or normality where the test needs it.
• Stating the decision without context - always finish with a sentence about the real question.
• Using a two-sample test on paired data (or the reverse).

How do hypothesis tests relate to confidence intervals?

They are two views of the same evidence. For a mean, a two-sided t-test at the 5% level rejects H₀ exactly when the claimed value falls outside the 95% t-interval built from the same data. For proportions the match is close but not always exact, because the test and the interval use slightly different standard errors. Intervals are often more informative, because they show the plausible size of an effect rather than only whether it is zero - which is why many teachers ask for both.

How a tutor helps with hypothesis testing

Hypothesis testing is where statistics stops being calculation and becomes reasoning - and that is where most students get stuck: choosing the test, checking the conditions, and writing a conclusion that earns full credit. A tutor works through your own homework or dataset with you and builds a process you can repeat on any question, whether it is AP Statistics free-response or a college research methods course.

Our online statistics tutoring pairs you with a specialist who explains the why behind every step, not just the formula. Your first trial lesson is free.

The bottom line

A hypothesis test asks how surprising your data would be if nothing were going on. State H₀ and H₁, choose α, pick the test that matches your data, compute the statistic and p-value, and conclude in context - rejecting H₀ when p ≤ α and otherwise failing to reject it. Get the logic right once, and every test in the course follows the same pattern.