Back to homeMathematics · AP Statistics · Change subject

Start for free

AP Stats · Mathematics

Statistics · Inference

Hypothesis Testing

Before you start, make sure you know: The normal distribution (Algebra 2)

Before you start, make sure you know: Normal dist. with mean 0 and std dev 1 N(0,1) (Algebra 2)

Tony's Pizza in Chicago promises that the mean delivery time is $30$ minutes. Jayden is tired of cold pizza, so he tracks $64$ random deliveries and finds a mean of $32.5$ minutes. From years of company data, delivery times are known to have a standard deviation of $8$ minutes. Is the $2.5$-minute gap just bad luck, or is the promise really false? A hypothesis test (also called a significance test) is the formal way to decide whether a sample contradicts a claim, or whether it is the kind of difference random chance produces all the time.

Stating the Hypotheses

Every test starts with two competing statements about a population parameter ($\mu$ or $p$).

The words in the question tell you which one to use. "Different from," "changed" → $\neq$. "Lower," "decreased," "less than promised" → $<$. "Higher," "slower than advertised," "more than" → $>$.

Practice★☆☆

A streaming platform claims that the mean time its users spend watching series is $120$ minutes a day. A group of students suspects that, due to exams, this time has…

A streaming platform claims that the mean time its users spend watching series is $120$ minutes a day. A group of students suspects that, due to exams, this time has decreased. To carry out a hypothesis test on this claim:

a) Clearly define the null hypothesis ($H_0$) and the alternative hypothesis ($H_1$).

b) State whether it is a one-tailed or two-tailed test.

Type I and Type II Errors

A test uses a sample, so it can be wrong even when we do everything correctly.

There is a trade-off: making $\alpha$ smaller (being harder to convince) makes $\beta$ larger. The only way to shrink both is a larger sample.

Practice★☆☆

Explain in your own words what it means to commit a Type I Error in the context of a study attempting to prove whether a new study method improves math scores (where…

Explain in your own words what it means to commit a Type I Error in the context of a study attempting to prove whether a new study method improves math scores (where $H_0$ is that the method produces no improvements).

Practice★★☆

A new model of running shoes designed to increase performance is being tested. The manufacturer sets the null hypothesis $H_0$: "The mean performance with the new model…

A new model of running shoes designed to increase performance is being tested. The manufacturer sets the null hypothesis $H_0$: "The mean performance with the new model is equal to that of the previous model" against the alternative $H_1$: "The mean performance is higher".

Explain with reasoning what it would mean to commit a Type I error and a Type II error in this practical context. Which of the two errors do you think would be more damaging to the company's brand image if discovered later?

The z-Test for a Mean

Suppose $H_0: \mu = \mu_0$ is true. Then the sample mean $\bar{x}$ should land near $\mu_0$. The test asks: how many standard errors is our $\bar{x}$ from $\mu_0$, and how likely is a result at least that far away?

z-test for a mean (σ known)

Conditions: a random sample; independent observations (if sampling without replacement, $n$ is at most $10\%$ of the population); and a normal population or a large sample ($n \ge 30$) so that $\bar{x}$ is approximately normal.

Test statistic:

$$z = \dfrac{\bar{x} - \mu_0}{\sigma/\sqrt{n}}$$

p-value: the probability, assuming $H_0$ is true, of getting a sample result at least as extreme as the one observed. For $H_a: \mu < \mu_0$ it is $P(Z \le z)$; for $H_a: \mu > \mu_0$ it is $P(Z \ge z)$; for $H_a: \mu \neq \mu_0$ it is $2P(Z \ge |z|)$.

Decision: if p-value $< \alpha$, reject $H_0$; otherwise fail to reject.

Why this works: if $H_0$ is true, $\bar{x}$ has mean $\mu_0$ and standard deviation $\sigma/\sqrt{n}$, so standardizing gives a $N(0,1)$ variable. That lets us use normal-curve areas to measure how surprising our sample is.

The same decision can be made with a critical value $z^*$: reject $H_0$ when $z$ falls in the rejection region.

$\alpha$	one-tailed $z^*$	two-tailed $\pm z^*$
$0.10$	$1.282$	$\pm 1.645$
$0.05$	$1.645$	$\pm 1.960$
$0.01$	$2.326$	$\pm 2.576$

(Some tables round $2.576$ to $2.575$.) On a TI-84, the p-value for a left tail is normalcdf(-1E99, z), and invNorm(1-α) gives a critical value. The sample mean in each problem is a single value from the sampling distribution, so the picture below is always the curve for $z$.

Do high school students sleep less? (four-step process)

Nationally, the mean sleep on school nights is $\mu_0 = 7.0$ hours with $\sigma = 1.2$ hours. A counselor thinks students at her school sleep less. She samples $n = 64$ students and finds $\bar{x} = 6.7$ hours. Test at $\alpha = 0.05$.

State. Let $\mu$ be the true mean sleep time at her school. $H_0: \mu = 7.0$, $H_a: \mu < 7.0$, $\alpha = 0.05$.

Plan. One-sample $z$-test for a mean. Random sample (given), $n = 64 \ge 30$ so $\bar{x}$ is approximately normal, and $64$ is under $10\%$ of the school's students.

Do. The standard error is $\dfrac{1.2}{\sqrt{64}} = 0.15$, so

$$z = \dfrac{6.7 - 7.0}{0.15} = -2.0$$

The p-value is $P(Z \le -2.0) \approx 0.0228$.

Conclude. Since $0.0228 < 0.05$, we reject $H_0$. There is convincing evidence that the true mean sleep time at her school is less than $7.0$ hours.

Your turn: what is the conclusion at $\alpha = 0.01$?

Now $0.0228 > 0.01$, so we fail to reject $H_0$. There is not convincing evidence at the $1\%$ level. Same data, different standard of proof.

Practice★☆☆

We are carrying out a two-tailed hypothesis test for the mean of a normal population with known $\sigma$. After calculating the test statistic with the data from a…

We are carrying out a two-tailed hypothesis test for the mean of a normal population with known $\sigma$. After calculating the test statistic with the data from a sample, we obtain a value of $z_{exp} = 2,15$.

If the critical values for a significance level of $\alpha = 0,05$ are $-1,96$ and $1,96$, reason whether we should accept or reject the null hypothesis.

Practice★★☆

An online shopping courier company claims that the mean delivery time for its parcels in large cities is $48$ hours, with a known standard deviation of $\sigma = 6$…

An online shopping courier company claims that the mean delivery time for its parcels in large cities is $48$ hours, with a known standard deviation of $\sigma = 6$ hours. A group of users suspects that the service is slower than advertised. To check this, they record the delivery times of $36$ randomly chosen orders, obtaining a mean of $50,5$ hours.

State the null and alternative hypotheses for this case and carry out a one-tailed (right-tailed) hypothesis test with a confidence level of $99\ \%$. Is there sufficient evidence to claim that the company takes longer than it claims?

Practice★★★

A study on youth mental health indicates that the mean daily time that high school seniors spend on TikTok is $120$ minutes, with a standard deviation of $30$ minutes. A…

A study on youth mental health indicates that the mean daily time that high school seniors spend on TikTok is $120$ minutes, with a standard deviation of $30$ minutes. A school counselor believes that in their school, due to the pressure of SAT exams, the mean time is significantly lower. They take a random sample of $50$ students and obtain a mean of $112$ minutes.

a) Define the appropriate hypothesis test for the counselor's belief.
b) Calculate the p-value of the sample obtained.
c) If a significance level of $1\%$ is used, what conclusion should the counselor draw? What if the significance level were $5\%$?

Rejection and Acceptance Regions

Sometimes it is handier to decide before collecting data which values of the sample mean will make you reject. Solve $z = \pm z^*$ for $\bar{x}$.

Cereal box acceptance region

A cereal box is labeled $12$ oz. Machine records give $\sigma = 0.4$ oz. An inspector weighs $n = 64$ boxes and wants a two-tailed test of $H_0: \mu = 12$ at $\alpha = 0.05$.

Standard error: $SE = \dfrac{0.4}{\sqrt{64}} = 0.05$ oz.

For $\alpha = 0.05$ two-tailed, $z_{\alpha/2} = 1.96$, so the margin is $1.96 \times 0.05 = 0.098$. The acceptance region is $(11.902,\ 12.098)$.

She finds $\bar{x} = 11.93$ oz. That is inside the region, so she fails to reject $H_0$: the weights are consistent with $12$ oz. Now try $\alpha = 0.01$ yourself.

With $z_{\alpha/2} = 2.576$ the margin is $2.576 \times 0.05 \approx 0.129$, so the region is about $(11.871,\ 12.129)$. A smaller $\alpha$ gives a wider acceptance region: you demand stronger evidence before rejecting.

Practice★☆☆

In the quality control of a potato chip factory, we want to check if the mean weight of the bags is really $150$ ounces, as indicated on the packaging. It is known that…

In the quality control of a potato chip factory, we want to check if the mean weight of the bags is really $150$ ounces, as indicated on the packaging. It is known that the population standard deviation is $\sigma = 5$ ounces. If it is suspected that the bags may weigh both more and less than indicated:

a) Set up the hypotheses $H_0$ and $H_1$.

b) If we set a significance level of $\alpha = 0,05$, calculate the critical values $z_{\alpha/2}$ that delimit the acceptance zone.

Practice★☆☆

Given a normal population with standard deviation $\sigma = 12$, we want to test $H_0: \mu = 50$ against $H_1: \mu > 50$ with a significance level of $\alpha = 0,05$.…

Given a normal population with standard deviation $\sigma = 12$, we want to test $H_0: \mu = 50$ against $H_1: \mu > 50$ with a significance level of $\alpha = 0,05$.

Calculate the critical value of the sample mean $\bar{x}_c$ for a sample of size $n = 36$ using the formula $\bar{x}_c = \mu_0 + z_{\alpha} \cdot \frac{\sigma}{\sqrt{n}}$, knowing that for $\alpha = 0,05$ the value of $z_{\alpha} = 1,645$.

Practice★★☆

A streaming platform suspects that the mean daily time its users spend watching content has changed compared to last year, which was $120$ minutes. They assume a normal…

A streaming platform suspects that the mean daily time its users spend watching content has changed compared to last year, which was $120$ minutes. They assume a normal distribution with $\sigma = 30$ minutes. They take a random sample of $100$ users and observe a mean of $126$ minutes.

Determine the acceptance region and the critical region for a two-tailed test with a significance level of $10\ \%$. What decision should the platform make regarding its suspicion?

The z-Test for a Proportion

The acceptance region for $\hat{p}$ is $p_0 \pm z^*\sqrt{p_0(1-p_0)/n}$, just like the mean.

Do more than 40% of seniors have a part-time job?

A report claims that $40\%$ of high school seniors have a part-time job. Marcus suspects the figure is different at his school. He surveys $200$ random seniors and $92$ have a job. Test at $\alpha = 0.05$.

$H_0: p = 0.40$, $H_a: p \neq 0.40$. Check Large Counts: $np_0 = 80 \ge 10$ and $n(1 - p_0) = 120 \ge 10$. Good.

$\hat{p} = \dfrac{92}{200} = 0.46$ and $SE = \sqrt{\dfrac{0.40 \times 0.60}{200}} \approx 0.0346$, so

$$z = \dfrac{0.46 - 0.40}{0.0346} \approx 1.73$$

Two-tailed p-value: $2 \times P(Z \ge 1.73) \approx 2(0.0418) = 0.083$. Since $0.083 > 0.05$, we fail to reject $H_0$. There is not convincing evidence that the proportion differs from $40\%$.

The same result shows up with the region: $0.40 \pm 1.96(0.0346) = (0.332,\ 0.468)$, and $\hat{p} = 0.46$ is inside it, as the figure shows.

Practice★☆☆

A sports shoe brand claims that $60\%$ of marathon runners use its products. A sports journalist suspects that the proportion is lower. To carry out the test, $100$…

A sports shoe brand claims that $60\%$ of marathon runners use its products. A sports journalist suspects that the proportion is lower. To carry out the test, $100$ runners are surveyed and it is found that $52$ of them use this brand.

a) Calculate the test statistic (sample proportion) $\hat{p}$.

b) Write the expression for the standardised test statistic $Z = \frac{\hat{p} - p_0}{\sqrt{\frac{p_0 \cdot q_0}{n}}}$ for this case.

Practice★☆☆

A sociologist claims that $35\%$ of American teenagers prefer physical format to digital for reading books. To test this claim against the suspicion that the proportion…

A sociologist claims that $35\%$ of American teenagers prefer physical format to digital for reading books. To test this claim against the suspicion that the proportion is different, a random sample is chosen. Determine the acceptance region for the sample proportion if a confidence level of $95\%$ is used and the sample size is $n = 400$ people.

Practice★★★

A manufacturer of electronic components for smartphones guarantees that the percentage of defective parts in its production is, at most, $2\%$. An external quality…

A manufacturer of electronic components for smartphones guarantees that the percentage of defective parts in its production is, at most, $2\%$. An external quality control decides to sample $2500$ components to verify this guarantee. After the analysis, $62$ defective parts are found.

a) Formulate the hypotheses for a right-tailed test.
b) Calculate the value of the test statistic.
c) Determine the critical significance level (p-value). In light of the result, do you think the manufacturer should review its production process? Justify your answer.

Tests and Confidence Intervals

A confidence interval and a two-tailed test are two views of the same information.

Using an interval to decide

A $95\%$ confidence interval for the mean commute time (minutes) of students at a school is $(24.3,\ 29.7)$. Decide each test at $\alpha = 0.05$, two-tailed.

$H_0: \mu = 30$. The value $30$ is outside the interval, so we reject $H_0$.

$H_0: \mu = 28$. The value $28$ is inside the interval, so we fail to reject $H_0$. The same interval answers every such question without recomputing $z$.

What about $\alpha = 0.01$ for $H_0: \mu = 30$? We need the $99\%$ interval. The center is $\bar{x} = \dfrac{24.3 + 29.7}{2} = 27$ and the margin of error is $\dfrac{29.7 - 24.3}{2} = 2.7$.

Scale the margin by the ratio of critical values: $2.7 \times \dfrac{2.576}{1.960} \approx 3.55$. The $99\%$ interval is $27 \pm 3.55 = (23.45,\ 30.55)$. Now $30$ is inside, so at $\alpha = 0.01$ we fail to reject $H_0: \mu = 30$. No new standardization was needed.

Practice★★☆

A study has been carried out on the monthly leisure expenditure of college students in a city, assuming it follows a normal distribution with standard deviation…

A study has been carried out on the monthly leisure expenditure of college students in a city, assuming it follows a normal distribution with standard deviation $\$\sigma = 20$. A $95\ \%$ confidence interval of $(75,1; 84,9)$ has been obtained for the mean.

Without performing new standardization calculations, use the relationship between confidence intervals and hypothesis testing to answer: Would the null hypothesis $\$H₀: \mu = 70$ be rejected against $\$H_1: \mu \neq 70$ with a significance level of $\alpha = 0,05$? What if the null hypothesis were $\$H₀: \mu = 80$? Justify your answer.

Practice★★★

We want to investigate whether a new study method improves performance in math. The historical mean score in a region is $6,4$ with a standard deviation of…

We want to investigate whether a new study method improves performance in math. The historical mean score in a region is $6,4$ with a standard deviation of $\sigma = 1,2$. The new method is applied to an experimental group of $36$ students and a mean score of $6,9$ is obtained.

a) Carry out a one-tailed hypothesis test for a significance level of $\alpha = 0,05$.
b) If the $95\%$ confidence interval for the mean of this group were $(6,508 ; 7,292)$, could you have answered the previous section without calculating the test statistic? Explain the relationship between confidence intervals and hypothesis testing.

Error Probabilities, Power, and Sample Size

Now we can put numbers on $\alpha$ and $\beta$.

The interactive below shows the sampling distribution under $H_0$ (centered at $0$) and under a true mean shifted by the slider. The more the curves overlap, the larger $\beta$.

Probabilities of error for a light bulb test

A company says its bulbs last $\mu_0 = 1000$ hours on average, with $\sigma = 100$ hours. A tester checks $n = 25$ bulbs and rejects $H_0: \mu = 1000$ (in favor of $H_a: \mu < 1000$) whenever $\bar{x} < 960$.

$SE = \dfrac{100}{\sqrt{25}} = 20$ hours.

Type I error probability. Assume $H_0$ is true ($\mu = 1000$): $z = \dfrac{960 - 1000}{20} = -2$, so $\alpha = P(Z < -2) \approx 0.0228$.

Type II error probability. Suppose the true mean is really $\mu = 940$. We fail to reject when $\bar{x} \ge 960$: $z = \dfrac{960 - 940}{20} = 1$, so $\beta = P(Z \ge 1) \approx 0.1587$.

Power $= 1 - 0.1587 = 0.8413$: if the bulbs really average $940$ hours, this test catches it about $84\%$ of the time.

Your turn: if the tester used $n = 100$ bulbs and kept $\alpha \approx 0.0228$, would the power at $\mu = 940$ go up or down?

It goes up. Now $SE = 10$, the cutoff is $1000 - 2(10) = 980$, and $\beta = P\!\left(Z \ge \dfrac{980 - 940}{10}\right) = P(Z \ge 4)$, which is essentially $0$. Power is nearly $1$.

Practice★★★

A brand of wireless headphones claims that the battery of its new "Pro-Z" model lasts, on average, $24$ hours. It is known that the battery lifespan follows a normal…

A brand of wireless headphones claims that the battery of its new "Pro-Z" model lasts, on average, $24$ hours. It is known that the battery lifespan follows a normal distribution with a standard deviation of $\sigma = 2,5$ hours. A group of technical testers analyzes $100$ headphones and finds a mean lifespan of $23,4$ hours.

a) Carry out a one-tailed hypothesis test to check if the battery lasts less than promised with a significance level of $\alpha = 0,05$.
b) Explain, in the context of the problem, what committing a Type II error would consist of.
c) What should the minimum sample size be so that, with a mean of $23,4$, the result is significant at $1\%$?

Practice★★★

In a debate on sustainability, it is claimed that $75\%$ of young people in a city separate plastic packaging from the rest of the trash. To verify this, a sample of…

In a debate on sustainability, it is claimed that $75\%$ of young people in a city separate plastic packaging from the rest of the trash. To verify this, a sample of $150$ young people is selected and it is observed that $105$ of them recycle correctly.

a) Set up the hypothesis test to check if the true proportion is lower than claimed.
b) Is the initial claim acceptable for a significance level of $10\%$?
c) Suppose the true proportion were $65\%$. What decision would most likely be made with this test if the sample were only $20$ people? Reason about the importance of sample size on the power of the test.

Practice★★★

A trade school claims that the mean starting salary of its graduates is $\$1550…

A trade school claims that the mean starting salary of its graduates is $\$1550$ per month. It is known that salaries follow a normal distribution with standard deviation $\$\sigma = 210$. A trade union selects a sample of $49$ graduates and obtains a mean salary of $\$1480$.

a) Test the school's hypothesis against the union's suspicion that the salary is lower, using $\alpha = 0,01$.
b) Calculate the probability of committing a Type I error if we decide to reject the null hypothesis for any sample mean lower than $\$1500$.
c) How would an increase in the population standard deviation affect the acceptance region?

Back to the start: Tony's Pizza claims a mean of $30$ minutes ($H_0: \mu = 30$ against $H_a: \mu > 30$). With the known $\sigma = 8$ minutes, $n = 64$, and $\bar{x} = 32.5$, we get $z = \dfrac{32.5 - 30}{8/\sqrt{64}} = 2.5$ and a p-value of $P(Z \ge 2.5) \approx 0.0062$. Since $0.0062 < 0.05$ (and even $< 0.01$), Jayden has convincing evidence that the true mean delivery time is more than $30$ minutes.

Summary

Summary

$H_0$ is the starting claim (with equality) and $H_a$ is the suspicion: $\neq$ gives a two-tailed test, $<$ or $>$ a one-tailed test. Hypotheses are about parameters, never statistics.
Type I error: rejecting a true $H_0$ (probability $\alpha$). Type II error: failing to reject a false $H_0$ (probability $\beta$). Power is $1 - \beta$ and grows with $n$.
Mean ($\sigma$ known): $z = \dfrac{\bar{x} - \mu_0}{\sigma/\sqrt{n}}$. Proportion: $z = \dfrac{\hat{p} - p_0}{\sqrt{p_0(1-p_0)/n}}$ (use $p_0$ in the standard error).
Reject $H_0$ if the p-value is below $\alpha$, or if $z$ is in the critical region. Critical values: $1.645$ (one-tailed) and $\pm 1.96$ (two-tailed) for $\alpha = 0.05$. Never say "accept $H_0$."
A two-tailed test at level $\alpha$ is equivalent to checking whether $\mu_0$ lies inside the $(1-\alpha)$ confidence interval $\bar{x} \pm z^*\sigma/\sqrt{n}$. To change the confidence level, multiply the margin of error by the ratio of the $z^*$ values. Always conclude in context, linking the p-value to $\alpha$.

Let's practice! 8 easy exercises in this section

★☆☆ A streaming platform claims that the mean time its users spend watching series is $120$…★☆☆ In the quality control of a potato chip factory, we want to check if the mean weight of…

Let's practice! 8 medium exercises in this section

★★☆ A well-known technology influencer* claims in one of their videos that $40\ \%$ of…★★☆ An online shopping courier company claims that the mean delivery time for its parcels in…★★☆ A new model of running shoes designed to increase performance is being tested. The…★★☆ In a study on sustainability, it is known that two years ago $30\ \%$ of young people in…

Let's practice! 8 hard exercises in this section

★★★ A well-known eSports organization claims that $40\%$ of teenagers who play "League of…★★★ A study on youth mental health indicates that the mean daily time that high school…★★★ A brand of wireless headphones claims that the battery of its new "Pro-Z" model lasts, on…

See all the exercises in this section