Exercises: Survey methodology sheets and critical reading

An election poll publishes the following methodology sheet: "Target population: electoral roll of the province ($650,000$ people). Sample: $1,200$ interviews. Margin of error: $\pm 2.8\ \%$. Confidence level: $95.5\ \%$".

• a) Explain what each of the four items of data means.

• b) What is the relationship between the confidence level and the certainty that the true result lies within the margin of error?

True or false, justifying your answer: "If a survey was conducted with $1,200$ people and a confidence level of $95\ \%$, its result coincides exactly with the true value for the entire population."

A survey on reading habits gives $34\ \%$ daily readers, with a margin of error of $\pm 3\ \%$ and a confidence level of $95 \%$.

• a) Between which values does the true percentage of daily readers lie, at that confidence level?

• b) A headline claims: "Exactly $34$ out of every $100$ people read daily". Correct the headline so that it is honest.

To find out about library usage in a high school with $900$ students, $50$ students from a single grade—those in the study hall during the first period—are surveyed.

• a) Identify which element of the methodology sheet is flawed: the target population, the sample, or the calculation of error?

• b) Explain why the sample is not representative and propose how to select it correctly.

An online newspaper runs the headline: "$60\ \%$ of young people reject the municipality's new app". In the fine print, it reads: survey of $80$ users of a municipal services complaint forum.

• a) Point out all the methodological issues with this headline (sample selection, size, and bias).

• b) Using the margin of error formula $E = 1.96 \times \sqrt{\frac{0.5 \times 0.5}{n}}$, estimate the margin of error that even a random sample of that size would have.

• c) Rewrite the headline honestly.

The methodology sheet of an opinion poll states: "Target population: $2,400,000$ voters. Sample: $2,400$ random interviews. Margin of error: $\pm 2\ \%$. Confidence level: $95\ \%$".

• a) Interpret each piece of data in the methodology sheet.

• b) Explain qualitatively why the margin of error depends on the sample size and hardly at all on the population size.

• c) The poll gives $21\ \%$ to one party and $22\ \%$ to another. Can it be claimed that the second party is ahead? Justify your answer using the margin of error.

Two surveys on the same population and in the same week give different results for the same event: the first estimates $38\ \%$ with a margin of error of $\pm 2\ \%$ (sample of $2,500$ people) and the second, $34\ \%$ with a margin of error of $\pm 4.5\ \%$ (sample of $450$ people), both at a $95\ \%$ confidence level.

• a) Write down the intervals in which the true value would lie according to each survey and check that they overlap.

• b) Using the formula for margin of error, explain why the second has a much larger margin.

• c) If a headline reads "The surveys contradict each other", is it correct? Justify your answer.

You want to estimate the percentage of students in your city ($25,000$ students) who eat a proper breakfast, with a maximum margin of error of $\pm 3\ \%$ and a confidence level of $95 \%$.

• a) Calculate the sample size required in the worst-case scenario ($p = q = 0.5$), using $n = \frac{1.96^2 \times 0.25}{E^2}$.

• b) Describe the population (universe) and a sampling procedure that guarantees representativeness.

• c) Write out the full methodology sheet (factsheet) of your study as it would appear published.