Survey Fact Sheets and Critical Reading

An election poll publishes the following methodology sheet: "Target population: electoral roll of the province ($650,000$ people). Sample: $1,200$ interviews. Margin of error: $\pm 2.8\ \%$. Confidence level: $95.5\ \%$".

• a) Explain what each of the four items of data means.

• b) What is the relationship between the confidence level and the certainty that the true result lies within the margin of error?

A survey on reading habits gives $34\ \%$ daily readers, with a margin of error of $\pm 3\ \%$ and a confidence level of $95 \%$.

• a) Between which values does the true percentage of daily readers lie, at that confidence level?

• b) A headline claims: "Exactly $34$ out of every $100$ people read daily". Correct the headline so that it is honest.

The methodology sheet of an opinion poll states: "Target population: $2,400,000$ voters. Sample: $2,400$ random interviews. Margin of error: $\pm 2\ \%$. Confidence level: $95\ \%$".

• a) Interpret each piece of data in the methodology sheet.

• b) Explain qualitatively why the margin of error depends on the sample size and hardly at all on the population size.

• c) The poll gives $21\ \%$ to one party and $22\ \%$ to another. Can it be claimed that the second party is ahead? Justify your answer using the margin of error.

To find out about library usage in a high school with $900$ students, $50$ students from a single grade—those in the study hall during the first period—are surveyed.

• a) Identify which element of the methodology sheet is flawed: the target population, the sample, or the calculation of error?

• b) Explain why the sample is not representative and propose how to select it correctly.

An online newspaper runs the headline: "$60\ \%$ of young people reject the municipality's new app". In the fine print, it reads: survey of $80$ users of a municipal services complaint forum.

• a) Point out all the methodological issues with this headline (sample selection, size, and bias).

• b) Using the margin of error formula $E = 1.96 \times \sqrt{\frac{0.5 \times 0.5}{n}}$, estimate the margin of error that even a random sample of that size would have.

• c) Rewrite the headline honestly.

True or false, justifying your answer: "If a survey was conducted with $1,200$ people and a confidence level of $95\ \%$, its result coincides exactly with the true value for the entire population."

Two surveys on the same population and in the same week give different results for the same event: the first estimates $38\ \%$ with a margin of error of $\pm 2\ \%$ (sample of $2,500$ people) and the second, $34\ \%$ with a margin of error of $\pm 4.5\ \%$ (sample of $450$ people), both at a $95\ \%$ confidence level.

• a) Write down the intervals in which the true value would lie according to each survey and check that they overlap.

• b) Using the formula for margin of error, explain why the second has a much larger margin.

• c) If a headline reads "The surveys contradict each other", is it correct? Justify your answer.

You want to estimate the percentage of students in your city ($25,000$ students) who eat a proper breakfast, with a maximum margin of error of $\pm 3\ \%$ and a confidence level of $95 \%$.

• a) Calculate the sample size required in the worst-case scenario ($p = q = 0.5$), using $n = \frac{1.96^2 \times 0.25}{E^2}$.

• b) Describe the population (universe) and a sampling procedure that guarantees representativeness.

• c) Write out the full methodology sheet (factsheet) of your study as it would appear published.