p-values and expected value

Expected value of a random variable

A random variable $X$ is a number that depends on the outcome of an experiment. For a die, $X$ is the score. For a raffle ticket, $X$ is the prize it wins.

If you repeat the experiment many times, the average of the results settles down to a fixed number. That number is the expected value (or mean) of $X$.

Why does this work? It is a weighted average. Each value counts in proportion to how often it turns up.

Roll the die many times and watch the average of the scores. It heads towards 3.5, even though you can never roll a 3.5.

Fair games and decisions

When you pay to play, what matters is the expected profit:

Why? Over many plays the player's average profit per game approaches the expected profit. A positive value means the player wins in the long run, and a negative value means the organiser does.

When the winnings depend on how long a game lasts, build a table of outcomes and probabilities first, then use $E(X) = \sum xP(x)$. Always check that the probabilities add to 1.

Variance and standard deviation of $X$

Two games can have the same expected value but very different risk. A game that always pays €1 is very different from one that pays €100 with probability 0.01 and €0 otherwise. We measure the spread with the variance and standard deviation.

The variance is the average squared distance from the mean, $E\big[(X-\mu)^2\big]$, and the formula above is a quicker way to calculate it.

Many independent plays

If you play the same game $n$ times, with the plays independent, then for the total $T$:

Adding or subtracting a fixed amount (such as the cost of a ticket) changes the mean but not the spread.

p-values and hypothesis tests

Suppose a coin is tossed 12 times and gives 10 heads. Is the coin fair? Even a fair coin can give 10 heads occasionally. The question is how unlikely such a result is.

A hypothesis test works like a court case. We start by assuming the claim is true, then ask whether the data are too surprising to accept it.

"At least as extreme" points in the direction of $H_1$. If $H_1: p > 0.5$, a high number of heads is extreme, so we use $P(X \ge \text{observed})$. If $H₁: p < p_0$, we use $P(X \le \text{observed})$. For a two-tailed test, we double the one-tailed probability.

A Type I error means rejecting $H_0$ when it is actually true. Its probability is the significance level. Using 1% instead of 5% makes this error less likely, but makes it harder to detect a real difference.

Back to the start: if the club's coin showed 12 heads in a row, then $P(X \ge 12) = \frac{1}{4096}$, far below 5% and 1%, so you would be right to doubt the claim.

Large samples: the normal approximation

For a large sample, adding up many binomial terms is slow. If $X \sim B(n, p)$ with $n$ large, we can use a normal distribution with

Because $X$ takes whole numbers but the normal distribution is continuous, we use a continuity correction: "$X \ge 60$" becomes "$Y > 59.5$", because the whole number 60 occupies the interval 59.5 to 60.5.

Summary

• $E(X) = \sum xP(x)$, a weighted average. $E(X^2) = \sum x^2P(x)$. Probabilities always add to 1.

• $\text{Var}(X) = E(X^2) - \mu^2$ and $\sigma = \sqrt{\text{Var}(X)}$. For $n$ independent plays, the mean is $n\mu$ and the standard deviation is $\sqrt{n}\,\sigma$.

• A game is fair if the expected profit is 0, that is, the cost equals $E(\text{winnings})$.

• A p-value is the probability, assuming $H_0$ is true, of a result at least as extreme as the one observed. If p-value < significance level, reject $H_0$.

• For a large sample, use $N(np, np(1-p))$ with a continuity correction.

• A Type I error means rejecting $H_0$ when it is true.