First movement · Difference under the microscope

Chapter 1 — The Average Gulp

Mean and standard deviation

1. What the gull saw

The gull stood on the elephant’s back.

The elephant stood by the lake and drank.

“I’ve seen people,” said the gull after a while. “The other day, from a post beside a path.”

The elephant thought about it, and drank again.

“And?”

“They look. And then they start counting. How often someone laughs in a conversation. How many steps it is to the tree. How often someone says ‘yes’ before they finally dare to say ‘no’.”

The elephant said nothing.

“With you, they’d count the gulps, I think. How much you drink in a day. In how many goes.”

“Well. Who for?”

“For themselves, I believe.”

The elephant looked at the water.

“And then?” he asked.

“Then they turn it into one number. One gulp. An average gulp.”

The elephant drank, slowly this time.

“And what does it taste of?”

The gull thought about it.

“Nothing.”

The elephant nodded. “Taste is all I know. This gulp was muddy — I went a little too deep. The one before was cold, and clear. Before that, sweet: a streak of rain that had swum in. Sometimes brackish, as if there were salt somewhere far off. No gulp tastes like another. An average gulp would be a gulp that tasted of nothing at all. And I have never had one.”

“But what if they wanted to sketch it anyway?”

“How much water, yes. How long. But then they know something about all my gulps together, and nothing about any one of my gulps on its own. And if that is all they know, at least they know they do not know my gulps.”

The gull was quiet for a moment. “Then they would also have to know how far the real gulps are from that one average gulp.”

“Then they know something,” said the elephant.

What the gull describes happens of its own accord. If you keep seeing differences — one more than the other, one more often than the other — the question comes up on its own:

Where is the middle?

That question is where counting turns into summarising. Where separate observations become one number.

One thing to get clear, straight away.

There is no such thing as an average gulp.

Not because it is difficult. Because there is nothing to taste.

And yet — with things we can measure — we do it all the time. We take differences, and turn them into one number. An average.

From here on, we need language.

2. The diamonds

One morning in September a beaver found something in his dam.

He had been working on that dam for weeks, with the fine toothwork beavers are known for, and not for the first time something unexpected had got caught in it — a branch, a fish, a shoe. But what he found now was different. A small wooden chest, not much bigger than his own head, soaked through and heavy.

He dragged it to the bank and called for help, because a beaver opens wooden things in only one way, and that leaves very little chest. The elephant came wandering over; he had nothing better to do that day. The gull landed on a branch to watch. A crow came too, because crows do not leave a treasure chest to other animals.

They opened the chest together. Inside, on a layer of faded velvet, lay twelve little diamonds.

“People stuff,” muttered the beaver.

“Lost,” said the elephant. “Long ago.”

They laid them all out on the moss. Some big and gleaming, some small and dull, some big and dull, one small and very, very bright.

Twelve diamonds. All different. No two alike — just what the gull described.

Where you can measure them, you can average them. Where you taste them, you can’t.

So: out of a beaver’s dam. We got to measure two things about them: their carat (the unit of weight for gemstones) and their shine (how brightly they give back the light, measured with a shine meter — a scale from 0 to 100). The diamonds have names — Anna, Bram, Cees, Dora, Else, Faas, Geer, Hans, Ida, Joop, Kees and Leen — which you’ll remember without trying, or you won’t, and either is fine. Table 1.1 shows all twelve of them, with their two measurements.

Carat (abbreviated ct) is the unit of weight for gemstones. 1 carat = 0.2 grams = 200 milligrams. Our largest diamonds (Kees, Leen) are 2.1 carats — 420 milligrams, about twenty grains of rice. All twelve together weigh 19.2 carats: almost four grams — not quite a sugar cube. Diamonds weigh little and count for much.

**Table 1.1.** *The twelve diamonds with their carat (the unit of weight for gemstones, 1 ct = 0.2 g) and shine (shine meter, scale 0–100).*
id	name	carat	shine (0–100)
1	Anna	1.0	20
2	Bram	1.1	30
3	Cees	1.1	30
4	Dora	1.3	30
5	Else	1.6	50
6	Faas	1.6	70
7	Geer	1.7	80
8	Hans	1.8	60
9	Ida	1.9	70
10	Joop	1.9	80
11	Kees	2.1	30
12	Leen	2.1	50

score value, observation also: measurement

A quick pointer for anyone who hasn’t worked with data much yet. Each row in the table — rows run horizontally, from left to right — stands for one case (statisticians also say observation), and for us that is one diamond. Each column — vertically, from top to bottom — stands for one property we measured in those cases: a variable. Whatever sits in one of those cells — Anna’s 1.0 in the carat column, for instance — is called a score (you’ll run into that word all the time in research articles, even for things you simply measure and nobody “scores”). The id column gives each diamond a row number; the name column is just to be sociable — to whoever counts them, diamonds are not anonymous.

And now that you have that table in front of you, I can hand you, in one go, two small words that researchers fumble all their working lives. You can see them right there, in the grid of the table.

Columns are variables. Rows are diamonds. So:

In points to a column. Between points to rows.
There is a difference in shine between Anna and Kees.

That’s all there is to it. A difference in carat, a difference in shine — that is about what you measured. A difference between Anna and Kees, between the heavy ones and the light ones — that is about who you are comparing.

As soon as you mix the two up, things go wrong, and that happens more often than you’d think. “The difference between shine” — between shine and what? “The difference in Anna and Kees” — in their carat, their shine, their mood? Then the reader no longer knows what the sentence is about, and usually the writer doesn’t either.

It sounds like nitpicking until you have a table like this in front of you. Then you see that there are two directions — down a column, along a row — and that each direction needs a preposition of its own. Figure 1.1 marks those two directions on a piece of the table.

: Figure 1.1. A piece of table 1.1. The upright band over the shine column is the difference in shine; the two bands lying across the rows of Anna and Kees are the difference between Anna and Kees. Where they cross sit the two numbers that differ.

Now for some sums.

3. From the middle to the spread

The mean

mean average also: expected value, M

What does an average diamond taste of? Nothing, I hope. But how big is an average diamond, and how shiny — that we can say something about.

The mean is what you get when you add up all the values and divide by how many there were. As a formula:

\[ \overline{X} = \frac{\sum_{i=1}^{n} X_i}{n} \tag{1.1a} \]

How to read this formula, piece by piece:

\(\overline{X}\) — the mean of \(X\). The bar over the letter is your signal: this is a summary, not a single value.
\(\sum_{i=1}^{n}\) — the summation sign. It says: add up everything that comes after it, for \(i\) from 1 up to and including \(n\). Officially it is the Greek capital letter sigma. We never call it that, because further on that name is already taken by a completely different symbol — and summation sign at least tells you what to do: sum, add up.
\(X_i\) — the value (score) of \(X\) for the \(i\)-th case. \(X_1\) is the first (Anna’s carat), \(X_2\) the second (Bram’s), …, \(X_n\) the last (Leen’s).
\(n\) — the number of cases. Here: 12.

Read it as: add up all the values, and divide by how many there were.

We can also write the same thing slightly differently, and we do that for a reason:

\[ \overline{X} = \frac{1}{n} \sum_{i=1}^{n} X_i \tag{1.1b} \]

Same calculation, different notation. Instead of dividing by \(n\) we write it here as multiplying by \(\frac{1}{n}\). That may look like a quirk of notation, but it isn’t. Statisticians like to think in terms of multiplying, and when we get to regression, transformations and expected values later on, form 1.1b becomes the handier one to work with. So remember that 1.1a and 1.1b give exactly the same number — just written down differently.

Two columns, two letters: from now on we call carat \(X\), and shine \(Y\). For our twelve diamonds:

\[ \begin{aligned} \overline{X} &= \frac{1.0 + 1.1 + 1.1 + 1.3 + 1.6 + 1.6 + 1.7 + 1.8 + 1.9 + 1.9 + 2.1 + 2.1}{12} \\[0.4em] &= \frac{19.2}{12} \\[0.4em] &= 1.6 \end{aligned} \]

\[ \begin{aligned} \overline{Y} &= \frac{20 + 30 + 30 + 30 + 50 + 70 + 80 + 60 + 70 + 80 + 30 + 50}{12} \\[0.4em] &= \frac{600}{12} \\[0.4em] &= 50 \end{aligned} \]

In symbols we write the mean of a variable \(X\) as \(\overline{X}\) (“X-bar”). In APA reporting (the house style of the American Psychological Association, which you’ll come across in articles), people more often use a capital M — M for mean. Either is fine, and you need to be able to read both. We’ll switch between them now and then.

So what is that mean, really? It’s the “average diamond” the gull would draw. One of 1.6 carats, with a shine of 50. And now I have to confess something that most books would keep quiet about at this point: that diamond exists. Have a look in the table, row five: Else. Exactly 1.6 carats, exactly 50 shine — the average diamond to the last decimal, in the flesh. And let me be honest about it straight away: that is luck. In most groups there is nobody standing where the two means cross — ours could just as well have been empty. But as it happens, somebody lives there, and we are going to make the most of our luck. For now, simply this: if you have to say how brightly a random diamond from this group shines, and you have no other information, the mean is your best guess. That’s why it is also called the expected value. And it is what the simplest model we can make of this reality predicts — twelve diamonds and their scores on carat and shine, and nothing else. Statisticians call that the null model.

A word about symbols

Before we calculate any further, a short pointer on how to read formulas. Statisticians are rather particular about this, and they have a reason.

Capital letters (\(X\), \(Y\)) are for the variable itself — the whole list of carat values, or the whole column of shine values. The concept, not one case.
Capitals with a subscript (\(X_i\), \(Y_i\)) are for one specific value. For example, \(X_5 = 1.6\) is Else’s carat (she is the fifth in the table), and \(Y_{11} = 30\) is Kees’s shine. (Other books use lower-case letters for this, \(x_i\) — same idea, different house style.)
Capitals with a bar (\(\overline{X}\), \(\overline{Y}\)) are the mean of a sample. Say “X-bar” and “Y-bar”. In APA style, in running text, you’ll also see the M from a moment ago — both stand for the same thing.
Greek letters (μ, σ) we keep for the population — the whole world of which we have only measured a sample. Mu instead of M, sigma instead of SD. This is the lower-case σ, and this is what the name sigma was being saved for — not for the summation sign from earlier. We’ll come back to the population in a later chapter.

One aside, because the distinction matters. A number that summarises a sample is called a statistic — \(\overline{X}\), \(S\), \(SS\), everything we calculate in this chapter. The same idea for the whole population is called a parameter — μ, σ. A pair of Levi’s 501s has parameters too: waist 28, leg 32. Those numbers describe the real jeans. Statistics are estimates of parameters, made from a sample. We’ll come back to this at length in a later chapter — for now it’s enough to know the names.

As a rule of thumb: capital = the variable, a subscript added = one observation, a bar on top = the mean (a statistic), Greek = the population (a parameter).

Do other statisticians do this too? Yes. Andrew Hayes (whose book on mediation and moderation many psychology students meet in their second year, or later) and Tabachnick & Fidell (the bible of multivariate statistics) keep to the same convention. Sometimes you’ll see a capital \(B\) for an unstandardised regression coefficient and a Greek β (“beta”) for the standardised one — a confusion we’ll learn to live with later, but not now.

The deviation — how far is a diamond from the middle?

residual deviation from the mean also: error

But the elephant is right. Not every diamond sits on the mean. Time for a game: the lucky dip.

Imagine: we put our twelve diamonds back in their chest, give it a good shake, and in a moment you’ll take one out without looking. What is your best guess for its shine?

The mean, of course — a shine of 50. That’s your best guess when you know nothing else. But as soon as you see the diamond you drew, you see how far it is from your guess. Below the mean, or above it. If you lay the shines out on a number line, low on the left and high on the right, you’re saying exactly the same thing with left and right. Which words you use just depends on how you lay them out. You draw Anna: 20 instead of 50 — you guessed 30 too high. Geer: 80 — 30 too low. Kees: 30 — too high again, by 20, a dull surprise for anyone who saw his carat of 2.1.

We call the difference between the real value and your guess the miss: how much your guess missed by. Sometimes positive, sometimes negative. Below or above the mean, left or right on the line. There are formal names for it too — deviation, residual, or error. Four words for the same thing.

For every diamond we can work out the miss — for carat too, not just for shine. As a formula:

\[\begin{aligned} \text{residual} &= \text{observation} - \text{expectation} \\ &= X_i - \overline{X} \end{aligned}\tag{1.2}\]

For Anna’s carat: \(1.0 - 1.6 = −0.6\). For Kees: \(2.1 - 1.6 = 0.5\). For Else (1.6 carats): \(1.6 - 1.6 = 0\) — Else falls exactly on the mean. Figure 1.2 draws the same kind of misses for shine: each stroke runs from the guess to the real value.

: Figure 1.2. The twelve diamonds against the dashed line of the mean: each stroke runs from the guess (50) to the real shine — Else and Leen lie exactly on it, so there is no stroke to draw.

Here’s a curious thing: if you add up all the residuals, you get zero. Not nearly zero, not roughly — exactly zero. Try it. That is precisely why we can’t simply take the mean of the ordinary residuals — the answer is always zero, which tells you nothing about the spread. And that’s no accident: the mean is, by definition, the point where the deviations balance out. Everything above the mean is balanced by everything below it. Otherwise it wouldn’t be a mean any more. Figure 1.3 lays the twelve out as blocks on a seesaw, with the pivot under the mean — and the seesaw stays level.

: Figure 1.3. The twelve shines as equally heavy blocks on a seesaw, the pivot under the mean of 50. On the left the deviations add up to −110, on the right to +110: together zero, so the seesaw does not tip. Anna sits far out, and on a seesaw that gives her more pull — distance counts, just as in a miss.

Annoying for us, because we wanted to say something about how far apart the diamonds lie. If the deviations cancel each other out, all we get is zero. A spread of zero. Another half-truth — just what the gull did not want.

Squaring — a small trick

Squaring means multiplying a number by itself. \(3^2 = 3 \times 3 = 9\). For a negative number: \((-3)^2 = (-3) \times (-3) = 9\) — a minus times a minus gives a plus, so a square is never negative.

The solution is beautifully simple. We want positive and negative deviations to count equally, without cancelling each other out. So we square all the deviations. The square of a negative number is positive, and the square of a positive number stays positive. The minus sign disappears.

Anna’s carat residual was \(−0.6\). Squared: \((−0.6)^2 = 0.36\). Kees: \((0.5)^2 = 0.25\). Else: \(0^2 = 0\).

If we add up all the squared residuals for carat, we get what is called the sum of squares, usually shortened to \(SS\):

sum of squares SS

The symbol \(\sum\) — the summation sign — is Greek for S, and it comes from sum. \(\sum_{i=1}^{n} X_i\) means: add up all the values \(X_i\), from \(i=1\) up to and including \(i=n\). Here: add up all 12 X-values.

\[ \begin{aligned} SS_X &= \sum_{i=1}^{n} (X_i - \overline{X})^2 \\[0.6em] &= 0.36 + 0.25 + 0.25 + 0.09 + 0 + 0 \\ &\quad + 0.01 + 0.04 + 0.09 + 0.09 + 0.25 + 0.25 \\[0.6em] &= 1.68 \end{aligned} \]

For shine we do the same:

\[ \begin{aligned} SS_Y &= (−30)^2 + (−20)^2 + (−20)^2 + (−20)^2 + 0^2 + 20^2 \\ &\quad + 30^2 + 10^2 + 20^2 + 30^2 + (−20)^2 + 0^2 \\[0.6em] &= 5200 \end{aligned} \]

square the shape also: a squared number, a town square

To square a number literally means to make a square. And a square is more than a shape: it’s a place, too. A town square is, at heart, a square with an area — Trafalgar Square in London, or its Dutch cousin, the Plein in The Hague — open spaces with sides and a floor. If you square a number \(S\), you get \(S^2\) — the area of a square with sides of length \(S\). So an \(SS\) — a sum of squares — is quite literally an added-up area: a whole row of squares.

That picture of the squares will come back later. All those little squares together then make a garden — all the variation together, with the sum of squares as its area. Explained variation (also called explained variance — both are fine, it’s the same number) is then the part of that garden where neat rows of flowers stand. Why both are fine, you’ll see in chapter 3. Areas, all of them. For now: remember square = area. The rest will follow. Figure 1.4 turns the strokes of figure 1.2 into squares.

: Figure 1.4. The misses of figure 1.2, flipped over into squares on squared paper — ten squares, because Else and Leen do not deviate and so have none; and the further from the mean, the faster the area grows.

Variance

variance notation: \(S^2\)

We divide \(SS\) not by \(n\), but by \(n-1\). Why?

Say I give you three envelopes with money in them, and tell you there are 10 euros in each envelope, on average. You open the first (7 euros), the second (11 euros). What’s in the third? Don’t guess — work it out: three envelopes with a mean of 10 makes 30 euros in total. We’ve seen 18 of it, so the third holds 12. The third is fixed as soon as you know the mean and have two values — only 2 (so \(n-1\)) are truly free. That is called degrees of freedom (\(df\)). Figure 1.5 lays the three envelopes side by side.

: Figure 1.5. Three envelopes, thirty euros together. Two lie open, seven and eleven, and nobody needs to guess the third any more: thirty minus eighteen is twelve. Two were free, the third was fixed — and that is why we divide by n − 1.

Back to our misses: with twelve diamonds and a known mean, only eleven of the twelve misses are free. The twelfth is already fixed — as soon as you know eleven of them, you can work out the twelfth, just as with those envelopes.

That’s why we divide \(SS\) by 11, not by 12: by the number of truly free misses. Feel it like this and there’s nothing to memorise: variance is still the average squared miss — add them up, divide by how many. Only here ‘how many’ means ‘how many free misses’ — one fewer than you thought. One odd correction, and that’s the end of it.

The formula, in two notations:

\[ \begin{aligned} S^2_X &= \frac{SS_X}{n-1} \\[0.4em] &= \frac{\sum_{i=1}^{n} (X_i - \overline{X})^2}{n-1} \end{aligned} \tag{1.3a} \]

\[ S^2_X = \frac{1}{n-1} \sum_{i=1}^{n} (X_i - \overline{X})^2 \tag{1.3b} \]

\[\begin{aligned} S^2_X &= \frac{SS_X}{n-1} \\[0.4em] &= \frac{1.68}{11} \\[0.4em] &= 0.153 \end{aligned}\]

\[\begin{aligned} S^2_Y &= \frac{SS_Y}{n-1} \\[0.4em] &= \frac{5200}{11} \\[0.4em] &= 472.7 \end{aligned}\]

This number is called the variance. Its unit: carats squared, shine squared. Rather odd — but then, we did square.

So variance is not yet the average miss itself. It’s the average squared miss — a stop along the way, not the destination. We’ll sort that out in a moment.

Before we take the square root — a moment’s pause. Do you see how often we’ve used the word mean — or average, which is the same word in everyday clothes? The average of the values (\(\overline{X}\)). The average of the residuals (= 0). The average squared residual (the variance). And in a moment the average miss (the SD). Four different averages — three times literally add-up-and-divide; the fourth takes a small detour via square and square root, but at heart it’s an average too. That word will be following you around for a long time yet.

Standard deviation — the average miss

The square root is the reverse of squaring. If \(S^2 = 9\), then \(S = \sqrt{9} = 3\). The square root asks: which positive number do I have to square to get this?

Figure 1.6 shows what the square root does here: the variance is one average square, and the square root gives you its side — an ordinary length, in ordinary shine points.

: Figure 1.6. All the squares of figure 1.4 together make a garden of 5200. Divide that garden into eleven equal strips, one for each truly free miss, and reshape one strip into a square: that is the variance, 472.7. The side of that square is the standard deviation, 21.74 shine.

To get rid of those odd squared units, we take the square root:

\[ S_X = \sqrt{S^2_X} \tag{1.4} \]

\[\begin{aligned} S_X &= \sqrt{S^2_X} \\[0.4em] &= \sqrt{0.153} \\[0.4em] &= 0.39 \end{aligned}\]

\[\begin{aligned} S_Y &= \sqrt{S^2_Y} \\[0.4em] &= \sqrt{472.7} \\[0.4em] &= 21.74 \end{aligned}\]

standard deviation SD notation: \(S\) also: the average miss

Standard deviation is the posh name. What is it, intuitively? Something like the average distance from a diamond to the average diamond, in the same unit as the variable itself. To put it another way: if you had to guess how far a random diamond lies from the mean, and you took the SD as your guess, you’d be close. That’s why we also call it the average miss. The elephant — still standing by the lake somewhere — would nod: then you know something.

For carat: an average miss of 0.39 carats. For shine: an average miss of 21.74 shine points.

That same SD doubles as your ruler: a measuring stick you use to measure how far a diamond lies from the mean. A diamond far from the middle lies many rulers away, one close by hardly any — remember that word, we’ll be measuring with it a lot. Figure 1.7 lays the ruler along the shine line.

: Figure 1.7. The shine line with the twelve above it, the mean at 50, and the standard deviation of 21.74 as a ruler: two to each side. Anna lies 1.4 rulers to the left of the mean, Geer exactly as far to the right. Most of the stones lie within one ruler.

The mean says what you’d guess; the standard deviation says how far off you’d be, on average.

In APA reporting, it looks like this:

M = 1.60, SD = 0.39 for carat.
M = 50.00, SD = 21.74 for shine.

Remember how that’s written. Capitals in italics, two decimals, separated by commas. You’ll be seeing it for the rest of the book, and long after.

One look at Kees

One last thought. Kees: carat 2.1 — that is 0.5 above the mean. Shine 30 — that is 20 below the mean. Kees is a big but dull diamond. Statistically speaking: his carat residual is positive and his shine residual is negative. Something there doesn’t quite line up.

For now, that needn’t mean anything. We’ve just measured Kees properly, along with the eleven others. But remember him — we’ll be coming back to him.

4. The cloud

We now have an average diamond and a feel for how far the others are from it. Before we close, one look at all twelve at once — carat on one axis, shine on the other, and the two means as dashed lines running through. That is figure 1.8.

: Figure 1.8. Each diamond one point — you can count eleven, because Bram and Cees lie on top of each other; that is why their dot carries ×2. Further right is heavier, further up is shinier, and the cloud creeps diagonally upwards.

What do you see? At the crossing of the two dashed lines stands exactly one point: Else — the average diamond in the flesh, as promised. Anna lies bottom left, Geer top right, and Kees lies to the right but low, far from the main direction. You can see that at a glance, without anyone pointing it out. But we’ll come back to Kees at length in a later chapter.

5. Closing

Right.

We now have an average diamond — one of 1.6 carats with a shine of 50 — and we know roughly how far the real diamonds are from it (the SD). The elephant would nod: there is an average diamond the others lie around, and you know how far ‘around’ is, on average.

But there is something else we can feel when we look at the cloud. The big diamonds mostly shine brighter. The small ones are mostly duller. Except Kees. Two kinds of difference that go together part of the way — a new question, a new chapter.

For now, we close. The lake still lies quiet, and the elephant has already taken another gulp.