Polls: what they can say, what they cannot

Questioning a few hundred or a few thousand people makes it possible to estimate a proportion in a far larger population, with a margin of uncertainty that depends above all on the size of the sample and not on that of the population. That margin is not an admission of weakness: it is the description of the instrument's precision.

A practical consequence follows: two results separated by a gap smaller than that margin cannot be distinguished. Commenting on such a gap as a trend amounts to interpreting noise, and it is the most widespread error of reading, including among professional commentators.

One technical point deserves to be known: the margin announced holds for the sample as a whole. As soon as one looks at a subgroup, the young, the inhabitants of a region, the voters of a party, the numbers collapse and the uncertainty grows accordingly. The boldest commentaries often bear on the smallest subgroups.

The wording of a question modifies the answer, sometimes strongly, and the order of questions produces context effects. People who refuse to answer are not distributed at random, which introduces a gap that the size of the sample does not correct.

An answer declared in an interview is not an act. Between the two lie the intensity of the preference, the probability of turning out, and sometimes reluctance to admit a choice. Institutes correct these effects by adjustments, whose methods are published but rarely read.

These limits are documented by the institutes themselves, which generally publish the exact wording, the dates and the means of collection. Consulting them is the most profitable gesture for reading a poll correctly.

A poll describes a state of opinion on a date, under the conditions of the question asked. Presenting it as a forecast presupposes that nothing will move between then and the election, which experience regularly contradicts.

This confusion is sustained by the format of the commentary, which treats the points as scores in an ongoing competition. A poll at a given moment is closer to a photograph than to a prediction, and its value lies in the series it forms with the previous ones.

Three reflexes suffice. Look at the question asked before the figure announced. Compare with the series of earlier measurements rather than with the single previous one. And ask what the question did not ask, because a proportion on a precise question never generalises to a broader subject.

A fourth reflex is added for subgroups: look for the actual numbers. If they are not given, prudence requires concluding nothing from them.

A well-read poll is one of the few ways of knowing what a population thinks rather than what one's circle says of it. It is the best known remedy for the false consensus effect, and a useful counterweight to impressions drawn from conversations and social networks.

The right posture is therefore neither blind trust nor rejection, but instrumented reading: knowing what the tool measures, with what precision, and what it does not measure at all.