The limits of an opinion test, stated frankly

An opinion questionnaire collects declared positions, at a given moment, on situations worded by someone else. That is already a great deal: aggregated over several items, these declarations form positions that are stable and coherent enough to be compared between people.

This material makes it possible to situate, to distinguish and to prompt reflection. It becomes misleading only when extended beyond what it contains, and that extension is almost always in the same direction: from the declared position towards the whole person.

A clarification about the nature of the questions must be added. A voluntary test, taken out of curiosity, does not collect the same thing as a survey on a representative sample: neither the same public, nor the same care, nor the same stakes. Both have their value, but they do not answer the same questions.

It does not predict conduct. Between a declared position and an act lie the intensity of the preference, the cost of the act and the circumstances, none of which is measured by the questionnaire.

It diagnoses nothing. A political position is not a personality trait, still less a psychological state, and models that slide from one to the other cross a frontier nothing justifies.

It does not say whether you are right. That is an obvious point that nonetheless has to be written down, because a result that is quantified and carefully presented gives an impression of authority that easily spills over onto content. A position may be perfectly measured and perfectly debatable.

The wording of items influences answers; the number of questions per dimension limits the achievable fineness; mood and the context of the sitting introduce a real variation from one time to the next. A person may therefore obtain a slightly different result two weeks apart without having changed their mind.

To this is added the effect of format: the length of the scale, the presence of a midpoint, the order of the questions. None of these choices is neutral, and a serious model should be able to say which it made and why.

A test that announces its limits loses a little prestige and gains a great deal in credibility, because it gives the reader the means to judge what they are holding. It is also the only tenable position over time: limits always end up being seen.

There is a more self-interested argument, and it is honest to mention it: a model that promises more than it can deliver manufactures disappointments, and a disappointment on a profile test translates into immediate abandonment.

The broad orientations are reliable, small gaps between close dimensions should not be over-interpreted, and a tight attachment to one profile rather than its neighbour is an indication, not a verdict.

The most useful thing is in any case not the result but the gap: between what you thought you would find and what you find. It is the only part of the report that nobody can write in your place.