Should surveys offer “no opinion”? The dilemma of the opt-out box

Without an opt-out box, a person who has never thought about a question must nonetheless choose an answer. They produce one, often on the spot, and that answer will be counted as an opinion just like the others.

With an opt-out box, the risk changes in nature. People who do have a view, even a hesitant one, may prefer this option because it requires less effort or avoids taking a stand on a sensitive subject. The first defect adds noise, the second removes information.

Philip Converse showed in 1964 that, from one survey wave to the next, some of the answers varied as if they had been drawn at random. He concluded that many answers did not reflect a settled attitude.

Howard Schuman and Stanley Presser, in Questions and Answers in Attitude Surveys in 1981, asked people about fictitious laws. A share of respondents gave an opinion, and that share fell markedly when an explicit non-response option was offered.

In 1991, Jon Krosnick proposed analysing certain answers as satisficing behaviour: the respondent provides an acceptable answer rather than the best possible one. A “no opinion” box offers exactly this shortcut.

From this perspective, offering the box does not guarantee more accurate data. It removes improvised answers, but also genuine positions expressed by people who are tired or cautious, and the exchange is not always a favourable one.

The same box can cover very different situations: not knowing the subject, not wanting to answer, being indifferent, or being torn between two positions. Lumping them together amounts to treating ignorance and ambivalence as a single thing.

Several solutions exist. A filter question first asks whether the person has a view, before recording it. Separate options distinguish the absence of a view from refusal. None is perfect, since each added step lengthens the questionnaire and may in turn wear people out, but each makes silence easier to read.

The opt-out box is not the midpoint of a scale. The middle expresses a position, however nuanced; the opt-out signals that no position is available. Choosing one for the other changes the meaning of the answer, and distorts the reading of the result, which counts the middle as a position but ignores the opt-out.

Before choosing the opt-out, one question helps decide: if I absolutely had to lean one way, would I know which at once? If so, a position exists, however weak, and it deserves to be expressed.

Conversely, forcing oneself to answer on an unfamiliar subject adds nothing. An honest result accepts blank areas, which reveal the questions one has not yet thought about, and which can be filled in later.