Asking about behaviour: what respondents can actually remember

"How many times did you contact our support team in the past year?" looks like a factual question with one right answer. For most respondents it isn't. Nobody keeps a running count of support contacts, so the number you get back was built on the spot, in a few seconds, from a vague sense of how often it tends to happen. Behaviour questions fail differently from opinion questions. The respondent is happy to tell you. The memory you are asking for just was never stored in the form your question assumes.

What happens between your question and their answer

Survey methodologists split answering into four steps, a model Roger Tourangeau, Lance Rips and Kenneth Rasinski set out in The Psychology of Survey Response (2000). The respondent has to understand the question, retrieve relevant memories, turn those memories into a judgement, and fit that judgement to the answer format you offer.

Behaviour questions break at the second and third steps. Retrieval is patchy, and the judgement step quietly swaps counting for estimating.

People estimate, they do not count

Memory doesn't keep a tally. When you ask "how many times", the respondent picks a strategy based on what's available.

Scott Burton and Edward Blair studied those strategies in Public Opinion Quarterly in 1991. For rare behaviour, people tend to enumerate: they recall each episode and add them up. For frequent behaviour they switch to a rate, as in "about twice a week, so roughly eight a month".

Both strategies fail in ways you can predict. Enumeration misses the forgettable episodes, so it undercounts. A rate describes someone's normal routine and leaves out the exceptions: the two weeks off sick, the month of daily visits during a project. Neither shows up in your data as an error. You get a clean number either way.

Match the reference period to the behaviour

The reference period is the window you ask about, such as "in the past 7 days". It's your biggest lever, and teams usually pick it to suit the quarterly report.

A rule that holds up: the more frequent and mundane the behaviour, the shorter the window. How often someone opened your app is a 7-day question at most. Whether they cancelled a subscription, moved house or filed a complaint is something people remember for a year or longer, because those events are rare and stand out.

"How many times did you use the search function in the past 12 months?" forces a rate estimate over a year nobody remembers in that detail. A daily user might answer 50 or 500, depending on how they did the sum. "On how many of the past 7 days did you use search?", answered from 0 to 7, is something people can actually count.

Short windows have a price. Fewer people will have done the thing at all, and one week may be untypical for any given person. Across a reasonable sample that evens out. You lose precision about each individual and gain accuracy about the group, which is usually what you wanted anyway.

Telescoping pulls old events into your window

People who remember an event still often misplace it in time. The classic study is by John Neter and Joseph Waksberg, published in the Journal of the American Statistical Association in 1964. They looked at how households reported spending on home repairs, and found respondents reporting jobs from before the reference period as if they fell inside it. The effect is called telescoping, after the way a telescope makes distant things look closer.

Forward telescoping inflates counts of memorable events. A customer who had a bad support experience fourteen months ago will often place it "in the past year". Your number goes up and nothing in the data says why.

How to bound the window

Neter and Waksberg's remedy was the bounded interview. In a panel, the previous interview marks what has already been reported, so the next one only counts new events. You can borrow the idea without a panel.

Anchor the window to a date or an event people can place. "Since 1 January" is easier to judge than "in the past 10 months". "Since the new pricing started in March" is easier still, if your respondents noticed the change.

Ask the longer period first, then narrow it. "Have you contacted support in the past year?" followed by "And in the past three months?" gives people somewhere to put the older events, so they don't squeeze them into the window you care about. You can throw the first answer away. Its job was to soak up the misplaced events.

"Usually" and "last week" are different questions

"How many hours a week do you usually spend in meetings?" and "How many hours did you spend in meetings last week?" look interchangeable. They aren't.

"Usually" invites a rate estimate of a typical week, and the typical week people describe is an idealised one. It drops the exceptions and leans towards how they see themselves. "Last week" points at days they can partly reconstruct, so some counting happens. Each individual answer is noisier. Across the sample it's less biased.

Choose on purpose. To classify individuals, "usually" may be the steadier signal. For an accurate team or company average, ask about a specific recent period and let the sample smooth out odd weeks.

Your answer categories tell people what is normal

Offer ranges instead of an open box and the ranges change the answers. Norbert Schwarz, Hans-Jürgen Hippler, Brigitte Deutsch and Fritz Strack showed this in Public Opinion Quarterly in 1985. They asked people how much television they watched a day. One group got a scale running from "up to half an hour" to "more than 2.5 hours". The other got one from "up to 2.5 hours" to "more than 4.5 hours".

On the high-range scale, 37.5 percent reported 2.5 hours or more a day. On the low-range scale, 16.2 percent did. Same population, same behaviour, more than twice the share. People read the middle of the scale as average and placed themselves against it. The low-scale group also rated television as more important in their lives, because the scale told them they watched more than most.

The effect is strongest for frequent, badly remembered behaviour that people estimate anyway. A product team that writes "Less than once a week / 1 to 3 times a week / 4 to 6 times a week / Daily / Several times a day" has already told respondents what normal use looks like.

Prefer numbers to ranges, and both to vague words

For countable behaviour, an open number field is usually better. There's no scale to anchor on, and you can bin the answers however you like afterwards. Set a sensible maximum in validation and add a short unit label such as "days" or "times".

If you do need ranges, base them on your own data or a pilot, and make sure the categories don't overlap or leave gaps. The post on designing multiple-choice questions covers the mechanics. Wherever you put the middle category, some people will take it as typical.

Skip vague quantifiers as answer options: "never, rarely, sometimes, often, always". "Often" means one thing for coffee and another for dentist visits, so two people with identical behaviour can tick different boxes. Vague scales are fine for perceptions, like how often someone feels rushed. For behaviour you could count, ask for the count.

Expect rounded numbers

Open number answers have their own quirk: heaping. People who estimate round to convenient values. Days cluster on 7, 14 and 30. A histogram of "how many times did you visit in the past month" will have spikes at 5, 10, 20 and 30 that have nothing to do with what people did.

Don't read those spikes as segments. They show you who estimated. Report medians or broad bands, because a handful of rounded large values drags a mean upwards. And if most answers sit on round numbers, your reference period is probably too long for the behaviour.

Filter questions teach people what "yes" costs

The usual structure for behaviour questions is a filter followed by detail. "Did you contact support in the past three months?" If yes: "How many times?" and "What was it about?" Conditional logic makes this easy and spares people who did nothing from questions that don't apply.

There's a side effect. When a survey asks several filters, each with a block of follow-ups behind it, people soon learn that "yes" makes the survey longer. Methodologists call what follows motivated misreporting: respondents say "no" to later filters to get out faster. The undercount grows further into the questionnaire, right where fatigue is already lowering quality, as the post on survey length and dropout describes.

One fix: ask all the filters first, as a grouped list, and only then the follow-ups for each "yes". Nobody can learn the pattern before they've answered every filter.

Do not ask what you can already measure

If you have usage logs, billing records or ticket history, a survey is the wrong tool for counting. A self-reported login count set against your logs mostly tells you how well people estimate, and you already know the answer to that.

Use the survey for what logs can't show, such as why someone did it. If you want behaviour and attitude side by side, link the response to the record you already hold, and say so in the invitation. The post on anonymity and confidentiality explains what you can still promise people after that.

Test how people arrived at the number

To see any of this, you have to ask about the process. In a cognitive interview, after someone answers a frequency question, ask "How did you work out that number?" The replies are telling: "I just guessed", "I thought about a normal week and multiplied", "I counted the ones I remember from the summer".

If most people describe a rate for something you wanted counted, shorten the reference period. If they mention events from outside the window, anchor it to a date. If they say "I picked the middle one", your scale is doing the answering. The post on pretesting a survey shows how to run those interviews with five to ten people before launch.