How long should a survey be? Designing for completion

You can write flawless questions and still ruin a survey by asking too many of them. Length works against you in a way most people never account for. Every item you add lowers the odds that someone finishes, and it drags down the answers to the questions they already reached. The survey that gets you the cleanest data is almost never the one that asks everything you were curious about.

Length is a decision you make

Most surveys get long the same way. Someone drafts the questions they care about. A few stakeholders each add "just one more." Nobody owns the total. So length ends up as the sum of everyone's wish list, a number that nobody actually chose. That is the mistake. How long your survey runs is one of the highest-leverage calls in the whole design, and it deserves the same care you would spend wording a single item.

It matters this much because length hits you twice. It lowers how many people start and finish, which shrinks your sample and skews it toward the patient. And among the people who do push through, it wears down the quality of their answers as they tire. You pay once in quantity and again in quality. Neither cost shows up in a tidy export. You just see a smaller dataset and slightly worse numbers, with nothing to point back at the length.

What the research actually found

The most cited experiment here is Galesic and Bosnjak, published in Public Opinion Quarterly in 2009. They ran a web survey where they told different respondents it would take 10, 20 or 30 minutes, and they moved the same questions to different positions so they could watch quality shift as people tired. That design pulls apart two things people usually blur together: what the announced length does to who bothers, and what a question's position does to how well it gets answered.

Both effects were real and both ran the same direction. The longer the stated length, the fewer people started, and the fewer of those starters actually finished. Completion dropped a long way from the 10-minute version to the 30-minute one. This is a 2009 study rather than fresh news, but the finding is evergreen and has held up across replications. Tell people a survey is long and you lose respondents before they answer a single question.

The back half is where quality dies

The other half of that result is the one people forget. Questions placed later in the questionnaire got answered faster, skipped more, drew shorter open-ended answers, and showed less variation in grids than the identical items placed earlier. That last one is the tell. Less variation in a grid means people stopped reading each row and started ticking one column straight down. That is satisficing, plain and visible.

So the damage is lopsided. Your first ten questions get fresh, attentive people. Your last ten get tired ones hunting for the exit. Park your most important measure at the end because it felt like a strong finish, and you handed it your worst data. It is the same failure described in monitoring response quality, except this time you caused it, by making the survey too long to hold attention.

Every question has a marginal cost

Think of each question as having a price, paid in dropout and in attention. The first few are nearly free. The respondent arrived willing. Each one after that costs a little more than the last, because you are spending down a budget of patience that never refills. Somewhere past the midpoint the price per question climbs steeply, and a question that would have earned a careful answer at position five gets a shrug at position twenty-five.

That reframes the usual argument in a meeting. The question to ask about any item is not whether it is interesting. Almost everything is interesting. The question is whether it earns its keep, given the answers it will cost you on every other question and the respondents it will cost you outright. Put it that way and a lot of nice-to-have items quietly die. The survivors get better data, because they are not buried under filler.

Sort every question into must, should, and nice

Before a survey ships, force every question into one of three buckets. A must-have maps to a decision you will make or an analysis you already planned. If the answer would not change anything you do, it is not a must. A should-have strengthens the analysis without holding it up. A nice-to-have is something you added because you were curious, or because someone asked nicely and you did not want to say no.

Then cut the whole nice-to-have bucket. Not most of it. All of it. Half-cutting is how surveys stay bloated, because every single item looks defensible on its own. The trick is to decide by category, not one question at a time. Here is a test for the must bucket: for each question, write the sentence you would put in the report if the answer came back one way versus the other. Can't write it? Then the question is not driving a decision, and it does not belong near the front, if it belongs at all.

Time is the number respondents feel

Respondents do not count your questions. They feel minutes, and their tolerance depends on how the survey reached them. Someone who clicked a link at the bottom of a transactional email gives you a minute or two. A recruited panellist, paid for their time, will sit through far more. Match the length to the channel and the relationship you have, not to how much you wish you could learn.

Measure the real completion time in pretesting instead of guessing, because your own estimate is always low. You know the questions, so you read them in half the time a stranger needs. And check it on a phone. A survey that takes four minutes on a laptop can take seven on a small screen with a fiddly dropdown, and most of your respondents are on phones. The layout choices in designing accessible surveys feed straight into completion time.

The announced length also decides who agrees to begin, so be honest and be specific. "This survey takes about 4 minutes" beats both silence and a mushy "a few minutes," because a concrete, believable number lets someone say yes with confidence. Overstate it and you scare people off. Understate it and you get a wave of angry dropouts the moment they realise you lied, usually right after they have given you enough to feel used. That promise is really part of the invitation, which is the subject of getting your survey invitation into the inbox.

Progress indicators help, until they hurt

A progress bar is meant to reassure, and on a short survey it does exactly that. On a long one it can turn on you. If the bar races ahead at first and then crawls, people see how much is left and quit right where the slowdown becomes obvious. The research on progress feedback is mixed for this reason. A bar showing fast progress nudges people to finish. A bar showing slow progress talks them out of it.

So show a progress indicator when the survey is genuinely short, or when the pace will feel steady the whole way. If yours is long and front-loaded with quick questions, an honest bar mostly advertises the slog ahead. Page-based progress, something like "page 2 of 4," is often kinder than a percentage. It hides the uneven weight of individual pages and gives people milestones they can actually reach.

One long page or many short ones

Pagination is a real lever on completion, well beyond a layout preference. One long scrolling page can feel efficient when the survey is short. Make it longer and that same page shows the whole mountain at once, which invites people to bail before they start. Splitting into short pages hides the total and creates a sense of movement, since finishing a page feels like a small win that tugs you toward the next.

There is a cost to it. More pages mean more server round-trips and more chances for a shaky connection to drop someone mid-survey, and that bites hardest on mobile. Group questions into pages that hang together as topics, keep each page short enough to feel quick, and never split one logical block across a page break where the context gets lost. It all pairs with how you sequence the survey, which is the subject of question order effects.

Let people stop and come back

For any survey long enough that someone might not finish in one sitting, save partial responses and let people resume. Two good things follow. You recover answers you would otherwise have binned, because someone who quits at question eighteen still gave you seventeen usable answers if you kept them. And you get a real dropout curve to learn from, since every abandoned response marks a spot where the survey lost someone.

Do not silently throw partials away. Even without a resume link, the questions someone answered before quitting are often analysable on their own, as long as you stay honest that the later questions rest on a smaller, more self-selected base. Bin every incomplete response and you have binned the exact evidence that would have told you the survey is too long.

Grids are a length trap

Matrix and grid questions look compact because they cram many items onto one screen. But they carry the fatigue of every row while wearing the face of a single question. That is why grids are where Galesic and Bosnjak watched variation collapse late in the survey. A tired respondent staring at a ten-row grid ticks one column down the list and moves on. The compactness is visual. The work is not.

If a grid is pulling real weight, keep it short and put it early, while attention is still high. If it is long, break it up, cut rows, or ask yourself whether you truly need every item. The scale inside those rows is its own decision with its own effect on quality, covered in designing scale questions. A short grid asked early beats a thorough one asked at question thirty that everyone straightlines.

Use branching to shorten the survey per person

The best way to make a survey shorter is to stop showing people questions that do not apply to them. Conditional logic lets each respondent answer only the relevant subset, so the survey feels short to everyone even when the full instrument is large. Someone who says they have never used a feature should not then hit five questions about that feature. Skip them straight to what fits.

The catch is that branching changes the length and the order along every path. A survey that feels tight on the route you tested can turn long and strange on one you never walked. Reason about the longest path a real respondent can land on, not the average, and confirm that even the worst case stays inside your time budget. The mechanics of building those paths live in conditional logic and feedback.

Measure your own dropout curve

Stop guessing where your survey loses people. Look. Plot completion against question position and the exact quitting points jump out: usually a steep early drop as the uncommitted leave, then smaller cliffs at specific questions. A sharp cliff at one question is a signal. It is too personal, too confusing, or the spot where built-up fatigue tips people over. That question, or its position, is what you fix.

This belongs in pretesting, not a post-launch autopsy, and it slots naturally into a soft launch, which is the subject of pretesting a survey before you send it. Send to a small slice of your sample first, read the dropout curve, cut or move whatever is shedding people, and only then open the gates. The alternative is finding out afterward that a third of your sample walked at question twelve, when all you can do is explain it away.

A length budget you can apply

Set the time budget before you write a single question, based on the channel and how much you are asking of these particular people. Sort every question into must, should and nice, and cut the nice bucket wholesale. Put your important measures early, while attention is fresh, and never hold a load-bearing question back for a strong finish. Branch so nobody answers what does not apply to them, and check the longest path rather than the average. Show progress only where it will reassure. Split a long survey into short pages. Save partials so a quitter still leaves you something. Then soft-launch, read the real curve, and cut again. Length is the rare design choice that damages both how many answers you collect and how good they are, which is the whole reason to decide it on purpose rather than let it pile up.