Designing a survey for the phone in your respondent's hand

You build the survey on a laptop and preview it on a laptop. Then a good share of your respondents open it on a phone, standing on a platform, thumb only. They get your words in a different shape. The grid scrolls sideways, the scale labels wrap onto three lines, and a fingertip covers half the slider handle. This post is about that gap.

Check your own device split first

The share depends on audience and channel. An employee survey sent during office hours mostly gets opened on desktops. A post-purchase email to consumers, or a link shared in a messaging app, mostly gets opened on phones.

Split starts, completes and breakoffs from your last few surveys by device. Say phones are a fifth of starts and a third of breakoffs. Then you know where the design fails. Never recorded device type? Start with the next survey.

Phone respondents answer as carefully as anyone

In a 2017 crossover experiment, Christopher Antoun, Mick Couper and Frederick Conrad invited 1,390 members of a probability-based web panel to complete a questionnaire twice, once on a smartphone and once on a PC. On the phone people multitasked more and more often had others around them. It made little difference. Respondents were at least as likely to give conscientious answers on the phone, and at least as likely to disclose sensitive information.

The phone fell short on the controls. Respondents struggled to move a small slider handle and a date-picker wheel to the value they meant. That pattern runs through this post: the person holding the phone is usually fine, and the trouble comes from design choices that assume a mouse and a wide screen.

Where phone respondents drop out

Breakoff is where a poor mobile design shows first. In a 2013 experiment, Aigul Mavletova assigned respondents to complete a web survey on either a PC or a mobile phone. Completion was 81.9 percent on the PC and 40.1 percent on the phone. Breakoff on the PC was almost 17 percentage points lower. Phone respondents also wrote shorter open answers, while socially undesirable and non-substantive answers were about as common on both devices.

Those are 2013 phones and a survey not built for them, so take the direction, not the size. A survey that is tolerable on a laptop is one people abandon on a phone. The post on survey length and dropout shows how fast answer quality falls as a survey grows. On a phone that curve starts earlier.

And breakoff skews your sample. If phone respondents leave and desktop respondents stay, your completes lean towards desktop users, who may differ from the people you lost in exactly the ways you wanted to measure.

Replace the grid first

A matrix question, statements sharing one scale across the top, is efficient on a wide screen and miserable on a narrow one. The columns shrink until the labels are unreadable, or spill off the side.

Aigul Mavletova, Mick Couper and Daniil Lebedev ran a two-wave experiment in an opt-in panel in Russia, published in 2018, with 1,678 respondents in the first wave and 1,079 in the second. They varied the format, grid or item by item, and the device, smartphone or PC. Grids produced more straightlining. Concurrent validity was lower in the grid on both devices. Grid respondents also reported more technical difficulties.

The grid did damage on PCs as well, so dropping it costs you nothing on desktop. Show each statement as its own question with a vertical list of options. The guide to spotting bad survey responses explains why straightlining is so hard to clean out once it is in your data.

Sliders, date wheels and drag-to-rank

A slider looks modern and works badly under a thumb. The finger hides the handle, the handle snaps to a value nobody chose, and many sliders start at a default that some people leave alone. Antoun, Couper and Conrad saw exactly this on phones.

Date wheels are no better. Spinning through decades to reach a birth year is slow, and an overshoot means spinning back. A short list of labelled radio buttons measures the same thing in one tap, and you can tell whether someone actually answered. For dates, ask only the precision you need. Month and year is often enough.

Drag-to-rank is worse. Dragging items into order on a small touchscreen is fiddly, and an accidental drop is recorded as an answer. If ranking matters, ask for the top choice, then the second.

One layout for every screen

Phone and laptop respondents should answer the same question. Responsive layouts quietly break that. A horizontal five-point scale on a laptop reflows into a vertical list on a phone. A long label gets cut off. A row of radio buttons turns into a dropdown that hides the options until someone taps it.

Each of those is a different stimulus, and a different stimulus can give a different distribution. Show phones a vertical scale with the positive end on top and laptops a horizontal one starting from the negative end, and part of any gap between devices is your layout.

So design for the smallest screen and use that layout everywhere. Don Dillman and his co-authors call this unified mode design in their book on the tailored design method: one presentation, the same for everyone. In practice that means vertical option lists on every device and labels short enough never to truncate. Skip dropdowns for anything under a dozen options. The guide to designing rating scales covers which scale formats hold up best.

Write questions that fit one screen

Shrinking the font makes a question fit and makes it harder to read. Cut words instead. A long stem pushes the options below the fold, and someone who scrolls down to the options and back up to reread the question has stopped reading carefully.

Compare:

• "Thinking about all of your interactions with our customer support team over the past three months, including email, chat and phone, how satisfied would you say you have been overall with the help you received?"
• "Over the past three months, how satisfied were you with the help from our support team?"

The second fits on one phone screen with its options underneath. If the channel matters, ask about it separately. Definitions that only some people need go in a short line under the question. The post on writing better survey questions goes deeper into wording.

Open questions on a phone keyboard

Typing on glass is slow, and it shows in shorter open answers. Ask fewer open questions and make them sharper.

"Any other comments?" gets a shrug on a laptop and nothing on a phone. "What almost stopped you from completing your order?" gives people a target and gets a usable sentence back. Put your most important open question early, before typing fatigue sets in. Do not make an open question compulsory unless you really need the answer. A forced text field on a phone fills up with "na" and "...".

When you analyse, a theme that turns up less often in phone answers may only reflect fewer words. The post on analysing open-ended answers covers coding them consistently, and where AI summaries help or invent themes that are not there.

Paging or one long scroll

One question per page means a tap and a load for every answer, which drags on a weak connection. One long page lets people lose their place, and conditional questions further down get missed.

A sensible default is short pages of related questions, one or two screen heights each, with the next button always within reach. Put a conditional follow-up directly beneath the question that triggers it, on the same page, so the respondent sees it appear. And keep progress honest. A bar stuck at 20 percent after five minutes tells a phone respondent to leave.

Tap targets and the thumb

Small radio buttons packed close together are hard to hit, and a mis-tap often goes unnoticed. Someone taps "Agree", the screen registers "Strongly agree", and the survey moves on.

WCAG 2.2 sets a minimum target size of 24 by 24 CSS pixels in success criterion 2.5.8. Apple's Human Interface Guidelines recommend 44 by 44 points for touch targets, and Material Design recommends 48 by 48 density-independent pixels. Use the WCAG number as the floor and the platform numbers as what you aim for. Make the whole option row tappable and leave space between rows. The guide to accessible surveys covers the rest, and nearly all of it helps one-handed respondents too.

Speed and the first screen

A survey that takes several seconds to load on mobile data loses people before question one. Skip big header images and heavy scripts.

Then remove whatever sits between the invitation and question one without a reason. A login wall costs starts. So does a cookie banner covering half the screen. If you need consent text, keep it to one readable screen and link to the full version. The post on getting invitations delivered covers the step before this.

Test on the phones your respondents use

A browser's responsive preview will not show the keyboard pushing the next button off screen, or how hard a radio button is to hit on a moving bus.

Before launch, complete the survey yourself on at least two real phones, one small and one large, iOS and Android if your audience uses both. Do it one-handed. Then watch two or three people from your audience do it without help. The guide to pretesting a survey explains how to run those sessions and a soft launch. For mobile, have the soft launch report breakoff by device, so a phone-only problem shows up before the full send.

Record the device and read results by it

Record device type with every response and analyse it like any other variable. Compare breakoff, completion time, straightlining, open-answer length and your headline results by device.

Phone and laptop respondents are rarely the same people, so a gap may come from who they are. Check whether it holds within comparable groups, such as one age band or customer segment. If it shrinks, the device was standing in for the population. If it stays, look for a layout difference in the question itself.

You can ask this of your data in plain language. With the SurveyLane MCP server, ask an assistant to split breakoff or a satisfaction score by device and you will see quickly whether one question is failing on phones.