Do survey incentives buy better data, or just more of it?
An incentive reliably lifts your response rate, but that is not the same as a better sample or cleaner answers. Here is what the evidence says about cash versus prizes, prepaid versus promised, and where paying people quietly backfires.
By SurveyLane · The team building SurveyLane
Offer people money and more of them fill in your survey. That much is settled, and it is where most of the thinking stops. The harder questions are the ones an incentive never answers. Are the extra responses the ones you were missing? Does paying change what people write? A raised response rate looks like a win on the dashboard, but whether it is a win for your data depends on who showed up because of the money, and who still stayed away.
What an incentive actually pays for
An incentive buys attention, not honesty. It moves your survey up the pile for someone who would otherwise have closed the tab, by changing the small private sum every recipient works out between the effort you ask for and what they get back. Usually that is nothing but a vague sense of having helped. A few euros, a gift card, a place in a draw: any of those tilts the trade.
What it does not buy is a better answer to any single question. Someone paid to be there is no more truthful than a volunteer. So the whole value of an incentive rides on pulling in people who differ from your eager volunteers, closing the gap between who you sampled and who actually replied.
Prepaid beats promised, and it is not close
The most robust finding here is about timing, not amount. Money included with the request works far better than money promised on completion. Alan Church's 1993 meta-analysis in Public Opinion Quarterly pooled 38 experiments and found that a small cash incentive placed in the envelope lifted mail response rates by roughly 19 percentage points on average. An incentive merely promised on return added under four, and only the up-front rewards produced a statistically reliable effect.
The mechanism is social, not economic. A prepaid incentive is a small gift, and a gift creates a light obligation to give something back. Money promised on completion is a transaction, and a few euros is a bad wage for ten minutes of work, so the recipient prices the deal and walks. That is why a two-euro coin taped to a letter can beat a promised ten that only lands if you finish. You are showing trust before you have asked for anything.
Cash beats the things you are tempted to replace it with
Given the choice, people value cash over a gift of the same nominal worth, and cash over a donation made to charity on their behalf. A branded pen, a keyring, an entry in a draw for a tablet: all of them underperform plain money, because their value to the recipient is uncertain and usually lower than the price you paid. If you cannot send cash, an unconditional voucher that spends like cash is the next best thing. The further you drift from spendable money, the more of the effect you give away. Think twice before the tote bag.
Small and prepaid beats large and promised
Returns on the amount diminish fast. The jump from nothing to a token prepaid sum is large. The jump from a token sum to a generous one is small, and sometimes invisible. A modest amount everyone receives up front does more per euro than a big amount a few people receive at the end, so the cost-effective design is usually a small prepaid incentive sent to the whole sample. The exception is a genuinely heavy ask, like a long diary study or a repeated panel wave, where a larger sum reads as fair payment rather than a nudge.
More responses is not a better sample
Here is the trap. An incentive lifts your response rate, and a higher response rate looks like lower nonresponse bias, so it is easy to assume the extra people fixed your skew. They might have. They might also be more of the same. If the reward mostly attracts the groups you already had plenty of, your rate climbs and your bias does not move.
Nonresponse bias is not a function of how many people declined. It is a function of how different the decliners are from the responders on the thing you are measuring. That is the same point running through who fills in your survey and who does not: a big pile of responses is not a representative one. An incentive earns its keep only when it converts the reluctant and the indifferent, the people whose absence was bending your estimates.
Does paying people wreck the answers?
The worry is fair. Pay people and you invite box-tickers who want the reward and not the conversation. The evidence mostly does not bear this out. Eleanor Singer and Cong Ye's 2013 review in The Annals of the American Academy of Political and Social Science, the standard synthesis of decades of incentive research, concluded that incentives have at most small effects on data quality, and that the effects on sample composition are inconclusive rather than clearly harmful. Paid respondents are not reliably lazier respondents.
That is not a licence to stop watching. An incentive raises the payoff for anyone willing to rush through nonsense to reach the reward, and on an open panel that pressure is real. The defence is to keep the quality checks you should be running anyway, the straightlining, speeding and copy-paste detection that monitoring response quality covers. Do not let the fear of a few bad rows leave the reluctant majority unmotivated.
The professional respondent problem
Where incentives genuinely distort things is at the extreme: online panels whose members complete surveys for a living. These are not your customers who happened to get paid. They are people optimising reward per minute across dozens of studies. They learn to pass attention checks and satisfice with practised efficiency.
An incentive did not create this person, but it is the water they swim in, and a poorly guarded panel fills up with them fast. If you buy responses from a panel, the incentive is baked into a supply chain you do not control, and your defence shifts from designing the reward to vetting the source: trap questions, consistency checks, and a hard look at anyone finishing far faster than the median.
Lotteries and prize draws are cheaper, weaker, fiddlier
The prize draw is popular because it is cheap. One tablet instead of a coin for everyone. It also works less well. A guaranteed small reward beats a tiny chance at a large one, because most people are not moved by a lottery ticket with near-zero expected value, and because it carries none of the prepaid gift's reciprocity.
Draws drag in complications that cash avoids. In many countries they count as promotions, with rules about published terms and eligibility. And to award the prize you have to identify the winner, so the anonymous survey now has a name and an email attached to it. If you promised anonymity, you have just broken it. The line between what anonymity and confidentiality actually mean is exactly where a prize draw trips people up.
Pay more for the hard-to-reach, but watch the fairness
Sometimes the reluctant groups are known in advance, and paying them more to haul them in can be the right call, because you are spending money where it reduces bias. It carries a cost, though: if respondents compare notes and find some were paid more for the same task, the unfairness can sour a panel you mean to survey again. For a one-off cross-section, paying the hard-to-reach a bit more is defensible. For a standing panel, consistency protects the relationship, and you reach the reluctant through reminders and design instead.
Incentives across repeated waves
In a longitudinal study the incentive is not a one-time nudge. It is the maintenance budget for a relationship. Pay well at recruitment and you set an expectation. Cut the reward in a later wave and dropout climbs, because people notice a downgrade more sharply than they noticed the original offer. The goodwill a prepaid incentive buys in wave one has to be renewed, or the panel erodes among exactly the people you were paying to keep. Burden matters here too: a panel that pays fairly and respects people's time keeps more of them than one that pays more but asks for an exhausting slog each round. The reward and the length of the survey are two halves of the same retention problem.
The consent and privacy angle
Paying a respondent almost always means identifying them. You cannot send a voucher to an anonymous session, so the moment you attach a reward you have linked a person to their answers, at least long enough to pay them. That link is personal data: it needs a lawful basis and a retention limit, and it has to be severed from the response data once the payment clears if you promised confidentiality. So design the payment path to stay separable, collect the payout detail somewhere that does not sit in the same row as the responses, and delete it on schedule. The failure mode is an incentive workflow that quietly re-identifies a survey you told people was private, which does more damage to trust than the incentive ever bought you.
A default worth starting from
Want a starting point rather than a research programme? For a one-off survey to your own list, a small prepaid or unconditional reward sent to everyone will out-earn a large prize promised to a few, and it keeps your promises about identity intact. Watch the output for the handful who came only for the money, and judge success by whether the reluctant groups moved, not by the response rate alone. Save the bigger sums for heavy asks, and the prize draw for when you cannot send anything else. Point the reward at that one job, converting the people whose silence was biasing your results, and it is money well spent. Point it at the response-rate number for its own sake and you are paying to make a vanity metric look better.
Frequently asked questions
Should the incentive be paid before or after someone completes the survey?
Before, wherever you can manage it. A small reward included with the invitation beats a larger one promised on completion, because the prepaid version works as a gift while a promised payment reads as a wage. Church's meta-analysis found up-front cash lifted response several times more than promised rewards, and only the prepaid form produced a reliable effect. If full prepayment is impractical, even a token amount sent in advance does more per euro than a big sum gated behind the finish line.
Does offering money make people give worse answers?
Mostly no. Singer and Ye's review found at most small effects on data quality and no clear harm to sample composition. The real risk sits with professional respondents on open panels who race through for the reward, not with your own audience getting a modest thank-you. So keep the usual checks running: flag the speeders, the straightliners and the copy-paste answers, paid or not.
Is a prize draw as good as paying everyone a small amount?
Usually not. A guaranteed small reward beats a tiny chance at a large prize, because a lottery ticket with near-zero expected value moves few people. Draws also pull in legal obligations around running a promotion, and they force you to identify a winner, which can quietly break an anonymity promise. They earn their place mainly when you cannot send everyone something.
A higher response rate means less bias, right?
Not on its own. Nonresponse bias depends on how different the people who declined are from those who replied, not on how many declined. An incentive that mostly rewards groups already well represented raises the rate without touching the skew. It helps only when it converts the reluctant whose absence was bending your estimates, so judge it by whether the hard-to-reach groups moved, not by the headline percentage.
