How to Write Survey Questions That Don't Lead People

“How much do you love our new packaging?” gives your customer a job before they answer. They have to decide how much they love it, or push back against the question itself. Someone who barely noticed the box has nowhere obvious to go.

A leading survey question steers people toward a particular answer through its wording, assumptions or context. My rule is simple: write questions that make it as easy to disagree with your idea as to support it.

Think of the question as instructions for taking a photograph. Ask someone to photograph the damaged corner of a parcel and you'll learn about that corner. You won't learn what they noticed when they first opened the door. Survey wording works similarly. It directs attention, supplies a frame and sometimes suggests which answer would please the person asking.

I find this hardest when I already have a preference. Once I've spent time on an idea, a question that flatters it can sound perfectly reasonable. That's when I want a deliberate review, before anyone answers.

What makes a survey question leading?

The obvious version contains praise: “How helpful was our excellent customer service?” But a question can steer people without an enthusiastic adjective.

It might assume they used a product. It might offer four positive answers and one negative answer. Or it might follow a paragraph about plastic waste, then ask whether they prefer a refill pouch.

People draw on the information you give them to work out what you're asking. Some will also choose the nearest available answer rather than stop to explain that none fits. Your results can then reflect the question's framing as well as the customer's experience.

For a DTC brand, that can mean approving packaging people don't actually prefer, or reading polite interest in a new flavour as evidence of demand. More responses won't remove a bias you've built into every question.

Not every weak question is technically leading. A double-barrelled question creates ambiguity; a purchase-intent question asks people to predict themselves. I check for these together because they can all produce a reassuring answer that doesn't support the decision I'm making.

How do you remove leading wording?

Start by deleting words that do the evaluating for the respondent. Watch for “convenient”, “premium”, “affordable”, “improved” and “sustainable”. Some may be accurate descriptions in context, but they can still tell people how to judge what follows.

Before: “How much easier does our convenient refill pouch make your routine?”

After: “Thinking about the last time you refilled the bottle, how easy or difficult was it?”

Offer answers from “Very difficult” through “Neither easy nor difficult” to “Very easy”. Ask this only of people who have used the refill, or include a separate “I haven't used it” response.

The revised question allows the experience to have been difficult. It also asks about a specific event rather than accepting “convenient” as a fact.

If you want to find problems you haven't anticipated, ask first:

“What, if anything, was difficult about refilling the bottle?”

That wording deliberately focuses on difficulty, which is useful for diagnosing problems. Don't use the resulting answers to claim that difficulty was everyone's first impression. For that, start more openly: “Tell us about the last time you refilled the bottle.”

Neutral questions can be specific. You just need to be clear about what each one can tell you.

How do you fix double-barrelled questions?

A double-barrelled question asks people to judge two things with one answer.

Before: “How satisfied are you with the quality and price of our protein bars?”

Someone might love the bars and think they're too expensive. Their “Somewhat satisfied” answer leaves you guessing which part needs work.

After: Split the question.

• “How satisfied or dissatisfied are you with the taste of the protein bars?”
• “How would you rate their value for money?”

For the second question, use a balanced scale from “Very poor” to “Very good”, with a midpoint and “Not sure” if appropriate.

I replaced “quality” with “taste” here because quality could mean ingredients, texture, freshness or several other things. Choose the attribute your decision depends on. If you're investigating texture, ask about texture instead.

Look for “and” or “or” joining things someone could reasonably judge differently. “Easy to open and store” needs two questions. So does “clear and trustworthy”. If you only have room for one, choose the one you need rather than bundling them back together.

What should you ask instead of future purchase intent?

“Would you buy this?” is tempting because it sounds close to the business decision. The respondent, though, doesn't have to spend anything when they say yes.

Before: “Would you buy our new mango and lime drink?”

Better starting questions:

• “When did you last buy a flavoured sparkling drink?”
• “What flavour did you choose on that occasion?”
• “What other drinks, if any, did you consider?”

Use a recall period that fits the category. A question about the past week might suit frequent grocery purchases; it could miss nearly everyone buying a mattress. Include “I haven't bought one” and “I don't remember” where people may need them.

Past behaviour gives you evidence of what someone has actually done, although memory is still imperfect. It helps you distinguish a regular category buyer from someone who simply likes the sound of mango.

You can still ask about a future concept. I would show the same factual description to everyone, including pack size and price when relevant, then ask how likely or unlikely they would be to consider it on their next purchase. Keep a balanced scale and an option for people who aren't planning another purchase.

Treat that answer as stated interest, not a sales forecast. Follow it with “What makes you say that?” Later, a test involving an actual purchase decision can tell you something the survey cannot.

What does a balanced response scale look like?

Sometimes the wording is fine and the answer options do the pushing.

Before: “How would you rate the new packaging?”

“Excellent / Very good / Good / Fair / Poor”

That gives people several shades of approval and little room to distinguish negative reactions. “Fair” is also open to interpretation.

After: “Overall, how positively or negatively do you feel about the new packaging?”

“Very negatively / Somewhat negatively / Neither positively nor negatively / Somewhat positively / Very positively”

Add “I haven't seen it” separately if some respondents may not have seen it. That is different from having seen it and feeling neutral.

I wouldn't remove a genuine midpoint just to make people take a side. A forced choice can be useful when you're explicitly asking someone to choose between alternatives, but it changes the task. Don't quietly treat it as a measure of how they naturally feel.

Price questions need the same care.

Before: “How affordable is this great-value bundle?”

After: “At the price shown, how would you rate this bundle's value for money?”

Show the contents, quantities and price clearly, including delivery charges if they apply to the scenario. Use a balanced scale, and allow “Not enough information to judge”. This measures perceived value at that price. It won't establish the best price for your business by itself.

How do you remove hidden assumptions?

A question can assume an experience, a preference or a problem that the respondent never had.

Before: “What do you love most about our subscription?”

After: “What, if anything, do you like about the subscription?”

Then ask separately: “What, if anything, do you dislike about it?”

Before either question, establish whether the person has used the subscription. Ask current subscribers about their current experience and former subscribers about the period when they used it. If you can't route people to different questions, make the instructions and non-applicable answers explicit.

Another common assumption hides inside “Why”.

Before: “Why did you stop buying because of the price increase?”

After: “What led you to stop buying?”

Ask openly before offering a list of possible reasons. Price may appear without prompting. If you later ask about the price increase, first establish whether the person noticed it.

“What, if anything” helps remove an assumption, but it can't rescue a sentence that still tells people what happened.

How does question order affect answers?

Imagine a packaging survey that starts by describing the amount of waste created by single-use containers. Next, it asks customers to choose between a pouch and a rigid tub.

You've made waste salient before the choice. That's a reasonable design if you're testing reactions after reading a specific environmental message. It doesn't tell you what people would prefer without that message.

For an unprompted reaction, I would use this order:

1. Establish relevant purchase or usage experience.
2. Show the packaging with only the information needed to understand it.
3. Ask for an overall reaction in the customer's own words.
4. Ask about specific features, such as storage or opening.
5. Show any marketing claims you want to test separately.

Specific questions can feed ideas into later open-ended answers. If you ask about recyclability first, don't describe later mentions of recyclability as spontaneous.

Presentation order can affect comparisons too. Where your tool supports it, randomise which concept appears first across participants. Otherwise, you can use two versions with reversed order and check whether the pattern changes. Keep rating scales in a logical sequence; shuffling “Very easy” and “Very difficult” would just make answering harder.

A small study for a new flavour

Suppose you sell sparkling drinks and want to decide whether a mango and lime concept deserves a taste test. You aren't trying to settle the name, price and packaging in the same study.

I'd recruit people who have recently bought flavoured sparkling drinks, with a timeframe chosen for the buying habits you're studying. I'd avoid screening for people who already love mango unless that narrower audience is the explicit target.

A short survey could ask:

1. “When did you last buy a flavoured sparkling drink?” Include a non-buyer option for screening.
2. “What flavour did you buy most recently?” Include “Don't remember”.
3. “What led you to choose that drink?” Ask this before showing your concept.
4. Show a plain concept description. Ask: “What is your first reaction to this drink?”
5. “How appealing or unappealing does this flavour combination sound?” Use a balanced scale and “Not sure”.
6. “What, if anything, would you want to know before deciding whether to try it?”

I'd leave “refreshing tropical escape” out of the description unless the study is specifically testing that copy.

You might find positive appeal ratings alongside doubts about sweetness. That's a reason to investigate sweetness in the taste test, rather than declare the flavour ready to launch. A description can tell you about expectations. People need to taste the drink before they can judge its taste.

How Probe handles a study like this

With Probe, you can design a survey or interview study and send it to your own audience. Own-audience studies are free up to 10 questions and 25 responses. That can work for an initial wording check or a small exploratory study, but those responses won't automatically represent your whole market.

If you need recruited participants, they come from our recruitment partner's vetted panel. We show one all-in price per study before launch. Your eligibility criteria still need to match the people your decision concerns.

For packaging or flavour concepts, you can add images and media to questions. Use comparable images and descriptions so one option doesn't get better lighting, more detail or more persuasive copy than another.

Results come back as charts, themes and quotes. I would still read them alongside the exact question wording. A theme that appears after you explicitly prompt for it means something different from one customers raise on their own.

A checklist before you launch

Read the survey in the order a participant will see it, and check:

• Can someone dislike the idea without fighting the wording?
• Does each question ask about one thing?
• Have I established that the person had the experience I'm asking about?
• Where possible, am I asking about a specific past event?
• Do the answer options allow equally clear positive and negative responses?
• Can people be neutral, unsure or unable to answer where those are genuine possibilities?
• Have earlier questions supplied an answer I later want to call unprompted?
• Does each answer help with the decision this study is meant to support?

Before sending the survey widely, ask someone from the intended audience to try it and tell you what they think each question means. Listen for the moment they say, “Well, it depends.” That's often the question to rewrite first.