Write effective Screener questions

July 16, 2024 16:53

Learn the nuances of writing effective Screener questions.

✅ Plan Availability: All Plans
👥 User roles: Owner, Admins, and Researchers
For more information, see our article on how to find your plan and user role.

On this page:

What are Screener questions?
Watch out for leading questions
Is your screener doing too much?
Tips and best practices
Additional resources

What are Screener questions?

Screener questions are used to find participants who match the target audience for your research criteria and weed out those who don't.
A good screener is the first defense against bad data and can reduce costs, ease data analysis, and reduce the level of response bias.
The first step in finding the right participants is asking yourself, "Who is the target audience?"
In general, it's helpful to think of screener questions and targeting criteria as a funnel. The widest part of the funnel is participant demographic information:
- Age
- Income
- Education
- Gender
- Country location
- Job title
You’ve narrowed down the funnel. Now start asking more topic-specific questions.
How to do this depends on your recruiting method.
- Reruit with UserZoom:
  - We've done some of the heavy lifting for you! Configure your segments with specific targeting criteria and we'll find participants who match those criteria for you.
  - Criteria is only half the battle. You'll still want to add screener questions to further narrow down the pool of participants to match your needs.
- Recruit your own participants:
  - When taking this approach, you don't have built-in recruitment options.
  - Ask for this information in your screener questions.
Refer to our Screener Questions Samples article to get some ideas based on different industries.

Watch out for leading questions

One thing to consider is to not make it obvious as to what you are looking for and lead participants to choose your desired answer.

Don't ask:

Are you considering purchasing a washer within a year?

This is obviously a leading question, and a savvy participant (or even a well-meaning participant) can take the bait. So the best thing to do is to keep the purpose of your study ambiguous, making it more difficult for potential candidates to guess which answer will prevent elimination.

Ask this instead:

Which of the following items do you plan on purchasing within the next year?

Television
Refrigerator
Washer
Dryer
Computer
Car

Is your screener doing too much?

Think of screeners as security guards in a building. They let the right people in and keep the wrong people out. This usually means they aren’t also the chef, front desk, or valet. If you try and have your screeners do too much you might find out they haven’t done a good job. Use these tips to avoid Screens that try to do too much:

Ask the elimination questions first
This way you’re being respectful of their time and not asking them to do too much before screening them out.
Avoid using the screener to gather information
This is in line with the above point. You can always ask follow-up questions once you know they’re your target participant.
Eliminate conflicts of interest
Someone biased against your product type might not be the best person to talk to.
Screen for experience
Depending on the kind of feedback you’re after, you might want beginners or you might want experts. Screen for experience if it’s important.
Eliminate the usual suspects
Screen out participants whose feedback won’t be pushing forward your research goals.

Tips and best practices

Try these tips and best practices to write effective screeners that yield the right participants.

Avoid starting a screener with Personally Identifiable Information questions

Personally Identifiable Information (PII) questions can make participants uneasy about continuing.

Avoid Yes/No questions in your screener.

Binary (Yes/No) questions make it easier for participants to guess what profile you’re looking for.
- Yes/No questions suggest what you are looking for and can lead the participant to a passing response.
- People typically like to please and respond to what they think is the correct answer.
- This is not because potential participants straight out lie (though sometimes they do!), but because people often read questions and say to themselves, “Yeah, that’s me, I do that.”
Think of this behavioral tendency as being similar to when someone fills out an online personality test.
- People tend to fill out such tests based on an ideal version of themselves.
- Given this behavioral tendency, we recommend avoiding yes/no or leading questions as qualifiers.

Example

Say you are looking to speak with small business owners who have at least 1 employee. Your first reaction might be to write screener questions like this:

Do you currently own a small business?
- Yes (MAY Select)
- No (Disqualify)
Do you have more than 25 employees (outside of the owner and partners)?
- Yes (Disqualify)
- No (MAY Select)

The person filling out the screener might look at these questions and say, “I did sell some stuff on eBay, so sure that’s me!” Instead, we recommend making your screener more opaque. A less obvious set of screener questions might look like this:

Which of the following best describes your employment?
- I work for the Federal or State Government (Disqualify)
- I work full time for a company (Disqualify)
- I am a business owner (May select)
- I am an independent contractor (Disqualify)
- I am unemployed (Disqualify)
- I am retired (Disqualify)
- I am a student (Disqualify)
- I work part-time for a company (Disqualify)
- Other (May select)
How many employees does your business have?
- Just me - Sole Proprietor (Disqualify)
- Me and a business partner (Disqualify)
- 2-10 employees (May select)
- 11-25 employees (May select)
- 25-50 employees (Disqualify)
- 50 employees (Disqualify)

The first question narrows down the pool of qualifying participants so that the farther down the screener a participant gets, the more nuanced and detailed the screener should be.

Use questions to verify participant responses

For this scenario, let's say it's very important that the participants you speak to have an incorporated business and are not independent contractors.
In the US, incorporated businesses have an Employee Identification Number that can be used as qualify criteria.

Example

Instead of asking if the participants have an Employee Identification Number in a yes/no format, we can ask the following multiple-choice question:

Which of the following does your business have?

Dedicated checking account (May select)
Employee Identification Number (Must select)
Small business loan (May select)
Incorporation papers (May select)
Business credit card (May select)
None of the above (Disqualify)

In this example, it is acceptable for the participant to have a dedicated checking account and a business credit card, but the participant MUST have an Employee Identification Number.

Ask what you want to know (Moderated studies)

When screening for Moderated studies, ask additional questions to get a detailed understanding of a participant’s targeting criteria. For example:

If you want to know exactly what country/state/city your participant is located in, include a screener question that asks for this information.
If you want to know a participant’s exact income (and not just a range), include a screener question that asks for this information.

Additional resources

Between your research into who exactly your product/site/service is targeting and finding the right way to invite them into your study, recruiting the right participants can be a lot of work. This is why, once you’ve invested the time and energy in inviting participants to your study, your screener is so important. It’s there to ensure all your hard work doesn’t go to waste by acting as a quality check on the feedback you’re receiving.

Use these resources to help build and refine your Screeners:


Want to learn more? Check out these Knowledge Base articles... Diversity and Inclusion in screening Skip Logic in Screeners Save a Screener to the Screener Library	Interested in growing your skills? Check out our University courses... Participant recruitment & screener tips Recruit Your Participants UserZoom Writing Effective Screeners (Live Class)

Need hands-on training? Find a live, virtual class that fits your needs. Enroll in one of our UserZoom certification programs.	Can't find your answer? Reach out to Support or our Professional Services team.