bcarroll-wise

screener-builder

"Use this skill to design recruitment screeners for user research studies. Takes research plan + sampling strategy as input, outputs survey questions + segmentation logic + qualification criteria. Stage-gated process with researcher approval at each step ensures questions map to research goals, avoid bias, and yield reliable participant selection. Hands off to screener-segmentation skill for analyzing responses. Always applies Straight Shooter style throughout."

bcarroll-wise 0 Updated 6d ago

Resources

16
GitHub

Install

npx skillscat add bcarroll-wise/quantitative-uxr

Install via the SkillsCat registry.

SKILL.md

Screener Builder

A recruitment screener is how you find the right participants for your study. A bad screener lets through people who don't match your research goals, waste your time and budget, or bias your findings. A good screener filters for the behaviors and characteristics that matter, avoids revealing the study purpose, and produces a prioritized list of qualified participants.

This skill is part of the recruitment chain:

  • screener-builder (this skill) — research plan → screener questions + segmentation logic
  • screener-segmentation — survey responses → prioritized interview list with confidence ratings

Always apply Straight Shooter style throughout.


How this skill works: staged check-ins

The skill runs as five required stages plus one optional stage (6) with a mandatory check-in after each one. Surface what you've done, ask the researcher to confirm or correct, then move on. Never collapse stages without explicit permission.

Stage What you do What you surface What the researcher does
0. Orient & commit to memory Read research plan + sampling strategy. Summarize goals, target users, key behaviors/attitudes, exclusions. Six-item summary of what you'll screen for Confirms or corrects
1. Strategy Map research goals to screening approach: what behaviors, attitudes, structural characteristics need to be asked Strategy table showing what you'll ask and why Confirms strategy before questions are written
2. Plan & structure Design screener structure following best practices. Propose question flow. No actual questions yet. No segmentation logic yet. Screener blueprint with question types, flow Confirms structure, splits or merges sections
3. Question drafting + survey logic Write questions one at a time, applying all best practices. After each question is approved, ask if there's branching/skip logic based on that question's answers. One question at a time with answer options, rationale, then survey logic for that question Approves question, then defines any skip patterns or branches
4. Assemble screener Put all approved questions together in final format with introduction and complete survey logic summary Complete screener ready for survey tool with all branching documented Reviews, confirms ready to use
5. Segmentation logic After screener is confirmed, build segmentation logic step-by-step through substages: 5a (target profile), 5b (segments), 5c (prioritization), 5d (exclusions), 5e (analysis notes) Each component of segmentation logic, one at a time Confirms each piece before proceeding to next
6. Generate script (optional) If researcher wants to automate segmentation, generate Python or R script that applies Stage 5 logic to CSV data Executable script with instructions Tests with their data, requests modifications if needed

Default behavior: stop and check in after every stage. Wait for "go" before moving on.

If the researcher says "do all six and surface at the end": you can run end-to-end, but you must still surface intermediate artifacts (strategy, structure, individual questions with rationale and survey logic, final screener, segmentation logic) — the check-ins still happen, just compressed into one delivery.

Why staged. Screener problems compound. A question that looks fine in isolation can reveal study purpose when combined with others, or fail to differentiate when the overall structure is wrong. Stage gates catch structural issues before you've written 20 questions.

Why survey logic is defined per-question. Defining branching and skip patterns as each question is created ensures the logic is correct and complete. It's easier to think through "what happens after this specific question" immediately after designing it, rather than trying to retroactively add logic to 10 questions at once.


Required inputs before starting

You need two things before you can build a screener:

  1. Research plan — what you're trying to learn, what decisions the research will inform, key research questions
  2. Sampling strategy — who you want to talk to (inclusion criteria), who you don't want (exclusion criteria), any segmentation needs, target sample size and quotas

If either is missing or unclear, ask before proceeding. Do not infer a sampling strategy from a vague research brief.

If the researcher provides a research plan but no explicit sampling strategy, ask:

  • Who are your target users for this study?
  • Are there specific behaviors or characteristics that matter?
  • Who should be excluded?
  • Do you need to compare across segments?
  • What's your target sample size per segment?

Stage 0: Orient & commit to memory

Read the research plan and sampling strategy in full. Summarize them back to the researcher in this format:

**Research goal:** [one sentence — what decision does this research inform]

**Key research questions:** [bullet list — what you're trying to learn]

**Target users (inclusion):** [who you want — behaviors, demographics, attitudes, usage patterns]

**Exclusions:** [who you don't want — professional testers, employees, wrong segment, etc.]

**Segmentation needs:** [if you need to compare groups, list them here with target quotas]

**Output format:** [survey tool format if specified, or format for segmentation skill]

**Screening approach:** [exclusion-based or collect-all-and-segment]

IMPORTANT: Ask about screening approach

Before confirming the orient, explicitly ask the researcher:

"How do you want to approach screening?"

Option A: Exclusion-based (traditional recruitment screener)

  • Hard disqualifications during survey (e.g., if doesn't meet criteria, terminate)
  • Only collect responses from target profile
  • Used when recruiting specific participants for interviews/studies
  • Survey logic in Stage 3 will include termination logic

Option B: Collect-all-and-segment (classification/profiling survey)

  • No terminations - all responses collected
  • Segmentation happens after data collection (Stage 5)
  • Used when goal is to classify/profile all respondents into segments
  • Survey logic in Stage 3 will have no terminations

Most user research screeners use Option A. Classification surveys (like market segmentation, customer profiling) use Option B.

Document the screening approach in the orient summary:

**Screening approach:** Exclusion-based (traditional recruitment screener)
- Hard disqualifications during survey
- Only collect responses from target profile
- Survey logic in Stage 3 will include termination logic

OR

**Screening approach:** Collect-all-and-segment (classification/profiling survey)
- No terminations - all responses collected
- Segmentation happens after data collection (Stage 5)
- Survey logic in Stage 3 will have no terminations

Check-in 0 → 1: Confirm the orient including screening approach. Once confirmed, move to Stage 1.


Stage 1: Strategy — map goals to screening approach

Before writing any questions, decide what you need to ask and why. Map each element of the sampling strategy to a screening approach.

Key principle: Prioritize behaviors and psychographics over demographics.

Demographics (age, gender, income, location) are the easiest to screen for, but they're often the least important. Unless demographics directly affect your research question, focus on:

  • What people do (behaviors, actions, usage patterns)
  • What people value (attitudes, priorities, motivations)
  • How people think about the problem space (psychographics)

Demographics are prone to bias and often irrelevant to how people interact with products. Screen for how people behave, not how they're categorized on a census.

Exception: Use demographics when:

  • They directly affect the research question (e.g., retirement planning tool for 55+)
  • You need geographic proximity (in-person studies)
  • You're testing for accessibility across diverse groups
  • Your recruiting panel doesn't already provide demographic data

Surface strategy in this format:

| Research need | What to screen for | Question type | Rationale |
|---|---|---|---|
| [from research plan] | [behavior / attitude / characteristic] | [behavioral / demographic / psychographic / firmographic] | [why this will differentiate] |

Example strategy table:

Research need What to screen for Question type Rationale
Understand why SMBs churn from Wise Has used Wise in past 12 months but stopped Behavioral Need active churners, not people who never tried
Compare UK vs US experiences Located in UK or US Demographic Research scope limited to these markets
Identify high-value vs low-value users Transaction volume and frequency Behavioral Value segment likely affects churn reasons
Exclude professional testers Participation in user research in past 6 months Behavioral Professional testers behave differently

Also surface:

  • What you won't ask — anything available from user profile data (age, gender, location if already known), anything that would reveal study purpose too early, demographics that don't affect the research question
  • Screening sequence — broad to narrow (funnel technique), behavioral/psychographic first, demographic last, most revealing questions at the end

Check-in 1 → 2: Researcher reviews strategy. Common corrections:

  • "You're screening for X but that's not actually what differentiates"
  • "Add screening for Y — that matters more than I mentioned"
  • "Don't ask about Z, it'll give away the study purpose"
  • "Reorder — ask behavioral before demographic"

Wait for confirmation before designing structure.


Stage 2: Plan & structure — design the screener blueprint

Now design the screener structure. No actual question wording yet — this is the architecture. No segmentation logic yet — that comes in Stage 5 after questions are confirmed.

Surface structure in this format:

**Section 1: [name, e.g., "Usage behaviors"]**
- Question type: [single-choice / multi-choice / open-ended / scale]
- Purpose: [what this section captures or what dimension it measures]
- Optional: Disqualification points [if hard terminations are needed]

**Section 2: [name]**
- Question type: [single-choice / multi-choice / open-ended / scale]
- Purpose: [what this section captures]
- ...

**Section 3: [name]**
- ...

**Question flow:**
- All respondents answer Q1-Qn
- [Or, if branching: respondents who answer X to Q3 skip to Q6]

**Optional: Hard terminations (if needed):**
- Terminate if [condition that makes someone completely ineligible]
- Note: Many screeners don't need terminations — instead, all responses are collected and segmentation happens after data collection

Note on terminations vs segmentation:

  • Hard terminations are for people who literally cannot participate (e.g., don't use the product category at all, wrong geographic location for in-person study, already participated recently)
  • Soft disqualification (not target profile but responses still useful) should be handled in segmentation logic at Stage 5, not by terminating the screener
  • When in doubt, collect all responses and let segmentation decide who to prioritize

Structure rules (apply throughout):

  1. Funnel from broad to specific. Start with questions that don't reveal study purpose. Save the most revealing questions for later sections.

  2. Behavioral and psychographic questions first, demographics last (if at all). The typical order:

    • Disqualifying behaviors (must-haves: uses X, does Y, experienced Z)
    • Psychographics (attitudes, values, priorities)
    • Articulation question (open-ended, tests communication ability)
    • Demographics (only if needed and not already available)

    Why this order: Behaviors and psychographics are what actually differentiate users. Demographics are usually less predictive of how someone will interact with your product.

  3. Eliminate unqualified people early. Put the most restrictive qualifying criteria first (e.g., location for in-person studies, must-have behaviors). Don't make participants complete 10 questions before finding out they don't qualify.

  4. Limit length. Aim for 5–10 questions maximum. Longer screeners narrow your pool too much and increase dropout.

  5. Multiple qualification paths. Where possible, allow different combinations of answers to qualify (e.g., "current user" OR "churned in past 6 months" OR "actively shopping").

  6. Explicit disqualification points. Mark where participants get terminated, so you can test the logic.

Check-in 2 → 3: Researcher reviews structure. Common corrections:

  • "Section 2 reveals the purpose too early — move it later"
  • "Add a section for [missing dimension]"
  • "Merge sections 3 and 4, they're asking about the same thing"
  • "Add a section for [missing dimension]"

Wait for confirmation before writing questions.


Stage 3: Question drafting + survey logic — one question at a time

Now write the actual questions, one at a time. For each question, follow this two-step process:

Step 3a: Draft the question

Surface the question with proposed routing logic:

**Question [number]:** [exact question wording]

**Answer options:**
1. [option 1]
2. [option 2]
3. [option 3]
...
[n]. Other / None of the above / I don't know (catchall)

**Rationale:** [what this question measures, why it's worded this way]

**Proposed routing logic:**
| IF Q[n] = [answer] → [action]
| IF Q[n] = [answer] → [action]
| ALL respondents → [default action]

[If no routing needed, state: "No routing logic - all respondents continue to next question"]

**Randomization:** [State if answer options should be randomized, or "No randomization"]

**Best practice check:**
- ✅ or ❌ Avoids yes/no?
- ✅ or ❌ Avoids leading language?
- ✅ or ❌ Uses clear, jargon-free language?
- ✅ or ❌ Provides catchall option?
- ✅ or ❌ Focuses on past behavior, not predictions?
- ✅ or ❌ Provides reference frame (if needed)?

Wait for approval of the question wording, answer options, and proposed routing logic before proceeding to Step 3b.

Step 3b: Define survey logic for this question

After the question is approved, ask:

"Does this question have any survey logic?"

Specifically ask about:

  • Skip patterns: Do any answers skip to a later question (bypassing questions in between)?
  • Branching (conditional display): Do any answers trigger display of conditional questions?
  • Terminations: Do any answers disqualify the respondent and end the survey?
  • Randomization: Should answer options be randomized across respondents?

Surface the logic in this format:

| Routing Logic:
| IF Q[n] = [answer] → Continue to Q[m]
| IF Q[n] = [answer] → Skip to Q[m]
| IF Q[n] = [answer] → Flag for [block/segment name]
| IF Q[n] = [answer] → Terminate survey
| ALL respondents → Continue to Q[m]

If there are conditional questions that only show based on this question's answers, document them with the arrow notation:

**→ Only shown if Q[n] = [answer]**

Q[m]: [Question that is conditionally displayed]

Randomization notes (if applicable):

| Randomization:
| Randomize answer options [list which ones]
| Keep [specific options] in fixed positions (e.g., "Other" always last)

If terminations are included in the routing logic, ask the researcher if they need termination message text:

After documenting a termination in the routing logic, ask: "Would you like me to draft a termination message for [condition]?"

Only draft termination messages if explicitly requested.

Example of complete survey logic documentation:

| Routing Logic:
| IF Q2 = "Yes, at least one is between 6 and 17" → Continue to Q2a
| IF Q2 = "Yes, but none are 6-17" → Skip Q2a, continue to Q3
| IF Q2 = "No" → Skip Q2a, continue to Q3

**→ Only shown if Q2 = "Yes, at least one is between 6 and 17"**

Q2a: Age Brackets of Children
[Multi-select question would appear here]

Optional: Include rationale and implementation notes

After documenting the routing logic, you may optionally include rationale boxes (explaining why this logic was chosen) or implementation notes (technical details for survey tool setup):

| Rationale:
| [Explanation of why this routing logic makes sense for the research goals]
| [Why certain answers lead to certain paths]
| [What this logic accomplishes]

| Implementation Notes:
| [Technical details for survey platform setup]
| [Validation requirements, piped text, display logic specifics]
| [Mobile considerations or platform-specific notes]

Example with rationale and implementation notes:

| Routing Logic:
| IF Q1 = "I live with a partner or spouse" → Flag for JA block (partner path)
| IF Q1 = "I live with housemates" → Flag for JA block (housemate path)
| ALL respondents → Continue to Q1a

| Rationale:
| Separating living situation from relationship status allows us to capture:
| (1) non-cohabiting couples (long-distance, planning to move in),
| (2) housemates who share expenses, and
| (3) people who live with family but have a partner elsewhere.
| All are potential JA candidates with different needs.

| Implementation Notes:
| Use Qualtrics display logic to set embedded data field "JA_path" = "partner" or "housemate"
| This field will control conditional text throughout the JA block

Note: If termination logic is included, ask the researcher if they want termination message text drafted. Don't automatically include messages unless requested.

Wait for confirmation of survey logic before moving to the next question.


Surface one question (Step 3a). Wait for approval. Then define its survey logic (Step 3b). Wait for approval. Then surface the next question.

Do not batch questions unless explicitly told to.

Note: Survey logic defined here affects the respondent's experience (what they see, where they go). Segmentation logic (who qualifies, how to prioritize) comes in Stage 5 after all questions are approved.

Question rules (apply to every question):

0. Use conditional text/piping when needed

When a question needs to adapt its wording based on previous answers, use conditional text (also called "piping" in survey tools).

Notation for conditional text:
Use square brackets with slash-separated alternatives to show where text varies:

*Do you currently share expenses with your [partner / housemate(s)]?*

This means:

  • If respondent indicated "partner" path earlier → "Do you currently share expenses with your partner?"
  • If respondent indicated "housemate" path earlier → "Do you currently share expenses with your housemate(s)?"

When to use conditional text:

  • Questions that need to reference the respondent's specific situation
  • Questions shown to multiple respondent types who need personalized wording
  • Follow-up questions that reference earlier selections

Document the logic clearly:

**Q5: Current Expense Sharing**

*Do you currently share or split any household expenses (like rent, bills, or groceries) with your [partner / housemate(s)]?*

| Implementation Notes:
| Conditional text throughout this section:
| IF respondent entered via romantic partner path (Q1 = partner/spouse, or Q1a = in a relationship)
|   → use "your partner"
| IF respondent entered via housemate path (Q1 = housemates)
|   → use "your housemate(s)"
| Implementation: Use Qualtrics piped text or display logic to swap wording.

Example with multiple conditional elements:

*How does [your child / your children] currently receive and spend money?*
*(If you have multiple children aged 6-17, think about the one who is most likely to need their own card soon.)*

| Implementation Notes:
| Use singular "your child" if Q2a = only one age bracket selected
| Use plural "your children" if Q2a = multiple age brackets selected
| Show instruction text in parentheses only if multiple age brackets selected

1. Limit answer options to 5-6 choices

To reduce cognitive load and prevent overwhelming respondents, keep answer options to a maximum of 5-6 choices (including catchall options like "Other" or "None of the above").

Why: Long lists of options increase dropout, reduce data quality, and make it harder for respondents to process their choices. If you need to ask about many specific options (e.g., a long list of software products), either:

  • Group them into broader categories
  • Ask the researcher which specific options matter most
  • Use a more targeted question that doesn't require listing everything

Exception: Multi-select questions where "select all that apply" is appropriate can have slightly more options (6-8), but still avoid excessive lists.

Bad (too many options):

Which accounting software do you use?

  • Xero
  • QuickBooks Online
  • QuickBooks Desktop
  • FreeAgent
  • Sage
  • Wave
  • Zoho Books
  • NetSuite
  • Kashoo
  • Other
  • None
    (11 options = cognitive overload)

Good (grouped or selective):

Which accounting software do you use?

  • Xero
  • QuickBooks (any version)
  • Other cloud-based accounting software (please specify)
  • I don't use accounting software
    (4 options + text entry)

Or, if specific products matter, ask the researcher: "Which accounting software products are most important to differentiate? I can focus the options on those."

2. Avoid yes/no questions

Yes/no questions are easy to game. Participants have a 50% chance of qualifying by guessing.

Bad:

Do you use mobile banking apps?

  • Yes (accept)
  • No (disqualify)

Good:

How often do you use mobile banking apps?

  • Daily
  • Weekly
  • A few times a month
  • Rarely
  • Never

3. Focus on past behavior, not predictions

Users are terrible at predicting what they'll do. Ask what they've done.

Bad:

Would you use a tool that helps you track business expenses?

  • Yes
  • No

Good:

How do you currently track business expenses? (select all that apply)

  • Spreadsheet
  • Accounting software
  • Paper receipts
  • I don't track expenses

4. Avoid leading questions

Don't reveal what answer you want or hint at the study purpose.

Bad:

Do you find it frustrating when mobile apps don't let you save your progress?

  • Yes (accept)
  • No (disqualify)

Good:

In the past month, have you stopped using a mobile app before finishing a task?

  • Yes, multiple times (accept)
  • Yes, once or twice (accept)
  • No (proceed)
  • I don't use mobile apps (disqualify)

5. Provide reference frames for time and context

Make sure all respondents are answering about the same timeframe and context. Don't let people guess.

Bad (vague timeframe):

How often do you feel frustrated with your banking app?

  • (People might think about today, this week, this year, or their entire life)

Good (specific timeframe):

In the past month, how often have you felt frustrated with your banking app?

  • Daily
  • A few times per week
  • Once or twice
  • Never

Bad (vague context):

How good is the economy?

  • (National? Local? Their personal economy?)

Good (specific context):

How would you describe the economy in your local area right now?

Reference frame checklist:

  • Frequency questions → specify timeframe ("in the past week/month/year")
  • Experience questions → specify recency ("last time you used", "most recent experience")
  • Opinion questions → specify scope ("national economy" vs "your local area" vs "your household")

6. Use clear, simple language

Avoid jargon, acronyms, and complex phrasing. Write for the least tech-savvy participant.

Bad:

Do you leverage CI/CD pipelines for your deployment workflows?

Good:

Do you use automated tools to release software updates?

  • Yes, regularly (accept)
  • Yes, occasionally (accept)
  • No (proceed)
  • I don't know what this means (disqualify)

7. Provide clear answer options and make scales ordinal

Use specific options, not vague terms like "regularly" or "a lot." If using a rating scale, ensure each point is clearly higher or lower than the others for all respondents.

Bad (vague terms):

How often do you commute?

  • Regularly (???)
  • Sometimes (???)
  • Rarely (???)

Good (specific terms):

How many days per week do you commute to an office or workplace?

  • 5+ days per week
  • 3–4 days per week
  • 1–2 days per week
  • I don't commute

Bad (ambiguous ordering):

How many job openings are available in your area?

  • Many
  • A lot (Is this more or less than "many"?)
  • Some
  • A few

Good (clear ordering):

How many job openings are available in your area?

  • A lot
  • Some
  • Only a few
  • None at all

For questions with numeric ranges (frequency, volume, counts):

Always check that ranges are appropriate for the target population before finalizing. If you don't know the distribution, ask the researcher:

  • "What's the typical range for [metric] among your users?"
  • "Are most users at the low end, high end, or distributed across?"
  • "What threshold would differentiate high vs low [metric]?"

Bad (ranges may not match reality):

How many employees does your company have?

  • 1-10
  • 11-50
  • 51-100
  • 100+
    (If you're recruiting SMBs and most have < 10 employees, this doesn't differentiate well)

Good (check first, then set ranges):
Ask researcher: "What's the typical employee count for your target SMBs?"
If answer is "mostly 1-20", then use ranges like:

  • Just me (solo)
  • 2-5 employees
  • 6-10 employees
  • 11-20 employees
  • More than 20 employees

Rule: Numeric ranges should distribute your target population across options, not concentrate them all in one bucket.

8. Always include a catchall option

Provide "Other," "None of the above," or "I don't know" to avoid forcing participants to pick an answer that doesn't fit.

Bad:

Which of these payment apps do you use?

  • PayPal
  • Venmo
  • Cash App

Good:

Which of these payment apps do you use? (select all that apply)

  • PayPal
  • Venmo
  • Cash App
  • Other
  • None of the above

9. Account for response order effects (primacy vs recency)

The order you present answer options can bias which ones get selected. The effect depends on survey mode.

Primacy effect (paper surveys, web surveys):

  • Respondents tend to pick options listed first
  • They see all options at once and gravitate toward top choices

Recency effect (phone surveys, read-aloud questions):

  • Respondents tend to pick options they heard most recently
  • They're holding options in memory and the last one is freshest

Solutions:

  • Randomize or rotate answer options when there's no natural order (e.g., list of product names, list of reasons)
  • Keep ordered scales in order (e.g., "Daily, Weekly, Monthly, Never" should not be randomized)
  • If randomization isn't possible, be aware that response distributions may be skewed

Example where randomization helps:

Which of these features would be most useful to you?

  • [Randomize: Feature A, Feature B, Feature C, Feature D]
  • None of the above (always last)

10. Ask sensitive questions (including demographics) near the end

Questions about personal topics (income, health conditions, political views) or demographics can feel intrusive early in a screener. Asking them too soon increases dropout.

Put these near the end:

  • Demographics (age, gender, income, education, marital status)
  • Sensitive personal information (health, finances, relationships)
  • Political or religious views

Why: Participants are more willing to answer personal questions after they've already invested time in the screener. Starting with these questions signals "this is invasive" and increases abandonment.

Exception: If a demographic is a hard disqualifier (e.g., must be 18+, must live in specific city for in-person study), ask it early to avoid wasting people's time.

11. Screen for industry insiders (if needed)

Exclude people who participate in user research frequently — they behave differently.

Example:

When was the last time you participated in a user research study (interview, usability test, focus group, etc.)?

  • In the past month (disqualify — professional tester)
  • 1–6 months ago (disqualify)
  • More than 6 months ago (accept)
  • Never (accept)

12. Include an articulation question

Screen out uncommunicative participants and professional testers by including at least one open-ended question that requires a thoughtful response.

A participant who provides one-word answers or vague responses ("it was fine", "good product") will likely be difficult to interview. Articulation questions help you identify people who can express their thoughts clearly and in detail.

Example:

Please describe the last time you used [product/service] and what you were trying to accomplish.

What you're looking for:

  • Specific details (not vague generalities)
  • Clear communication
  • Depth of thought
  • Authentic experience (not someone trying to game the screener)

Red flags:

  • One-word or one-sentence answers
  • Vague responses ("it was good", "I use it a lot")
  • Copy-pasted responses (if you see identical wording across multiple respondents)
  • Responses that don't actually answer the question

13. Ask about NDA willingness and deal-breakers early

If your study has deal-breakers (NDA requirements, travel requirements, time commitments, comfort discussing sensitive topics), screen for them upfront. Don't waste the participant's time only to disqualify them at the end.

Example (NDA):

Are you willing and able to sign a non-disclosure agreement (NDA) before the session?

  • Yes (accept)
  • No (disqualify)
  • I'm not sure (disqualify)

Example (sensitive topics):

This study will require you to share openly about your experience with [health condition / financial situation / other sensitive topic]. Are you comfortable discussing this topic in detail?

  • Yes, I'm comfortable (accept)
  • No, I'm not comfortable (disqualify)
  • I'm not sure (disqualify)

14. Use multi-stage filtering for complex criteria

For difficult-to-screen behaviors, use a funnel of 2–3 questions that build on each other.

Example (screening for Amazon Prime churners):

Q1:

Which of the following online retailers have you purchased from in the past 12 months? (select all that apply)

  • Amazon
  • eBay
  • Walmart
  • Target
  • Other
  • None

→ If Amazon not selected, disqualify.

Q2:

Have you ever had an Amazon Prime subscription?

  • Yes, I currently have one (disqualify)
  • Yes, but I canceled it (accept → Q3)
  • No, never (disqualify)

Q3:

When did you cancel your Amazon Prime subscription?

  • In the past 6 months (high priority)
  • 6–12 months ago (medium priority)
  • More than 12 months ago (low priority)

Check-in 3 → 4: After each question and its survey logic are approved, move to the next question. When all questions and their survey logic are approved, move to Stage 4.


Stage 4: Assemble screener

Assemble the final screener ready for deployment, including all survey logic defined in Stage 3.

Include:

  1. Introduction that manages participant expectations:
    • Clarify this is a screener, not the paid study itself
    • State approximate time to complete (keep under 10 minutes)
    • Explain what happens next if they qualify
    • Mention any deal-breakers upfront (NDA, travel, time commitment)
    • NOT reveal company name, product name, or specific study purpose

Example introduction:

Thank you for your interest in participating in this research study about [generic topic]. This short survey will help us determine if you're a good fit. It should take about 5 minutes to complete.

If you qualify, we'll contact you within [timeframe] to schedule a [interview/usability test/focus group] session. The session will take approximately [duration] and you'll receive [incentive amount] for your time.

Please note: This is a screening survey only. You will not be compensated for completing this survey, but qualified participants who complete the full study will be paid.

  1. Complete question set — all approved questions in order, numbered

  2. Complete survey logic summary — consolidated view of all branching, skip patterns, terminations, and randomization defined in Stage 3

Survey logic summary format:

Present a comprehensive routing overview that shows:

  • Which respondents see which sections/blocks
  • All skip patterns and branches
  • Conditional question display rules
  • Termination conditions
  • Any block randomization

Format option 1: Routing overview table by respondent type

| Respondent Profile | Screening Results | Sections Shown | Logic Notes |
|---|---|---|---|
| [Profile description] | Q[n] = [answer] | Section A + Section B | [Any special routing] |
| [Profile description] | Q[n] = [answer], Q[m] = [answer] | Section A only | [Any special routing] |

Format option 2: Question-by-question logic list

**Complete Survey Logic:**

Q[n]: [Question topic]
| Routing Logic:
| IF Q[n] = [answer] → [action]
| IF Q[n] = [answer] → [action]

**→ Only shown if Q[n] = [answer]**
Q[m]: [Conditional question]

Q[x]: [Next question]
| Routing Logic:
| [etc.]

**Termination conditions:**
- IF Q[n] = [answer] → Terminate with message: "[message]"
- IF Q[m] = [answer] → Terminate with message: "[message]"

**Block randomization:**
- [Description of any section/block randomization]

**Answer randomization:**
- Q[n]: Randomize options [list], keep [option] fixed
- Q[m]: Randomize options [list], keep [option] fixed

Format option 3: Visual flow diagram

For complex routing, create a visual survey flow:

START
  ↓
Q1: [Topic]
| IF Q1 = [answer] → Flag for [block]
  ↓
Q2: [Topic]
  │
  ├─→ IF Q2 = [answer] ────────────┐
  │                                 │
  ├─→ IF Q2 = [answer]              │
  │       ↓                         │
  │   Q2a: [Conditional question]  │
  │       ↓                         │
  └─────────┴─────────────────────┘
  ↓
Q3: [Topic - all paths converge]
  ↓
[etc.]

Choose the format that best communicates the survey's routing logic. Complex surveys may benefit from combining formats (e.g., respondent type table + visual flow diagram).

  1. Estimated completion time

  2. Suggested incentive amount (if researcher provided budget context)

Output format:

If the researcher specified a survey tool (Typeform, Qualtrics, Google Forms, etc.), format questions for that tool.

UI best practices for web surveys:

  • Use radio buttons, not dropdown menus — radio buttons show all options at once, reducing cognitive load and making it clearer what choices are available
  • Use checkboxes for multi-select — "select all that apply" questions
  • Randomize/rotate options when appropriate — reduces primacy bias (people picking first option)
  • Show progress indicator — helps reduce dropout
  • Mobile-friendly layout — many respondents will complete on phones

Consider using validated questions when available:
If you're measuring a common attitude, behavior, or concept (e.g., tech savviness, privacy concern, job satisfaction), check if a validated measure already exists. Benefits:

  • Reliability and validity already tested
  • Allows comparison across studies or benchmarking
  • Often better worded than what you'd create from scratch

Common sources for validated screener questions include academic research, industry benchmarks, and established survey instruments. However, most UX research screeners will need custom questions tailored to your specific product and research goals.

Check-in 4 → 5: Researcher does final review of the screener and complete survey logic. Common corrections:

  • "Question 3 and Question 7 are redundant — drop one"
  • "Add a warmup question at the start — it's too abrupt"
  • "Estimated time is wrong — this will take 8 minutes, not 4"
  • "Change the introduction to mention [X]"
  • "The skip logic from Q2 to Q5 should actually go to Q6"

Once the screener and survey logic are approved, move to Stage 5: Segmentation logic.


Stage 5: Segmentation logic

Now that the screener questions are finalized and approved, define the logic for analyzing responses. Build this step by step with researcher approval at each substage.

Substage 5a: Define target profile

First, identify who the ideal participants are — the baseline criteria everyone must meet.

Build the target profile criterion by criterion. Present each criterion individually for approval before adding the next.

For each criterion, surface:

**Target criterion #[n]:**
- Condition: [Q# = answer]
- Rationale: [why this matters for inclusion]

Wait for approval, then present the next criterion.

After all criteria are approved, surface the complete target profile:

**Target profile (ideal participants):**
- Criteria: [All approved criteria combined with AND]
- Rationale: [overall why this is the target]

Check-in 5a → 5b: Once complete target profile is confirmed, proceed to segments.


Substage 5b: Define segments

Next, define how target participants will be divided into comparison groups (if needed).

For each segment, build definition criterion by criterion:

  1. Present segment name and purpose
  2. Present each defining criterion individually for approval
  3. Present target quota
  4. Present rationale
  5. Wait for approval of complete segment before proceeding to next segment

For each segment criterion, surface:

**Segment [n]: [name]**

**Defining criterion #1:**
- Condition: [Q# = answer]
- Why this criterion: [rationale]

Wait for approval, then present next criterion if needed.

After all criteria approved, surface complete segment:

**Segment [n]: [name]**
- Definition: [All approved criteria combined]
- Target quota: [n participants]
- Rationale: [why this segment matters to research goals]

Check-in 5b → 5c: After all segments are defined and approved, proceed to prioritization.


Substage 5c: Define prioritization criteria

Now define how to rank participants within each segment. By default, only define "high priority" — everyone else is automatically lower priority. Only create medium/low tiers if the researcher explicitly requests more granular prioritization.

Build high priority definition criterion by criterion:

  1. Present each criterion individually for approval
  2. After all criteria approved, surface complete high priority definition
  3. Ask if additional priority tiers (medium/low) are needed

For each high priority criterion, surface:

**High priority criterion #[n]:**
- Condition: [Q# = answer]
- Why this matters: [rationale]

After all criteria approved, surface complete high priority:

**High priority:**
- Conditions: [All approved criteria combined]
- Rationale: [overall why these are highest priority]

Then ask: "Do you need medium and low priority tiers defined, or is high priority sufficient?"

  • If researcher says high priority is sufficient → proceed to 5d
  • If researcher requests additional tiers → define medium priority, then low priority, using same criterion-by-criterion approach

Check-in 5c → 5d: Wait for confirmation that prioritization logic is correct before proceeding.


Substage 5d: Define exclusions and deprioritizations

Finally, identify non-target profiles — people whose responses will be collected but shouldn't be recruited (or should be deprioritized).

Surface each profile one at a time:

**Profile A: [name, e.g., "Already connected"]**
- Definition: [Q# = answer]
- Action: Exclude (different research population)
- Rationale: [why exclude]

Then:

**Profile B: [name, e.g., "Professional tester"]**
- Definition: [Q# = answer]
- Action: Exclude (biases research)
- Rationale: [why exclude]

Continue for all exclusion/deprioritization profiles.

Check-in 5d → 5e: After all profiles defined and approved, proceed to final notes.


Substage 5e: Additional analysis notes

Add any guidance for manual review, tiebreakers, or edge cases.

Surface:

**Additional notes for analysis:**
- [Guidance on evaluating open-ended responses, e.g., "Review Q7 for articulation quality"]
- [Tiebreaker rules, e.g., "If both segments at quota, prioritize higher transaction volume"]
- [Edge case handling, e.g., "Q4 'Other' responses: review and assign to segment manually"]

Check-in 5e → 6 or handoff: Final review of complete segmentation logic. Once approved, ask researcher: "Would you like me to generate a Qualtrics export, or proceed to segmentation script generation?"

  • If researcher wants Qualtrics export → proceed to Stage 6
  • If researcher wants segmentation script → proceed to Stage 7
  • If neither needed → hand off to screener-segmentation skill or recruitment team

Why staged substages: Segmentation logic is complex. Breaking it into approval steps (target → segments → prioritization → exclusions → notes) catches errors early and ensures the researcher agrees with each decision before building on it.


Stage 6: Qualtrics Export (Optional)

If the researcher uses Qualtrics and wants an optimized format for upload, generate Qualtrics-specific documentation and survey flow export.

Step 1: Ask for preferences

  • Do you want a QSF file (Qualtrics Survey Format - can be imported directly)?
  • Or documentation for manual setup in Qualtrics?

Step 2: Generate Qualtrics export

Option A: Qualtrics Survey Flow Documentation

Provide a structured guide for setting up the screener in Qualtrics, including:

  1. Survey Flow Architecture

    • Block structure (screening block, trust measurement block, demographics block, etc.)
    • Branch logic rules
    • Embedded data fields to set
    • Randomization settings
    • End of survey customization
  2. Question-by-Question Setup

    • Question type (MC - Single Answer, MC - Multiple Answer, Text Entry, Matrix Table, etc.)
    • Question text with formatting
    • Answer choices with coding values
    • Display logic
    • Validation requirements
    • Piped text specifications
  3. Survey Flow Logic

    • Branch conditions in Qualtrics syntax (e.g., "If Q1 Selected Choices = 5, Then Branch to Block: Terminate")
    • Embedded data to capture (segment, priority flags, etc.)
    • Skip logic between questions
    • Randomization of blocks or questions
  4. Termination Messages

    • End of Survey messages for each termination path
    • Redirect URLs if needed
  5. Quotas Setup (if applicable)

    • Quota logic for balanced segment collection
    • Cross-quota conditions

Example Qualtrics Survey Flow Documentation Structure:

## Qualtrics Survey Flow Setup

### Survey Flow Architecture

1. **Set Embedded Data**
   - segment (empty)
   - trust_service_security (empty)
   - trust_pricing_regulation (empty)
   - trust_social_proof (empty)
   - qualified (empty)

2. **Block: Screening Questions**
   - Branch logic after block:
     - If Q1 = 5 (zero interest), Branch to "Terminate - Not Qualified"

3. **Block: Trust Measurement**
   - Randomize question order
   - All respondents see this block

4. **Block: Demographics** (optional)

5. **End of Survey**

---

### Question Setup

**Q1: International Transfer Interest**

- Question Type: Multiple Choice - Single Answer
- Question Text: "Have you ever used or considered using international money transfer services?"
- Choices:
  1. Yes, I currently use them regularly (code: 1)
  2. Yes, I've used them occasionally (code: 2)
  3. Yes, I've used them once or twice (code: 3)
  4. I've considered but never used them (code: 4)
  5. No, I have no interest in international transfers (code: 5) [TERMINATE]

- Display Logic: None
- Validation: Force Response

- Survey Flow Branch After Q1:
  - If Q1 = 5, Branch to Block "Terminate - Not Qualified"
  - Else, Continue to next question

---

**Q2: Trust Factor - Customer Support**

- Question Type: Multiple Choice - Single Answer
- Question Text: "How important is reliable customer support when choosing a money transfer service?"
- Choices:
  1. Extremely important (code: 5)
  2. Very important (code: 4)
  3. Moderately important (code: 3)
  4. Slightly important (code: 2)
  5. Not at all important (code: 1)

- Display Logic: None
- Validation: Force Response

[Continue for each trust measurement question...]

---

### Termination Messages

**Terminate - Not Qualified**
- Message: "Thank you for your interest. Unfortunately, you don't meet the criteria for this study. We appreciate your time."
- End Survey: Yes

Option B: QSF File Generation

Generate a Qualtrics Survey Format (QSF) JSON file that can be imported directly into Qualtrics. This includes:

  • Complete survey structure
  • All questions with proper Qualtrics question types
  • Survey flow with branches and logic
  • Embedded data fields
  • Answer randomization settings

Note: QSF files are complex JSON structures. For most use cases, Option A (documentation) is more practical and easier to customize. Option B requires precise JSON formatting and may need manual adjustments after import.

Check-in 6 → 7 or handoff: Provide Qualtrics export documentation or file. Researcher imports or sets up in Qualtrics. If segmentation script is needed, proceed to Stage 7. Otherwise, handoff.


Stage 7: Generate Segmentation Script (Optional)

If the researcher wants to automate the segmentation analysis, generate a script that applies the logic defined in Stage 5.

Step 1: Ask for preferences

  • Language preference: Python or R?
  • Input format: CSV file path, column names

Step 2: Generate script

The script should:

  1. Load the survey response data (CSV)
  2. Apply target profile criteria (filter to ideal participants)
  3. Assign respondents to segments based on Stage 5b definitions
  4. Apply prioritization logic from Stage 5c (flag high priority)
  5. Flag exclusions/deprioritizations from Stage 5d (if defined)
  6. Output results with columns: [respondent_id, segment, priority, target_profile_match, exclusion_flags, notes]

Script should include:

  • Clear comments explaining each step
  • Column name mappings (researcher provides actual column names)
  • Defensive coding (handle missing values, unexpected responses)
  • Summary statistics (how many in each segment, priority distribution)
  • Export to CSV with all segmentation fields added

Example Python script structure:

import pandas as pd

# Load data
df = pd.read_csv('screener_responses.csv')

# Map column names (researcher provides these)
Q1_col = 'international_transfer_experience'
Q2_col = 'service_security_ranking'
# ... etc

# Apply target profile criteria
df['target_profile'] = df[Q1_col] != 5  # Not "No interest"

# Calculate trust dimension scores
# Service & Security: Check if scam protection ranked as 1
df['service_security_high'] = df[Q2_col + '_scam_protection'] == 1

# Pricing & Regulation: Average of Q3, Q4, Q5
df['pricing_regulation_score'] = df[[Q3_col, Q4_col, Q5_col]].mean(axis=1)

# Social Proof: Average of Q6, Q7, Q8, Q9
df['social_proof_score'] = df[[Q6_col, Q7_col, Q8_col, Q9_col]].mean(axis=1)

# Assign segments
def assign_segment(row):
    if not row['target_profile']:
        return 'Excluded - No Interest'
    
    # High Trust Seekers
    if (row['service_security_high'] and 
        row['pricing_regulation_score'] >= 4.0 and 
        row['social_proof_score'] >= 4.0):
        return 'High Trust Seekers'
    
    # Low Trust Skeptics
    elif (not row['service_security_high'] and 
          row['pricing_regulation_score'] <= 3.0 and 
          row['social_proof_score'] <= 3.0):
        return 'Low Trust Skeptics'
    
    # Security Skeptics
    elif (not row['service_security_high'] and 
          row['pricing_regulation_score'] >= 4.0 and 
          row['social_proof_score'] <= 3.0):
        return 'Security Skeptics'
    
    else:
        return 'Unclassifiable'

df['segment'] = df.apply(assign_segment, axis=1)

# Apply prioritization (high priority = clear classification + harder-to-reach segments)
df['high_priority'] = (
    df['segment'].isin(['Low Trust Skeptics', 'Security Skeptics']) &
    (df['segment'] != 'Unclassifiable')
)

# Export
df.to_csv('screener_segmented.csv', index=False)
print(df['segment'].value_counts())
print(f"\nHigh priority: {df['high_priority'].sum()} respondents")

Check-in 7 → handoff: Provide script with instructions for running it. Researcher tests with their data.


Quality checks (apply throughout)

Over-screening. If your qualification logic is so strict that fewer than 5% of respondents will qualify, flag it. You'll struggle to recruit. Ask the researcher which criteria can be relaxed.

Under-screening. If your qualification logic is so loose that 80%+ of respondents will qualify, flag it. You're not differentiating enough. Ask the researcher what's missing.

Reveals study purpose. If a question combination makes the study purpose obvious, flag it. Participants who guess the purpose may modify behavior to "help" you.

Jargon check. For every technical term, acronym, or industry-specific phrase, ask: "Would my least tech-savvy target user understand this?" If no, rewrite.

Bias check. For every question, ask: "Does this wording favor a particular answer?" If yes, rewrite.

Demographic redundancy. If demographic data is already available from the recruitment panel or user database, don't ask for it again. Exception: when accuracy is critical (e.g., exact age for age-restricted products).

Professional tester check. Always screen for research participation frequency unless explicitly told not to.

Articulation check. Always include at least one open-ended question to screen for communication ability. Participants who can't articulate thoughts in a screener won't suddenly become articulate in your study.

Expectation management. Make sure participants know this is a screener (not the paid study itself), what happens if they qualify, and any deal-breakers (NDA, travel, time commitment) upfront.

Pre-test your screener. Before launching to your full audience, test with 3–5 people (colleagues, friends, or a small pilot group). Have them think aloud as they answer. Ask "What does that question mean to you?" Watch for:

  • Questions they find confusing or ambiguous
  • Answer options that don't fit their situation
  • Times when they're guessing what you want to hear
  • Technical terms or jargon they don't understand
  • Questions that take too long to answer

Pre-testing catches problems that are invisible to you as the author but obvious to respondents.

Cognitive load and fatigue checks. Screeners fail when respondents stop reading and start clicking. Apply these guardrails:

  1. No loops. Never repeat the same set of questions. "Pick 3 products and answer 10 questions about each" causes engagement to drop 20% by the third iteration and "don't know" responses to jump 50%.

  2. Hard questions first, easy questions last. Respondents are sharpest at the start. Put important behavioral/segmentation questions early. Save demographics and simple questions for the end.

  3. No back-to-back multi-selects. Multiple "select all that apply" questions in a row compound fatigue. Scanning 10+ options and making comparative judgments repeatedly is where straight-lining starts. Break them up or cut the weakest one.

  4. Single-choice questions: 3-5 options (low cognitive load). These cause minimal fatigue.

  5. Multi-select questions: 5-7 options max (medium cognitive load). These cause moderate fatigue. Limit to 1-2 per screener if possible.

  6. Matrix/grid questions: avoid entirely (high cognitive load). These cause high fatigue. If you must use one, limit to 5-7 rows max.

  7. Open-ended questions: 1-2 maximum (high cognitive load). Short open-ended questions cause high fatigue. Detailed ones cause severe fatigue. Use sparingly—ideally just one articulation question.

  8. Randomize option order in multi-selects. People gravitate to the first option (primacy effect). Randomizing across respondents reduces bias and prevents patterned responding.

  9. Use skip logic. Every irrelevant question damages motivation. If a question doesn't apply to someone, don't show it to them.

  10. Explain why the screener matters. A clear introduction explaining the study's purpose raises completion rates. Respondents who understand why they're answering work harder.

Cognitive load hierarchy (from least to most taxing):

  • Lowest: Single-choice (3-5 options), Likert scales
  • Medium: Multi-select (select all that apply)
  • High: Matrix/grids, ranking tasks, open-ended questions
  • Highest: Detailed open-ended questions, repeated loops

Rule of thumb: A well-designed 10-question screener with low cognitive load outperforms a 5-question screener with high cognitive load. Fatigue is driven by effort, not length.


Common screener types

1. Behavioral segmentation (most common)

Screens based on actions, usage patterns, purchase behaviors, or habits.

Use when: You need users who have done (or not done) specific things.

Example criteria:

  • Purchased in past 30 days
  • Uses product weekly
  • Switched from competitor in past year
  • Has never used feature X

2. Demographic segmentation

Screens based on age, location, role, income, education, household composition, etc.

Use when: Demographics directly affect the research question (e.g., testing a retirement planning tool for people 55+).

Caution: Demographics are usually less important than behavior. Don't over-index on demographics unless they matter.

3. Psychographic segmentation

Screens based on attitudes, values, interests, beliefs, or motivations.

Use when: You need users with specific mindsets or priorities (e.g., "people who value sustainability over convenience").

Caution: Harder to screen for reliably. People may answer aspirationally rather than honestly.

Technique: Ask about past behavior that reveals the attitude, rather than asking about the attitude directly.

Bad (aspirational):

Do you care about sustainability when shopping?

Good (behavioral proxy):

In the past month, have you chosen a product specifically because it was labeled eco-friendly or sustainable?

4. Firmographic segmentation (B2B)

Screens based on company size, industry, revenue, job title, decision-making authority.

Use when: Researching B2B products, enterprise software, or workplace tools.

Example criteria:

  • Works at company with 50–200 employees
  • Job title includes "manager" or above
  • Has budget authority for software purchases
  • Industry = healthcare or finance

Multi-segment screeners

If you need to compare across segments (e.g., churned vs active users, or SMB vs enterprise), design the screener to:

  1. Qualify all segments with a shared baseline. Everyone must meet minimum criteria.
  2. Route to segment-specific questions. Use branching logic so each segment answers relevant follow-ups.
  3. Set quotas per segment. Track how many qualified participants you have in each segment and stop recruiting when quotas are met.

Example structure:

Q1–3: Baseline qualification (everyone answers)
Q4: Segment routing question
  → If "churned," route to Q5–6 (churn-specific)
  → If "active," route to Q7–8 (active-specific)
Q9–10: Final questions (everyone answers)

Sample question library by criteria type

Use these examples as inspiration, not copy-paste templates. Every question should be tailored to your specific research goals and audience.

Industry or occupation (B2B/firmographic)

Ask employment questions when you need people familiar with an industry or want to exclude competitors.

Example:

What industry do you currently work in?

  • Retail
  • Healthcare
  • Financial services
  • Education
  • Technology / IT
  • Manufacturing
  • Other
  • I don't currently work / Retired

Example:

Which category best describes your job function?

  • Marketing / advertising
  • Sales
  • Engineering / product development
  • Operations / logistics
  • Finance / accounting
  • Human resources
  • Other

Familiarity with a product or service

Screen for novices, experienced users, or a mix.

Example:

How would you rate your familiarity with [product/service]?

  • Expert — I use it extensively and know it inside-out
  • Experienced — I use it regularly and understand how it works
  • Intermediate — I've used it a few times, still learning
  • Beginner — I've heard of it but never used it
  • Never heard of it

Example:

Which of these tools do you use for [task]? (Select all that apply)

  • [Tool A]
  • [Tool B]
  • [Tool C]
  • [Tool D]
  • Other
  • None of the above

Frequency of performing specific tasks

Screen for people who regularly do (or don't do) specific things.

Example:

How often do you [perform task]?

  • Daily
  • A few times per week
  • Once a week
  • A few times per month
  • Once a month
  • Less than once a month
  • Never

Example:

When was the last time you [performed action]?

  • In the past week
  • In the past month
  • In the past 6 months
  • In the past year
  • More than a year ago
  • Never

Comfort with sharing personal information

Screen for willingness to discuss sensitive topics.

Example:

This study will require you to discuss your experience with [health condition / financial situation / personal topic]. Are you comfortable sharing openly about this?

  • Yes, I'm comfortable discussing this
  • No, I'm not comfortable
  • I'm not sure

Articulation and communication ability

Screen for participants who can express themselves clearly.

Example (open-ended):

Please describe the last time you [used product / performed task] and what you were trying to accomplish. Include specific details about what happened.

What you're looking for:

  • 3+ sentences with specific details
  • Clear communication
  • Authentic experience (not generic or copy-pasted)

Synchronous (phone) screeners

Most screeners are written surveys. But for high-stakes studies, or when the written screener might reveal too much, add a short phone screener (2–5 minutes) after the written one.

Use a phone screener when:

  • You need to assess communication style (articulate, engaged, verbose)
  • You need to probe ambiguous written answers
  • You want to ask potentially revealing questions conversationally
  • You need to confirm honesty (e.g., "Tell me about the last time you used X")

Phone screener rules:

  • Keep it under 5 minutes — you typically don't compensate for screener time
  • Use it to elaborate on written responses, not to ask new topics
  • Listen for red flags: vague answers, contradictions, disengagement

What good looks like

Example 1: Complete question with routing logic documentation

This example shows how to document a question with its routing logic, rationale, and implementation notes in the format used throughout Stage 3:

Question 2: Children / Dependents Screener

Do you have any children or dependents that you are financially responsible for?

Format: Single-select

☐ Yes, and at least one is between the ages of 6 and 17

☐ Yes, but none are between 6 and 17 (all are under 6 or over 18)

☐ No, I don't have children or dependents I'm financially responsible for

| Routing Logic:
| IF Q2 = "Yes, at least one 6-17" → Continue to Q2a, then Young Explorers block (Phase 1)
| IF Q2 = "Yes, but none 6-17" → Skip Q2a, skip Young Explorers block
| IF Q2 = "No" → Skip Q2a, skip Young Explorers block

| Rationale:
| Combines the screener gate and age-bracket detail into a single question for most respondents.
| Only those with 6-17 children proceed to Q2a for detailed age segmentation.
| Inclusive language ("children or dependents") captures guardians, grandparents, etc.
| The 6-17 age gate is embedded directly in response options, eliminating a separate routing step.
| Parents with children across age groups (e.g., a 5-year-old and a 12-year-old) can accurately
| select option 1 ("at least one is between 6 and 17").

→ Only shown if Q2 = "Yes, at least one is between 6 and 17"

Question 2a: Age Brackets of Children (6-17)

Which of the following age groups do your children or dependents fall into? Select all that apply.

Format: Multi-select (checkboxes)

☐ 6-9 years old

☐ 10-13 years old

☐ 14-17 years old

| Rationale:
| Provides developmental-stage segmentation (early primary, pre-teen, teen) without the overhead
| of a number-entry grid. Multi-select handles parents with children across multiple brackets.
| This is sufficient for TAM segmentation. If the team later needs exact child counts per bracket,
| that can be captured in qualitative follow-up.

| Implementation Notes:
| At least one checkbox must be selected (validation).
| Mobile-friendly: checkboxes are easier to tap than number-entry grids.


Example 2: Screener for Wise SMB churn study

Research goal: Understand why UK and US SMBs churn from Wise after initial signup.

Sampling strategy:

  • Inclusion: SMB owners (1–50 employees), used Wise in past 24 months but no longer active, makes international payments at least quarterly
  • Exclusion: Enterprise (50+ employees), never activated Wise account, participated in user research in past 6 months
  • Segments: UK vs US, high-transaction-volume vs low-transaction-volume
  • Quotas: 12 UK, 12 US; 12 high-volume, 12 low-volume

Stage 1 strategy:

Research need What to screen for Question type Rationale
Understand churn reasons Used Wise in past 24 months but stopped Behavioral Need active churners, not never-users
Compare UK vs US Located in UK or US Demographic Research scope
Identify high-value segment International payment frequency Behavioral Volume affects churn reasons
Exclude non-SMBs Company size 1–50 employees Firmographic Research scope
Exclude professional testers Research participation in past 6 months Behavioral Bias risk

Stage 3 example question:

**Question 3:** When did you last use Wise to send money internationally?

**Answer options:**
1. In the past month (disqualify — still active)
2. 1–6 months ago (accept, high priority)
3. 7–12 months ago (accept, medium priority)
4. 13–24 months ago (accept, low priority)
5. More than 24 months ago (disqualify — too long ago)
6. I created an account but never sent money (disqualify — never activated)
7. I've never used Wise (disqualify)

**Qualifying logic:**
- Accept: options 2, 3, 4
- Disqualify: options 1, 5, 6, 7
- Segmentation priority: option 2 = high, option 3 = medium, option 4 = low

**Rationale:** Screens for recent churners (1–24 months) without being obvious about it. Disqualifies still-active users (past month) and distant churners (24+ months). Avoids yes/no format.

**Best practice check:**
- ✅ Avoids yes/no
- ✅ Focuses on past behavior
- ✅ Uses clear time ranges
- ✅ Includes catchall options
- ✅ Doesn't reveal study purpose (doesn't mention "churn" or "why did you stop")

Handoff to screener-segmentation

After Stage 5 is complete, hand off to screener-segmentation with:

  1. Final screener questions (all questions with answer options from Stage 4)
  2. Segmentation logic (from Stage 5: target profile, segment definitions, prioritization criteria, exclusions)
  3. Target quotas per segment

The segmentation skill will take survey responses (CSV or similar) and produce a prioritized interview list with confidence ratings.


When to stop and ask

  • No research plan provided — ask for it. Don't infer research goals.
  • No sampling strategy provided — ask for inclusion/exclusion criteria before proceeding.
  • Qualification logic too strict — flag that fewer than 5% will qualify and ask which criteria can be relaxed.
  • Conflicting criteria — if inclusion criteria contradict each other (e.g., "must use product daily" AND "churned 6+ months ago"), flag and ask which takes priority.
  • Researcher asks to skip stages — confirm explicitly. Stage-skipping should be deliberate.
  • Halfway through Stage 3 you realize the structure is wrong — stop, surface the problem, go back to Stage 2. Don't push forward with a broken foundation.
  • At Stage 5 you realize questions don't capture what's needed for segmentation — stop, flag the gap, ask if questions need to be added before finalizing segmentation logic.

What to avoid

Don't reveal study purpose, company name, or product name. If participants guess what you're researching or who you are, they may modify their answers to qualify or to "help" you. Keep study titles and descriptions generic (e.g., "social media habits study" not "Instagram photo editing study"). Don't mention your company or product name until after they're qualified and scheduled.

Don't write questions that reveal what you're screening for. If participants guess what you're looking for, they'll game the screener. Use the funnel technique and bury revealing questions among neutral options.

Don't use percentages or precise numbers when ranges will do. "1–5 times" is better than "exactly how many times."

Don't ask aspirational questions. "Do you care about X" gets aspirational answers. "When did you last do X" gets honest ones.

Don't force answers. Always provide "Other," "None of the above," or "I don't know" options.

Don't skip the professional tester screen. They behave differently and will bias your research.

Don't skip the articulation screen. A participant who provides vague, one-word responses to an open-ended screener question will do the same in your study.

Don't combine multiple criteria in one question. If you need to know both "uses product" AND "pays for product," ask separately. Compound questions are hard to answer and harder to analyze.

Don't ask demographics you already have. If your recruitment panel already has age, gender, location, don't ask again (unless accuracy is critical).

Don't screen for demographics unless necessary. Psychographics and behaviors are almost always more important than age, gender, or income. Only include demographic criteria if they directly affect your research question or if you need them to ensure a diverse participant pool.

Categories