MaxDiff

Rank a long list of features or messages from best-worst picks, with bootstrapped confidence intervals and an optional anchor check to separate what wins head-to-head from what’s actually essential. This example has been tested and validated with Claude.

When to use this

You have more candidate items than a rating scale can responsibly compare, and you need a ranking built from real trade-offs rather than generous scores. MaxDiff is purely relative- it tells you what beat what, not whether anything matters much on its own. Add the anchor question (step 6) if that distinction matters for your decision.

Versions of this Prompt

• Base MaxDiff: steps 1, 2, 3, 5, 7, 8, 9, 10. Complete analysis on its own.
• Anchored MaxDiff: add step 6 to compare relative ranking against absolute importance ratings and flag hygiene factors and false differentiators.
• Segmented: add step 4 to either version.

For the full walkthrough,  including a side-by-side comparison of two plain-prompt attempts that each caught only one of two anchor mismatches (for two unrelated reasons), the z-score false-positive problem in detail, and all 12-item results, see the full MaxDiff guide.

Prompt

Fill in the setup block with your study details, then send the full prompt as written. Every column name must match your file’s literal header row.

I have best-worst (MaxDiff) survey data on [PRODUCT/SERVICE/CATEGORY].
The items tested were: [LIST YOUR ITEMS]
Each respondent saw [K] items per screen, across [J] screens, with each
item appearing [approximately how many] times total. [DESCRIBE HOW THE
DESIGN WAS GENERATED.]
The data is in [DESCRIBE FORMAT]. Columns are: [LIST YOUR COLUMNS —
use actual header strings, not question labels like "Q7"].

[IF ANCHORED: I also asked respondents to independently classify each
item as "must-have," "nice-to-have," or "not important." Compare
relative MaxDiff importance against this absolute measure, separating
real differentiators (high on both) from hygiene factors (low relative
rank, high absolute rate) and false differentiators (high relative
rank, low absolute rate).]

[IF SEGMENTING: I also want to compare results across [SEGMENT VARIABLE].]

Please:
1. Load the data defensively. Do not treat "None," "NA," or "NULL" as
   missing values — these may be valid category labels. Verify that
   every task shows exactly [K] items with exactly one "best" and one
   "worst," and that the two are never the same item. Confirm every
   item appears the expected number of times. Stop and flag any
   failures rather than modeling around them.
2. Fit a sequential best-worst logit: "best" as a conditional logit
   over the items shown; "worst" as a conditional logit over the
   remaining items with utilities negated, after removing the chosen
   "best." State explicitly which item is the reference (utility fixed
   at 0) — do not let a tool's alphabetical default pick it. Report
   fitted utilities with standard errors, confirm none are degenerate,
   check the gradient at the solution, and confirm a different starting
   point lands in the same place.
3. Calculate each item's preference share as exp(utility) ÷ sum of
   exp(utility) across all items, as a percentage summing to 100%.
   Confirm this is unchanged if you refit with a different reference
   item — if it changes, that is a coding error to fix before
   continuing. Do this in code at full precision, rounding only for
   final display.
4. [SEGMENTING ONLY] Before splitting, check the sample size in each
   segment. Flag any segment under [N, e.g., 30] respondents —
   including zero — as too small to model separately; keep those
   respondents in the overall model from step 2. Refit steps 2-3 for
   every segment that clears the threshold.
5. Resample respondents with replacement and recompute steps 2-3 (and
   step 4 if segmenting) from scratch on each resample, at least
   [N, e.g., 1,000] times. Report a bootstrapped confidence interval
   for each item's preference share (and anchor rate, if anchored).
6. [ANCHORED ONLY] Using the same bootstrap replicates from step 5:
   In each replicate, rank items by preference share and separately
   by anchor (must-have) rate. Use rank position, not a z-score —
   preference share is more right-skewed than a plain rate, so
   z-scoring and subtracting compares magnitude of standing, not
   order. Calculate each item's divergence score as preference-share
   rank minus anchor-rate rank, averaged across all replicates.
   Sort items by |mean divergence|. Find the largest gap between
   consecutive values and set the threshold at its midpoint; divide
   that gap by the median of the other gaps and require it to clear
   3x. Flag an item only if both hold: its confidence interval
   excludes zero AND its divergence magnitude clears the threshold.
   State the direction: worse preference rank + better anchor rank =
   hygiene factor; the reverse = false differentiator.
7. Before finalizing, recompute every number through a second,
   genuinely different implementation — not the same code path run
   twice. Use both absolute and relative tolerance together
   (|a − b| ≤ atol + rtol × |b|) to judge matches. If a number
   doesn't match, report both values, the likely cause, and which
   you trust. For bootstrap distributions, verify a random subsample
   of at least 10 resamples rather than the full set, and say plainly
   it's a subsample check.
8. Present a results table: item, preference share (% with CI),
   anchor rate (% with CI, if anchored), divergence score with CI
   (if anchored), and any flag from step 6.
9. Build a horizontal bar chart sorted by preference share, with a
   second bar for anchor rate if anchored, and flagged items labeled
   directly. Build it only from the step 8 table — do not recompute
   for the chart.
10. Summarize in 2-3 sentences. Be explicit about what the data
    actually supports versus what is directional but not something
    to act on with confidence.

Setup

Code execution must be enabled for step 4’s verification to run. It’s on by default for Team and Enterprise accounts. Free, Pro, and Max users: enable it under Settings > Capabilities.

If your survey data lives in Sprig, skip the export, the Sprig MCP connects live surveys and responses directly into Claude so the analysis runs on current data.

How it works

1. Defensive data loading

   Checks that category labels like “None” or “NA” aren’t silently read as missing values. Verifies every task has exactly one best and one worst pick, and that they’re never the same item.

2. Sequential best-worst logic

   Fits “best” and “worst” picks as a single choice model rather than counting picks independently, using all the information in each task. Reference levels are stated explicitly before estimation — not left to alphabetical default. Convergence is verified from a second starting point.

3. Preference share

   Converts fitted utilities to a preference share that sums to 100%, at full precision. Confirms the share is stable under a different reference item, if it shifts, that’s a coding error to fix before continuing.

4. Segment size check (optional)

   Checks respondent counts before splitting. Flags any segment below your threshold, keeps them in the pooled model, and only refits for segments that clear the bar.

5. Bootstrapped confidence intervals

   Resamples at the respondent level and refits the full model on each resample. Returns a CI for every preference share (and anchor rate, if anchored). In plain-prompt testing, two independent attempts produced correct rankings but no uncertainty on any reported number.

6. Anchor divergence check (anchored version only)

   Compares relative ranking against absolute must-have rates using rank position, not a z-score. (Preference share is more skewed than a plain rate; z-scoring both and subtracting compares how extreme a number looks, not how it ranks, an early pass on the test dataset flagged six of twelve items as mismatched before this fix was applied.) An item is flagged only if its CI excludes zero and its divergence clears the magnitude threshold. Direction is stated: hygiene factor vs. false differentiator.

7. Independent verification

   Every number is recomputed through a genuinely different implementation. Bootstrap distributions are spot-checked across a random subsample of resamples rather than fully re-run, with that caveat stated explicitly in the output.

8. Output

   A verified results table, a bar chart built only from that table, and a 2–3 sentence summary that separates confirmed findings from directional ones.

9. Output (Continued)

   A verified results table, a bar chart built only from that table, and a 2–3 sentence summary that separates confirmed findings from directional ones.

10. Output (Continued)

    A verified results table, a bar chart built only from that table, and a 2–3 sentence summary that separates confirmed findings from directional ones.

Things to check before trusting MaxDiff output

• State your reference item. If you don’t, the tool will pick one alphabetically, and backwards reference levels will silently flip utility signs. Confirm the reference is what you intended.
• Pair relative ranking with an anchor question when it matters. A list of 12 mediocre features produces a confident ranking just as easily as 12 genuinely important ones. Relative rank alone can’t tell the difference.
• Don’t use a z-score to compare two differently-shaped measures. Preference share from a choice model is skewed; a must-have rate usually isn’t. Rank position is the right unit of comparison.
• A CI excluding zero is not the same as a gap worth reporting. At a few hundred respondents, nearly any consistent difference clears that bar. Pair it with a magnitude check.
• Check convergence yourself. Verify gradient norm at the solution and confirm a second starting point lands in the same place. A solver can flag a correct answer as unresolved just as easily as it can wave through a bad one.