Paired Comparison Analysis: A Simple Worked Example

Paired comparison analysis turns "I can't choose" into a ranking you can defend. You compare your options two at a time, give each winner a point, and the totals rank the whole list. Five options means ten quick questions, which fits on one piece of paper.

Below you'll get a blank grid to copy, a fully worked weekend-trip example, and the math on how many comparisons any list needs. I used this method to help my sister pick a wedding venue in fifteen minutes. Her "favorite" came in third, and the one she'd been avoiding won by a mile.

What goes into the score

This article uses a simple win count. Each option earns one point when you choose it over another. A tied comparison gives half a point to each.

You'll also see weighted versions. MindTools' paired comparison walkthrough scores the strength of the preference; CIToolkit's version counts how often each option is chosen. Decide which scoring rule you're using before filling in the table. Mixing them halfway through will give you misleading totals.

For the example below, a narrow win and an easy win both earn one point.

Set up the question

Write a shortlist of options you'd genuinely consider. Give them letters so the table stays readable. Four to eight is a workable starting range for a handwritten grid.

Cross off anything that fails a requirement. For a trip, that could be the available dates or somebody's budget. Don't spend ten minutes comparing a cabin nobody can reach.

Then write the criterion above the table. "Which would leave us most recharged on Monday?" is the one used below. Keep referring to it. If you start choosing by price halfway down a relaxation grid, your totals are answering different questions.

You may decide that several criteria matter. In that case, agree how you'll combine them, or run separate comparisons and inspect the disagreement. A cheap trip and a restful trip can be different choices.

A blank paired comparison grid (copy this)

Put the same option letters across the top and down the side. Fill only the unshaded cells. The diagonal compares each option with itself, and the lower half would repeat pairs.

There is space for six options. For five, ignore F's row and column.

Take the first empty cell in row A. It compares A with B. Write the winner's letter there, then move right to A against C. Continue across the row before starting B against C on the next row. Mark a tie explicitly; an empty cell should mean you haven't answered yet.

Leave room beside the table for a few notes. "Too much driving" tells you more next week than a bare letter D. You don't need a paragraph for every cell, but record a reason when a pair takes some thought.

Worked example with five weekend trips

Imagine a group choosing how to spend a three-day weekend in October. They have already checked that all five choices fit their budget. The destinations and preferences here are illustrative.

• A is a mountain cabin with hiking nearby.
• B is an off-season beach town. Cheap accommodation appeals, though the forecast is chilly.
• A city break gets letter C, with a show penciled in.
• D is wine country, leaving most of the schedule empty apart from a dinner.
• E means staying home.

The question above the grid is "Which would leave us most recharged on Monday?" Five options give ten pairs.

1. A against B. A wins. For this group, sitting by the cabin window sounds more restful than a cold beach.
2. A against C. A again. The city itinerary feels busy.
3. A against D. D wins. They prefer slow mornings to the hike they had planned from the cabin.
4. A against E. A wins. Home would probably turn into catching up on chores.
5. B against C. Tie. Neither stands out for rest, so both get half a point.
6. B against D. D wins. The group favors the quieter schedule.
7. B against E. B wins. They'd like to be away from the house.
8. C against D. D wins. The show requires a late night, which doesn't appeal this weekend.
9. C against E. C wins. Even with the late night, they'd rather take the city break than stay home.
10. D against E. D wins. They prefer the trip.

These judgments belong to the example group. Somebody else could happily put staying home first, especially after a week spent traveling.

Add up every appearance of a letter

Count winners wherever they appear in the grid. D wins in several other options' rows, so adding only across row D would miss most of its score.

All the scores together should equal ten. Each of the ten comparisons hands out exactly one point, including a tie's two halves. If you get 9.5 or 11, check the tally and the blank cells.

D ranks first. A is second, B and C share third, and E is fifth when using competition-style ranking.

Leave B and C tied if you don't need to separate them. If the group agreed in advance to use price for equal scores, check the actual trip totals. Lower accommodation cost alone may disappear once transport is included.

When a pair takes too long

Write "tie" if you have no preference under the stated criterion. You can come back to it after finishing the other cells.

But check for missing information first. If you haven't looked up how long the journey takes, "equally restful" may just mean you don't yet know. Put a note beside that pair and get the information before counting it.

A final-score tie is different. Two options can collect equal points while having a clear winner in their direct matchup. You may use that result as a tiebreaker if you agreed to do so. It won't settle every situation with three or more tied options.

For a low-stakes decision where you remain indifferent, a coin flip between the final two is available. The grid can keep the shared rank.

A cycle can happen

You prefer A to B. Then B to C. And C to A.

Each has one win. All three direct comparisons exist, but they don't give you an order that satisfies every preference. That is a preference cycle.

Recheck whether you used the same criterion throughout. Sometimes you did, and the cycle remains. Keep the tie visible or use a previously agreed secondary criterion. Do not quietly change one result just to produce a tidy podium.

I would rather leave a tied third place on the page than invent a distinction nobody cares about.

How many comparisons you'll need

For n options, a complete grid has n(n − 1) / 2 pairs. Each option has n − 1 opponents. Counting from every option counts A against B and B against A separately, so divide by two.

Adding one option to an existing list of n adds n more pairs. An eleventh idea adds ten comparisons. It deserves a place if it's a real contender; "maybe we could also..." gets expensive in a long handwritten grid.

For a large list, filter against your requirements before comparing. You can also use survey software designed for pairwise data, though sampled comparisons and their scoring need their own explanation. Raw win counts are hard to compare when options have faced different numbers of opponents.

What a bracket leaves unanswered

In a single-elimination bracket, an option drops out after one loss. Starting with n options requires n − 1 decisive matchups to leave one winner, because each matchup removes one option.

The runner-up might be your second choice. Or your second choice might have met the winner in the opening round and disappeared immediately. The bracket never asked enough questions to establish every position.

A full grid gives every option the same set of opponents. Its scores can still tie, and a cycle can still make the ranking awkward. Changing the shortlist can also change the totals.

The tournament and spinner comparison goes into those two approaches. If you're happy with all the remaining options, the Random Decision Maker can simply choose one.

Doing the grid with other people

Agree on the options and criterion together. Then each person can fill in a complete grid privately before comparing results. Give everyone the same scoring rules, including the half point for a tie.

With equal participation, you can sum or average the scores. Keep the individual sheets, though. A combined total can hide that one person strongly dislikes the winning trip.

• Missing answers. Ask for them before treating the totals as comparable.
• Someone changes their mind after learning a fact? Update the relevant comparisons and record why.
• Requirements stay outside the vote. A group score shouldn't override an access need or a spending limit someone already gave you.

For decisions that don't fit comfortably into a grid, how to make decisions when you can't decide covers other ways to narrow a choice.

Further reading

CIToolkit describes the win-count matrix. MindTools uses preference-strength scores, which will produce different totals from the one-point method used here.