McNemar's Test Calculator

Background.

Ask the same 100 people a yes-or-no question before and after something happens, a campaign, a treatment or a training course, and you cannot compare the two percentages with an ordinary two-sample test. The answers come in pairs from the same people, and most people give the same answer both times. The psychologist Quinn McNemar pointed out in 1947 that those unchanged pairs carry no information about whether the proportion shifted. Only the people who changed their answer do.

Lay the pairs out in a 2 × 2 table. Cell a holds pairs that were yes both times and d those that were no both times; b counts pairs that went from yes to no, and c those that went from no to yes. The change in the proportion answering yes is (c − b) divided by the number of pairs, so a and d drop out entirely. If nothing systematic happened, each person who switched was as likely to move one way as the other, and the split between b and c should look like heads and tails from b + c tosses of a fair coin.

This page applies that coin-toss logic directly, which is the exact form of McNemar's test. It finds the binomial probability of a split at least as lopsided as yours, in either direction, and reports it as a two-sided p-value. The better-known chi-square version, (b − c)² ÷ (b + c), approximates the same thing; it works well when there are many discordant pairs and poorly when there are few.

The same table fits other paired designs: two diagnostic tests read on the same patients, or cases matched one-to-one with controls. In each, a small p-value says the discordant pairs lean one way more than chance easily explains. It does not measure how large the effect is, check that the pairing was sound, or show what caused the shift.

What is mcnemar's test calculator?

McNemar's test asks whether a yes-or-no outcome is equally common under two paired conditions, using only the discordant pairs, those in which the two answers differ. Under the null hypothesis, the b pairs that changed one way and the c pairs that changed the other are a fair binomial split of their total. The exact version computes the two-sided probability of a split at least as uneven as the one observed, and that probability is the p-value. Equivalently, it is a sign test applied to the pairs that changed.

How to use this calculator.

1. 01Build the 2 × 2 table of pairs first, so that each pair is counted once. Enter as b the pairs that were positive under the first condition and negative under the second, and as c the pairs that went the other way.
2. 02Leave out the pairs that gave the same answer both times. There is no field for them because they do not affect the p-value.
3. 03Both counts must be whole numbers, at least one must be above zero, and together they can total no more than 10,000.
4. 04Read the exact two-sided p-value with the scope note beside it. Swapping b and c gives the same p-value, so the direction of change comes from which count is larger.

The formula.

p = 2 × P[X ≤ min(b, c)], capped at 1, where X follows a binomial distribution with n = b + c trials and probability 0.5. Written out, P[X ≤ k] is the sum of C(n, i) ÷ 2ⁿ for i from 0 to k, where C(n, i) counts the ways of choosing i of the n discordant pairs. The 0.5 is the null hypothesis: a pair that changed was equally likely to change in either direction. Doubling the lower tail makes the test two-sided, counting splits just as lopsided in the opposite direction; because a binomial with probability 0.5 is symmetric, both tails are the same size. When b and c are equal or nearly so, doubling can take the sum past 1, and the answer is capped there. What drives the p-value is the imbalance between b and c relative to their total: a 4-to-12 split gives 0.0768, while the same 1-to-3 ratio with twice as many pairs, 8 to 24, gives 0.0070.

A worked example.

Frequently asked questions.

References& sources.

1. [1]McNemar Q. Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika. 1947.
2. [2]NIST/SEMATECH e-Handbook, McNemar Test.
3. [3]BIPM, The International System of Units (SI Brochure), 9th ed., version 3.01, coherent derived units and quantity equations.