Don’t Compare the Map to the Almighty
Why it's really, really hard to measure gerrymandering
There are at least 18 ways to measure gerrymandering, most of them developed since 2014. For each metric, there is a wealth of criticism—most of it quite technical/academic/in-the-weeds, and specific to the metric being criticized.1
There are a few non-academic articles explaining why measuring gerrymandering is hard. They’re worth reading—Drutman explains the many facets of “fairness”, and Yglesias and Whitcomb show how a fair-looking map can, by virtue of geography, favor one party.
But metric-makers are sophisticated folks—political scientists, economists, mathematicians. Why has the problem proven intractable?
My goal is to uncover the shared conditions that make measuring gerrymandering hard. I argue it’s hard to measure for three reasons. First, the thing being measured is discontinuous: single-member districts convert tiny vote-share differences around 50% into different outcomes. Second and third, the relevant comparison is counterfactual. To tease apart partisan gerrymandering, a metric needs to know what elections are plausible, and what maps were actually available.
Consider this my grand unified theory of why shit’s hard.
Conditions that make it hard
Discontinuity. A district lost with 49.9% of votes, as a matter of vote distribution, essentially the same as a district won with 50.1%. But it’s a different outcome.
Electoral baseline. Not all statewide vote outcomes are equally likely. Some states are reliably Democratic; others are reliably Republican. A good metric accounts for the actual likelihood of statewide outcomes.
Geographic & legal contingency. What maps are geographically and legally available? A metric that only takes into account votes and seats fails to consider these constraints. That’s lossy. In Joe Biden’s words: don’t compare me to the Almighty, compare me to the alternative.
Points two and three can be combined into something like “counterfactual baseline”—is the metric assuming an unrealistic distribution of votes (geographic contingency), implausible maps (legal contingency), and unlikely statewide outcomes (electoral baseline)? I choose to separate electoral baseline, and group together geographic & legal contingency, but you could of course make other choices.
To see geographic and legal contingency, imagine a hypothetical state, Fractalia, divided into thousands of precincts. In every precinct, Democrats receive exactly 49 votes and Republicans receive 51. If districts are contiguous unions of precincts, then every possible district is 49% Democratic. Democrats receive 49% statewide and win zero seats—not because of a gerrymander, but because no legally available map can do otherwise. If the same voters were arranged differently, it would very much look like a gerrymander. But in Fractalia, it’s the geography, stupid.
This may seem implausible; but consider a different hypothetical state—call it “America”—where one party is overwhelmingly urban, with small pockets of support elsewhere. The urban party—with or without gerrymandering—will produce landslides in the urban districts. For a party, landslides are bad. You’d rather some of those urban folks sprawl out and help you win a few more seats.
Bring out the metrics!
Now consider the 4 most cited ways to measure gerrymandering. This gets a little technical, but the tl;dr is—each of these four very important gerrymandering metrics is injured by these conditions.
Partisan Symmetry: Asks whether the map treats parties equally at the same vote share. If Democrats win 9 of 10 seats with 60% of the statewide vote, would Republicans also win 9 seats with 60%?
Efficiency Gap (EG): Compares each parties “wasted votes”: all votes cast for losing candidates, plus surplus winning votes (every vote over 50%+1). Were Democrats or Republicans more efficiently distributed?
Mean-Median Difference (MMD): Compare a party’s mean and median district vote share. If its mean is significantly higher than its median, the party’s vote is probably concentrated in high-margin districts. You’d typically rather win by slimmer margins.
Declination: Measures the difference in magnitude between losing and winning districts, i.e. asymmetry across the 50% won/lost line. A party that is losing narrowly and winning by a lot is competing on unfavorable terms.
EG and declination inherit the discontinuity. For EG, for instance, a district won by 50.1% is essentially the “perfect district” for that party—they waste no votes—while a district lost with 49.9% of votes is the worst-possible district. Therefore, a tiny shift in votes commits a massive difference in efficiency. The same is true for declination.
The other two metrics—symmetry and MMD—smooth over the discontinuity. A tiny shift in votes does not change either of these metrics. But that creates a new problem. Now the metric is removed from actual outcomes. If Democrats win 2 of 3 districts with 40%, 51%, and 62% of the vote, MMD detects no difference from Democrats winning 3 of 3 districts with 51%, 51%, and 51% of the vote.
Electoral baseline is particularly devastating for symmetry. Who cares whether Republicans would theoretically earn 9 of 10 seats with 60% of votes, if they can never hope to earn 60%? Symmetry does not limit its investigation to plausible outcomes.2
The other metrics are also affected. Declination becomes unstable in landslide states and undefined altogether if the minority party loses every district.3 Let’s say District A tends to be 7 points below than the state average for Democrats. EG will be significantly different if the state votes 55% for Democrats (they lose the district with 48%) vs if the state votes 60% for Democrats (they win with 53%).
All four metrics require only district vote shares and seat outcomes—none account for geographic & legal contingency.
It is quite plausible Democrats are winning landslides, and losing by smaller margins, because they have overwhelming support in cities, and small pockets of support in other areas. Indeed, this is the thesis of Jonathan Rodden’s excellent book.4
These conditions would organically create an “asymmetric” map, where Democrats win fewer seats with equal vote share. It also means Democrats likely “waste” more votes (EG), have higher mean than median district vote share (MMD), and win with higher margins than they lose (declination).
These are not the only problems with these metrics. But this is, I think, a fair synthesis of the background conditions that make this a fickle problem.
There is a solution
Or at least a better framework.5 Simulate a bunch of maps. Use some basic geographic unit—e.g. precincts—and require maps meet contiguity/compactness/VRA requirements. Applying actual election returns, ideally from several elections, record how each party fares under each simulated map. Perhaps also test counterfactual vote shares.
Compare the enacted map to the simulated ensemble. Test how the enacted map compares to the ensemble on raw seat count or any other metric (including, if you’d like, symmetry, EG, MMD, etc.). If the enacted plan is more favorable to Democrats than 99.5% of simulated valid plans, that is really strong evidence of a gerrymander. The old metrics, too, can be interpreted against an ensemble.
Complexity hides in the detail. It’s not easy to model the VRA, there are different ways to define “compactness”, and there are different possible sampling distributions.
Still, simulation largely solves the baseline problem. It asks whether the enacted map is extreme relative to a stated universe of lawful, geographically plausible alternatives.
Gerrymandering is hard to measure because maps are not just partisan artifacts. Partisan manipulation occurs in the shadow of geography, law, and a particular political moment. A good test has to see the alternatives.
Criticism of partisan symmetry: Stephanopoulos & McGhee, Partisan Gerrymandering and the Efficiency Gap; Chen & Rodden, Cutting Through the Thicket (2015); DeFord et al., Implementing Partisan Symmetry (2023); DeFord & Veomett, Bounds and Bugs (2025);
Criticism of the efficiency gap: Bernstein & Duchin, A Formula Goes to Court (2017); Chambers et al., Flaws in the Efficiency Gap (2017); Cho, Measuring Partisan Fairness (2017); Cover, An Evaluation of the Efficiency Gap Proposal (2018); Veomett, The Efficiency Gap, Voter Turnout, and the Efficiency Principle (2018);
Criticism of Mean-Median Difference: Stephanopoulos & McGhee, The Measure of a Metric (2018); Zhu, Partisan Gerrymandering and Political Geography (2024);
Criticisms of declination: Campisi et al., Declination as a Metric to Detect Partisan Gerrymandering (2018); Katz et al., Theoretical Foundations and Empirical Evaluations (2018).
Criticism of all metrics: Ratliff et al., Don’t Trust a Single Gerrymandering Metric (2025)
Jonathan N. Katz, Gary King & Elizabeth Rosenblatt, Theoretical Foundations and Empirical Evaluations of Partisan Fairness in District-Based Democracies (2020).
Thomas Ratliff, Stephanie Somersille & Ellen Veomett, Don’t Trust a Single Gerrymandering Metric (2025).
Why Cities Lose (2019)
See, e.g., Chen & Rodden, 2013 & 2015; McCartan & Imai, 2023