The Prova Method · Standards
The Identification Atlas
The natural experiments hiding in how programs already run.
Most social programs already contain credible natural experiments: points where some people, places, or times were treated differently for reasons that let you read a causal effect from data you already have. They are rarely surfaced, because finding them takes econometric and operational fluency at once, and the two seldom sit in the same place. This is the field guide we look with: the families of natural experiment, where each one is credible, and what it can and cannot support.
It is the standard behind the Counterfactual Hunt.
Hiding in plain sight
The experiment was already there.
The administrative border
A program stops at a district line. Households just inside and just outside are alike but for access, and the border reads the effect.
The families
Six patterns, and where each one holds.
1 · The eligibility cutoff → Regression discontinuity
The pattern: A program admits people on one side of a threshold: an income line, a test score, an age. People just above and just below are alike in everything except the program.
Credible when applicants can’t finely control which side they land on, and nothing else changes at that line. Supports a clean causal effect for people near the cutoff. Boundary: it says nothing about people far from it. Shows up in: means-tested benefits, scholarship cutoffs, risk-score thresholds.
2 · The staggered rollout → Difference-in-differences
The pattern: A program reaches sites at different times; sites not yet reached are the comparison for those already reached.
Credible when the order of rollout isn’t driven by the outcome (struggling sites pushed to the front, say), and the groups would have moved in parallel without the program. Supports the causal effect of the rollout. Boundary: collapses if rollout order tracks the outcome’s trajectory. Shows up in: anything phased across districts, cohorts, or years.
3 · Oversubscription → the lottery you already ran
The pattern: Demand exceeds capacity and places are given out by lottery or draw, a randomized experiment you didn’t have to design.
Credible when allocation really was random, and takeup doesn’t differ systematically between winners and losers. Supports a causal effect with the rigor of a trial, for the applicant pool. Boundary: it is the effect for the applicant pool, and doesn’t extend to the general population. Shows up in: oversubscribed slots, randomized waitlists, balloted places.
4 · The capacity constraint → an instrument
The pattern: Something other than the participant decides who actually gets the program (a caseworker’s load, distance to the nearest site, a quota), and that external nudge can serve as an instrument for participation.
Credible when the nudge affects the outcome only through whether someone takes part (the exclusion restriction); this is the hardest assumption to defend, and the one to be most honest about. Supports a causal effect for those whose participation the nudge actually moves. Boundary: a local effect, on that subgroup. Shows up in: distance to services, capacity rationing, assignment quirks.
5 · The administrative border → a geographic discontinuity
The pattern: A program stops at a district or catchment boundary; households just inside and just outside are similar except for access.
Credible when the boundary isn’t also a line for other services or policies, and people don’t move across it to get in. Supports a causal effect at the border. Boundary: local to the border; sorting across it is the main threat. Shows up in: programs run by one jurisdiction and not its neighbor.
6 · No internal comparison → a built comparison (synthetic control)
The pattern: Sometimes there’s no variation inside the program at all, just one unit treated all at once. A comparison can still be built: a weighted blend of untreated units chosen to track the treated unit’s path before the program began.
Credible when there’s a close pre-program match and no other shock hits only the treated unit. Supports a causal estimate for that one unit. Boundary: it rests entirely on the quality of the pre-period match, the weakest of these, used only when nothing stronger is available. Shows up in: a single state, city, or system adopting a policy at once.
The discipline
Finding one is not the same as certifying it.
Every candidate is graded for how credible its key assumption actually is, and where none of these holds, we say so.
A program with no credible natural experiment in its data is a real finding, honestly reported rather than papered over. That discipline is the same one in the Honesty Layer: a design is only worth as much as the assumption beneath it can bear.