An Automatic Finite-Sample Robustness Metric: When Can Dropping a Little Data Make a Big Difference?

Understand

Study samples often differ from the target populations of inference and policy decisions in non-random ways.

• Researchers typically believe that such departures from random sampling -- due to changes in the population over time and space, or difficulties in sampling truly randomly -- are small, and their corresponding impact on the inference should be small as well.
• We might therefore be concerned if the conclusions of our studies are excessively sensitive to a very small proportion of our sample data.
• We propose a method to assess the sensitivity of applied econometric conclusions to the removal of a small fraction of the sample.