2015

External Validity: From Do-Calculus to Transportability Across Populations

Pearl, Judea, Bareinboim, Elias

Understand

The generalizability of empirical findings to new environments, settings or populations, often called "external validity," is essential in most scientific explorations.

This paper treats a particular problem of generalizability, called "transportability," defined as a license to transfer causal effects learned in experimental studies to a new population, in which only observational studies can be conducted.
We introduce a formal representation called "selection diagrams" for expressing knowledge about differences and commonalities between populations of interest and, using this representation, we reduce questions of transportability to symbolic derivations in the do-calculus.
This reduction yields graph-based procedures for deciding, prior to observing any data, whether causal effects in the target population can be inferred from experimental findings in the study population.

Built on

Westergaard, H.H. (1916). Scope and method of statistics. Publ. Amer. Statist. Assoc. 15 229–276
1916
Earlier work this paper cites.
Wright, S.S. (1921). Correlation and causation. J. Agricultural Research 20 557–585
1921
Earlier work this paper cites.
Yule, G. U.G. U. (1934). On some points relating to vital statistics, more especially statistics of occupational mortality. J. Roy. Statist. Soc. 97 1–84
1934
Earlier work this paper cites.
Haavelmo, TrygveT. (1943). The statistical implications of a system of simultaneous equations. Econometrica 11 1–12
1943
Earlier work this paper cites.
Berkson, J.J. (1946). Limitations of the application of fourfold table analysis to hospital data. Biometrics 2 47–53
1946
Earlier work this paper cites.
Strotz, Robert H.R. H. andWold, H. O. A.H. O. A. (1960). Recursive vs. nonrecursive systems: An attempt at synthesis. Econometrica 28 417–427
1960
Earlier work this paper cites.
Campbell, D.D. andStanley, J.J. (1963). Experimental and Quasi-Experimental Designs for Research. Wadsworth, Chicago
1963
Earlier work this paper cites.
Rubin, D. B.D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. J. Educational Psychology 66 688–701
1974
Earlier work this paper cites.
Heise, D. R.D. R. (1975). Causal Analysis. Wiley, New York
1975
Earlier work this paper cites.
Glass, Gene V.G. V. (1976). Primary, secondary, and meta-analysis of research. Educational Researcher 5 3–8
1976
Earlier work this paper cites.
Heckman, James J.J. J. (1979). Sample selection bias as a specification error. Econometrica 47 153–161
1979
Earlier work this paper cites.
Lane, P. W.P. W. andNelder, J. A.J. A. (1982). Analysis of covariance and standardization as instances of prediction. Biometrics 38 613–621
1982
Earlier work this paper cites.
Davis, James A.J. A. (1984). Extending Rosenberg’s technique for standardizing percentage tables. Social Forces 62 679–708
1984
Earlier work this paper cites.
Hedges, Larry V.L. V. andOlkin, IngramI. (1985). Statistical Methods for Meta-Analysis. Academic Press, Orlando, FL
1985
Earlier work this paper cites.
Robins, JamesJ. (1986). A new approach to causal inference in mortality studies with a sustained exposure period—Application to control of the healthy worker survivor effect. Math. Modelling 7 1393–1512
1986
Earlier work this paper cites.
Pearl, J.J. (1988). Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann, San Mateo, CA
1988
Earlier work this paper cites.
Ellenberg, S. S.S. S. andHamilton, J. M.J. M. (1989). Surrogate endpoints in clinical trials: Cancer. Stat. Med. 8 405–413
1989
Earlier work this paper cites.
Prentice, R. L.R. L. (1989). Surrogate endpoints in clinical trials: Definition and operational criteria. Stat. Med. 8 431–440
1989
Earlier work this paper cites.
Neyman, J.J. (1923). Sur les applications de la thar des probabilities aux experiences Agaricales: Essay des principle. English translation of excerpts by D. Dabrowska and T. Speed in Statist. Sci. 5
1990
Earlier work this paper cites.
Verma, T.T. andPearl, J.J. (1988). Causal networks: Semantics and expressiveness. In Proceedings of the Fourth Workshop on Uncertainty in Artificial Intelligence 352–359. Mountain View, CA. Also in Uncertainty in AI 4
1990
Earlier work this paper cites.

Similar

Adelman, L.L. (1991). Experiments, quasi-experiments, and case studies: A review of empirical methods for evaluating decision support systems. IEEE Transactions on Systems, Man and Cybernetics 21 293–301
1991
Cited alongside, same era.
Pearl, J.J. (1993). Graphical models, causality, and intervention. Statist. Sci. 8 266–273
1993
Cited alongside, same era.
Spirtes, PeterP., Glymour, ClarkC. andScheines, RichardR. (1993). Causation, Prediction, and Search. Lecture Notes in Statistics 81. Springer, New York
1993
Cited alongside, same era.
Balke, AlexanderA. andPearl, JudeaJ. (1995). Counterfactuals and policy analysis in structural models. In Proceedings of the Eleventh Conference on Uncertainty in Artificial Intelligence (P.P. Besnard andS.S. Hanks, eds.) 11–18. Morgan Kaufmann, San Francisco, CA
1995
Cited alongside, same era.
Pearl, JudeaJ. (1995). Causal diagrams for empirical research. Biometrika 82 669–710
1995
Cited alongside, same era.
Rosenthal, R.R. (1995). Writing meta-analytic reviews. Psychological Bulletin 118 183–192
1995
Cited alongside, same era.
Lehmann, E. L.E. L. andCasella, GeorgeG. (1998). Theory of Point Estimation, 2nd ed. Springer, New York
1998
Cited alongside, same era.
Pearl, JudeaJ. (2000). Causality: Models, Reasoning, and Inference. Cambridge Univ. Press, Cambridge
2000
Cited alongside, same era.
Spirtes, PeterP., Glymour, ClarkC. andScheines, RichardR. (2000). Causation, Prediction, and Search, 2nd ed. Adaptive Computation and Machine Learning. MIT Press, Cambridge, MA
2000
Cited alongside, same era.
Dawid, A. P.A. P. (2002). Influence diagrams for causal modelling and inference. Internat. Statist. Rev. 70 161–189
2002
Cited alongside, same era.
Shadish, W. R.W. R., Cook, T. D.T. D. andCampbell, D. T.D. T. (2002). Experimental and Quasi-Experimental Designs for Generalized Causal Inference, 2nd ed. Houghton-Mifflin, Boston
2002
Cited alongside, same era.
Tian, J.J. andPearl, J.J. (2002). A general identification condition for causal effects. In Proceedings of the Eighteenth National Conference on Artificial Intelligence 567–573. AAAI Press/The MIT Press, Menlo Park, CA
2002
Cited alongside, same era.

Then

Glymour, M. M.M. M. andGreenland, S.S. (2008). Causal diagrams. In Modern Epidemiology, 3rd ed. (K. J.K. J. Rothman, S.S. Greenland andT. L.T. L. Lash, eds.) 183–209. Lippincott Williams & Wilkins, Philadelphia, PA
2008
Later among the works it cites.
Robins, JamesJ., Orellana, LilianaL. andRotnitzky, AndreaA. (2008). Estimation and extrapolation of optimal treatment and testing strategies. Stat. Med. 27 4678–4721
2008
Later among the works it cites.
Joffe, Marshall M.M. M. andGreene, TomT. (2009). Related causal frameworks for surrogate outcomes. Biometrics 65 530–538
2009
Later among the works it cites.
Koller, DaphneD. andFriedman, NirN. (2009). Probabilistic Graphical Models: Principles and Techniques. Adaptive Computation and Machine Learning. MIT Press, Cambridge, MA
2009
Later among the works it cites.
Owen, Art B.A. B. (2009). Karl Pearson’s meta-analysis revisited. Ann. Statist. 37 3867–3892
2009
Later among the works it cites.
White, HalbertH. andChalak, KarimK. (2009). Settable systems: An extension of Pearl’s causal model with optimization, equilibrium, and learning. J. Mach. Learn. Res. 10 1759–1799
2009
Later among the works it cites.
Cole, Stephen R.S. R. andStuart, Elizabeth A.E. A. (2010). Generalizing evidence from randomized clinical trials to target populations: The ACTG 320 trial. Am. J. Epidemiol. 172 107–115
2010
Later among the works it cites.
Höfler, M.M., Gloster, A. T.A. T. andHoyer, J.J. (2010). Causal effects in psychotherapy: Counterfactuals counteract overgeneralization. Psychotherapy Research 20 668–679. DOI: \doiurl
2010
Later among the works it cites.
Hernán, Miguel A.M. A. andVanderWeele, Tyler J.T. J. (2011). Compound treatments and transportability of causal inference. Epidemiology 22 368–377
2011
Later among the works it cites.
Pearl, JudeaJ. (2011). The structural theory of causation. In Causality in the Sciences (P. McKay Illari, F. Russo and J. Williamson, eds.) 697–727. Clarendon Press, Oxford
2011
Later among the works it cites.
Pearl, J.J. andBareinboim, E.E. (2011). Transportability across studies: A formal approach. Technical Report R-372, Cognitive Systems Laboratory, Dept. Computer Science, UCLA
2011
Later among the works it cites.
Petersen, Maya L.M. L. (2011). Compound treatments, transportability, and the structural causal model: The power and simplicity of causal graphs. Epidemiology 22 378–381
2011
Later among the works it cites.
Bareinboim, EliasE., Brito, CarlosC. andPearl, JudeaJ. (2012). Local characterizations of causal Bayesian networks. In Graph Structures for Knowledge Representation and Reasoning. Lecture Notes in Artificial Intelligence 7205 1–17. Springer, Berlin
2012
Later among the works it cites.
Bareinboim, E.E. andPearl, J.J. (2012). Transportability of causal effects: Completeness results. In Proceedings of the Twenty-Sixth National Conference on Artificial Intelligence 698–704. AAAI Press, Menlo Park, CA
2012
Later among the works it cites.
Bareinboim, EliasE. andPearl, JudeaJ. (2013c). Meta-transportability of causal effects: A formal approach. In Proceedings of the Sixteenth International Conference on Artificial Intelligence and Statistics (AISTATS 2013). J. Mach. Learn. Res. 31 135–143
2013
Later among the works it cites.
Bollen, K. A.K. A. andPearl, J.J. (2013). Eight myths about causality and structural equation models. In Handbook of Causal Analysis for Social Research (S. L.S. L. Morgan, ed.) Chapter 15. Springer, New York
2013
Later among the works it cites.
Pearl, J.J. (2013). Linear models: A useful “microscope” for causal analysis. J. Causal Inference 1 155–170
2013
Later among the works it cites.
Bareinboim, EliasE., Tian, JinJ. andPearl, JudeaJ. (2014). Recovering from selection bias in causal and statistical inference. In Proceedings of The Twenty-Eighth Conference on Artificial Intelligence (Carla E.C. E. Brodley andPeterP. Stone, eds.). AAAI Press, Menlo Park, CA. To appear
2014
Later among the works it cites.
Pearl, J.J. (2014). Trygve Haavelmo and the emergence of causal calculus. Econometric Theory, Special Issue on Haavelmo Centennial . Published online: 10 June 2014. DOI: \doiurl
2014
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…