Efficient Automatic Differentiation of Implicit Functions

[author] Karush, WW. (1939). Minima of Functions of Several Variables with Inequalities as Side Constraints. (M.Sc. thesis). Dept. of Mathematics, Univ. of Chicago, Chicago, Illinois

1939

Earlier work this paper cites.

[author] Kuhn, H. W.H. W. and Tucker, A. W.A. W. (1951). Nonlinear programming. Proceedings of 2nd Berkeley Symposium 481–492

1951

Earlier work this paper cites.

[author] Pontryagin, LSL., Boltyanskii, VGV., Gamkrelidze, RVR. and Mishechenko, EFE. (1963). The Mathematical Theory of Optimal Processes

1963

Earlier work this paper cites.

[author] Errico, MM. (1997). What is an adjoint model? Bulletin of the American Meteorological Society 78 2577 – 2591

1997

Earlier work this paper cites.

[author] Betancourt, MM., Margossian, CCC. and Leos-Barajas, VV. (2020). The Discrete Adjoint Method: Efficient Derivatives for Functions of Discrete Sequences. arXiv:2002.00326

Original

2002

Earlier work this paper cites.

[author] Cao, YY., Li, ShengtaiS., Petzold, LL. and R, SerbanS. (2002). Adjoint Sensitivity Analysis for Differential-Algebraic Equations: The Adjoint DAE System and Its Numerical Solution. SIAM Journal on Scientific Computing 24 1076 – 1089

2002

Earlier work this paper cites.

[author] Bell, Bradley MB. M. and Burke, James VJ. V. (2008). Algorithmic Differentiation of Implicit Functions and Optimal Values. In Advances in Automatic Differentiation. Lecture Notes in Computational Science and Engineering, (C. HC. H. Bischof, H. MH. M. Bücker, PP. Hovland, UU. Naumann and JJ. Utke, eds.) 64 Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-68942-3_17

2008

Earlier work this paper cites.

[author] Griewank, AndreasA. and Walther, AndreaA. (2008). Evaluating derivatives, Second ed. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA

2008

Cited alongside, same era.

[author] Carpenter, BobB., Hoffman, Matthew D.M. D., Brubaker, Marcus A.M. A., Lee, DanielD., Li, PeterP. and Betancourt, Michael J.M. J. (2015). The Stan Math Library: Reverse-Mode Automatic Differentiation in C++. arXiv 1509.07164

2015

Cited alongside, same era.

[author] Carpenter, BobB., Gelman, AndrewA., Hoffman, MattM., Lee, DanielD., Goodrich, BenB., Betancourt, MichaelM., Brubaker, Marcus A.M. A., Guo, JiqiangJ., Li, PeterP. and Riddel, AllenA. (2017). Stan: A Probabilistic Programming Language. Journal of Statistical Software 76 1 –32. 10.18637/jss.v076.i01

2017

Cited alongside, same era.

[author] Dillon, Joshua VJ. V., Langmore, IanI., Tran, DustinD., Brevdo, EugeneE., Vasudevan, SrinivasS., Moore, DaveD., Patton, BrianB., Alemi, AlexA., Hoffman, MattM. and Saurous, Rif AR. A. (2017). Tensorflow distributions. arXiv preprint arXiv:1711.10604

Original

[author] Rackauckas, ChristopherC., Ma, YingboY., Dixit, VaibhavV., Guo, XingjianX., Innes, MikeM., Revels, JarrettJ., Nyberg, JoakimJ. and Ivaturi, VijayV. (2018). A Comparison of Automatic Differentiation and Continuous Sensitivity Analysis for Derivatives of Differential Equation Solutions. arXiv:1812.01892

Original

2018

Later among the works it cites.

[author] Lorraine, JJ., Vicol, PP. and Duvenaud, DD. (2019). Optimizing Millions of Hyperparameters by Implicit Differentiation. Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, PMLR 108 1540 – 1552

2019

Later among the works it cites.

[author] Margossian, Charles C.C. C. (2019). A Review of automatic differentiation and its efficient implementation. Wiley interdisciplinary reviews: data mining and knowledge discovery 9. 10.1002/WIDM.1305

2019

Later among the works it cites.

[author] Hindmarsh, AA. and Serban, RR. (2020). User Documentation for CVODES v5.1.0. Technical Report

2020

Later among the works it cites.