2017

Optimism-Based Adaptive Regulation of Linear-Quadratic Systems

Faradonbeh, Mohamad Kazem Shirani, Tewari, Ambuj, Michailidis, George

Understand

The main challenge for adaptive regulation of linear-quadratic systems is the trade-off between identification and control.

An adaptive policy needs to address both the estimation of unknown dynamics parameters (exploration), as well as the regulation of the underlying system (exploitation).
To this end, optimism-based methods which bias the identification in favor of optimistic approximations of the true parameter are employed in the literature.
A number of asymptotic results have been established, but their finite time counterparts are few, with important restrictions.

B. M. Brown et al. , “Martingale central limit theorems,” The Annals of Mathematical Statistics , vol. 42, no. 1, pp. 59–66, 1971
1971
Earlier work this paper cites.
Y. Bar-Shalom and E. Tse, “Dual effect, certainty equivalence, and separation in stochastic control,” IEEE Transactions on Automatic Control , vol. 19, no. 5, pp. 494–500, 1974
1974
Earlier work this paper cites.
T. L. Lai and C. Z. Wei, “Least squares estimates in stochastic regression models with applications to identification and control of dynamic systems,” The Annals of Statistics , pp. 154–166, 1982
1982
Earlier work this paper cites.
A. Becker, P. Kumar, and C.-Z. Wei, “Adaptive control with the stochastic approximation algorithm: Geometry and convergence,” IEEE Transactions on Automatic Control , vol. 30, no. 4, pp. 330–338, 1985
1985
Earlier work this paper cites.
T. L. Lai and H. Robbins, “Asymptotically efficient adaptive allocation rules,” Advances in applied mathematics , vol. 6, no. 1, pp. 4–22, 1985
1985
Earlier work this paper cites.
M. Green and J. B. Moore, “Persistence of excitation in linear systems,” in American Control Conference, 1985 . IEEE, 1985, pp. 412–417
1985
Earlier work this paper cites.
T. L. Lai, “Asymptotically efficient adaptive control in stochastic regression models,” Advances in Applied Mathematics , vol. 7, no. 1, pp. 23–45, 1986
1986
Earlier work this paper cites.

T. L. Lai and C.-Z. Wei, “Extended least squares and their applications to adaptive control and prediction in linear systems,” IEEE Transactions on Automatic Control , vol. 31, no. 10, pp. 898–906, 1986
1986
Cited alongside, same era.
L. Guo and H. Chen, “Convergence rate of els based adaptive tracker,” Syst. Sci & Math. Sci , vol. 1, pp. 131–138, 1988
1988
Cited alongside, same era.
P. Kumar, “Convergence of adaptive control schemes using least-squares parameter estimates,” IEEE Transactions on Automatic Control , vol. 35, no. 4, pp. 416–424, 1990
1990
Cited alongside, same era.
T. L. Lai and Z. Ying, “Parallel recursive algorithms in asymptotically efficient adaptive control of linear stochastic systems,” SIAM journal on control and optimization , vol. 29, no. 5, pp. 1091–1127, 1991
1991
Cited alongside, same era.
D. P. Bertsekas, Dynamic programming and optimal control . Athena Scientific Belmont, MA, 1995, vol. 1, no. 2
1995
Cited alongside, same era.
M. C. Campi, “Achieving optimality in adaptive control: the” bet on the best” approach,” in Decision and Control, 1997., Proceedings of the 36th IEEE Conference on , vol. 5. IEEE, 1997, pp. 4671–4676
1997
Cited alongside, same era.
M. C. Campi and P. Kumar, “Adaptive linear quadratic gaussian control: the cost-biased approach revisited,” SIAM Journal on Control and Optimization , vol. 36, no. 6, pp. 1890–1907, 1998
1998
Cited alongside, same era.

S. Bittanti and M. C. Campi, “Adaptive control of linear time invariant systems: the “bet on the best” principle,” Communications in Information & Systems , vol. 6, no. 4, pp. 299–320, 2006
2006
Later among the works it cites.
Y. Abbasi-Yadkori and C. Szepesvári, “Regret bounds for the adaptive control of linear quadratic systems.” in COLT , 2011, pp. 1–26
2011
Later among the works it cites.
M. Ibrahimi, A. Javanmard, and B. V. Roy, “Efficient reinforcement learning for high dimensional linear quadratic systems,” in Advances in Neural Information Processing Systems , 2012, pp. 2636–2644
2012
Later among the works it cites.
J. A. Tropp, “User-friendly tail bounds for sums of random matrices,” Foundations of computational mathematics , vol. 12, no. 4, pp. 389–434, 2012
2012
Later among the works it cites.
M. K. S. Faradonbeh, A. Tewari, and G. Michailidis, “Optimality of fast matching algorithms for random networks with applications to structural controllability,” IEEE Transactions on Control of Network Systems , vol. 4, no. 4, pp. 770–780, 2017
2017
Closest in time.
M. K. S. Faradonbeh, A. Tewari, and G. Michailidis, “Finite time adaptive stabilization of LQ systems,” IEEE Transactions on Automatic Control , 2018
2018
Closest in time.
M. K. S. Faradonbeh, A. Tewari, and G. Michailidis, “Finite time identification in unstable linear systems,” Automatica , vol. 96, pp. 342–353, 2018
2018
Closest in time.

alphaXiv searches the wider corpus for related work and actual follow-ups.

alphaXiv is searching for related work…

Reading the bibliography…