Training-efficient density quantum machine learning

T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei, “Language models are few-shot learners,” in Proceedings of the 34th International Conference on Neural Information Processing Systems , ser. NIPS ’20. Red Hook, NY, USA: Curran Associates Inc., Dec. 2020, pp. 1877–1901

1901

Earlier work this paper cites.

G. E. Crooks, “Gradients of parameterized quantum gates using the parameter-shift rule and gate decomposition,” May 2019. [Online]. Available: http://arxiv.org/abs/1905.13311

Original

1905

Earlier work this paper cites.

G. Verdon, M. Broughton, J. R. McClean, K. J. Sung, R. Babbush, Z. Jiang, H. Neven, and M. Mohseni, “Learning to learn with quantum neural networks via classical neural networks,” Jul. 2019. [Online]. Available: http://arxiv.org/abs/1907.05415

Original

1907

Earlier work this paper cites.

I. Kerenidis, J. Landman, and A. Prakash, “Quantum Algorithms for Deep Convolutional Neural Networks,” in International Conference on Learning Representations , 2020. [Online]. Available: http://arxiv.org/abs/1911.01117

Original

1911

Earlier work this paper cites.

F. Rosenblatt, “The perceptron: A probabilistic model for information storage and organization in the brain,” Psychological Review , vol. 65, no. 6, pp. 386–408, 1958

1958

Earlier work this paper cites.

D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning representations by back-propagating errors,” Nature , vol. 323, no. 6088, pp. 533–536, Oct. 1986. [Online]. Available: https://www.nature.com/articles/323533a0

1986

Earlier work this paper cites.

R. A. Jacobs, M. I. Jordan, S. J. Nowlan, and G. E. Hinton, “Adaptive Mixtures of Local Experts,” Neural Computation , vol. 3, no. 1, pp. 79–87, Mar. 1991. [Online]. Available: https://ieeexplore.ieee.org/document/6797059

1991

Earlier work this paper cites.

M. Jordan and R. Jacobs, “Hierarchical mixtures of experts and the EM algorithm,” in Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan) , vol. 2, 1993, pp. 1339–1344 vol.2

1993

Earlier work this paper cites.

R. Collobert, S. Bengio, and Y. Bengio, “A Parallel Mixture of SVMs for Very Large Scale Problems,” in Advances in Neural Information Processing Systems , T. Dietterich, S. Becker, and Z. Ghahramani, Eds., vol. 14. MIT Press, 2001. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2001/file/36ac8e558ac7690b6f44e2cb5ef93322-Paper.pdf

2001

Earlier work this paper cites.

G. Brassard, P. Hoyer, M. Mosca, and A. Tapp, “Quantum Amplitude Amplification and Estimation,” 2002. [Online]. Available: http://arxiv.org/abs/quant-ph/0005055

2002

Earlier work this paper cites.

L. Cao, “Support vector machines experts for time series forecasting,” Neurocomputing , vol. 51, pp. 321–339, Apr. 2003. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0925231202005775

2003

Earlier work this paper cites.

C. A. M. Lima, A. L. V. Coelho, and F. J. Von Zuben, “Hybridizing mixtures of experts with support vector machines: Investigation into nonlinear dynamic systems identification,” Information Sciences , vol. 177, no. 10, pp. 2049–2074, May 2007. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0020025507000382

2007

Earlier work this paper cites.

P. Baldi and P. J. Sadowski, “Understanding Dropout,” in Advances in Neural Information Processing Systems , C. J. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger, Eds., vol. 26. Curran Associates, Inc., 2013. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2013/file/71f6278d140af599e06ad9bf1ba03cb0-Paper.pdf

2013

Earlier work this paper cites.

N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, “Dropout: a simple way to prevent neural networks from overfitting,” J. Mach. Learn. Res. , vol. 15, no. 1, pp. 1929–1958, Jan. 2014

2014

Earlier work this paper cites.

D. Eigen, M. Ranzato, and I. Sutskever, “Learning Factored Representations in a Deep Mixture of Experts,” Mar. 2014. [Online]. Available: http://arxiv.org/abs/1312.4314

Original

2014

Earlier work this paper cites.

Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature , vol. 521, no. 7553, pp. 436–444, May 2015. [Online]. Available: https://www.nature.com/articles/nature14539

2015

Earlier work this paper cites.

L. Theis and M. Bethge, “Generative Image Modeling Using Spatial LSTMs,” Sep. 2015. [Online]. Available: http://arxiv.org/abs/1506.03478

Original

2015

Earlier work this paper cites.

M. Deisenroth and J. W. Ng, “Distributed Gaussian Processes,” in Proceedings of the 32nd International Conference on Machine Learning , ser. Proceedings of Machine Learning Research, F. Bach and D. Blei, Eds., vol. 37. Lille, France: PMLR, Jul. 2015, pp. 1481–1490. [Online]. Available: https://proceedings.mlr.press/v37/deisenroth15.html

2015

Earlier work this paper cites.

D. Bahdanau, K. Cho, and Y. Bengio, “Neural Machine Translation by Jointly Learning to Align and Translate,” May 2016. [Online]. Available: http://arxiv.org/abs/1409.0473

Original

2016

Earlier work this paper cites.

A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is All you Need,” in Advances in Neural Information Processing Systems , I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, Eds., vol. 30. Curran Associates, Inc., 2017. [Online]. Available: https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf

2017

Earlier work this paper cites.

D. Silver, T. Hubert, J. Schrittwieser, I. Antonoglou, M. Lai, A. Guez, M. Lanctot, L. Sifre, D. Kumaran, T. Graepel, T. Lillicrap, K. Simonyan, and D. Hassabis, “Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm,” Dec. 2017. [Online]. Available: http://arxiv.org/abs/1712.01815

Original

2017

Earlier work this paper cites.

A. Kandala, A. Mezzacapo, K. Temme, M. Takita, M. Brink, J. M. Chow, and J. M. Gambetta, “Hardware-efficient variational quantum eigensolver for small molecules and quantum magnets,” Nature , vol. 549, no. 7671, pp. 242–246, Sep. 2017. [Online]. Available: https://www.nature.com/articles/nature23879

2017

Earlier work this paper cites.

M. B. Hastings, “Turning gate synthesis errors into incoherent errors,” Quantum Info. Comput. , vol. 17, no. 5-6, pp. 488–494, Mar. 2017

2017

Earlier work this paper cites.

E. Campbell, “Shorter gate sequences for quantum computing by mixing unitaries,” Phys. Rev. A , vol. 95, no. 4, p. 042306, Apr. 2017. [Online]. Available: https://link.aps.org/doi/10.1103/PhysRevA.95.042306

2017

Earlier work this paper cites.

N. Shazeer, A. Mirhoseini, K. Maziarz, A. Davis, Q. Le, G. Hinton, and J. Dean, “Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer,” Jan. 2017. [Online]. Available: http://arxiv.org/abs/1701.06538

Original

2017

Earlier work this paper cites.

K. Mitarai, M. Negoro, M. Kitagawa, and K. Fujii, “Quantum circuit learning,” Phys. Rev. A , vol. 98, no. 3, p. 032309, Sep. 2018. [Online]. Available: https://link.aps.org/doi/10.1103/PhysRevA.98.032309

2018

Earlier work this paper cites.

J. G. Vidal and D. O. Theis, “Calculus on parameterized quantum circuits,” Dec. 2018. [Online]. Available: http://arxiv.org/abs/1812.06323

Original

2018

Earlier work this paper cites.

J. R. McClean, S. Boixo, V. N. Smelyanskiy, R. Babbush, and H. Neven, “Barren plateaus in quantum neural network training landscapes,” Nat Commun , vol. 9, no. 1, p. 4812, Nov. 2018. [Online]. Available: https://www.nature.com/articles/s41467-018-07090-4

2018

Earlier work this paper cites.

J. Bradbury, R. Frostig, P. Hawkins, M. J. Johnson, C. Leary, D. Maclaurin, G. Necula, A. Paszke, J. VanderPlas, S. Wanderman-Milne, and Q. Zhang, “JAX: composable transformations of Python+NumPy programs,” 2018. [Online]. Available: http://github.com/google/jax

2018

Earlier work this paper cites.

S. Cheng, J. Chen, and L. Wang, “Information Perspective to Probabilistic Modeling: Boltzmann Machines versus Born Machines,” Entropy , vol. 20, no. 8, p. 583, Aug. 2018. [Online]. Available: https://www.mdpi.com/1099-4300/20/8/583

2018

Earlier work this paper cites.

J.-G. Liu and L. Wang, “Differentiable learning of quantum circuit Born machines,” Phys. Rev. A , vol. 98, no. 6, p. 062324, Dec. 2018. [Online]. Available: https://link.aps.org/doi/10.1103/PhysRevA.98.062324

2018

Earlier work this paper cites.

M. Schuld, V. Bergholm, C. Gogolin, J. Izaac, and N. Killoran, “Evaluating analytic gradients on quantum hardware,” Phys. Rev. A , vol. 99, no. 3, p. 032331, Mar. 2019. [Online]. Available: https://link.aps.org/doi/10.1103/PhysRevA.99.032331

2019

Cited alongside, same era.

S. Khatri, R. LaRose, A. Poremba, L. Cincio, A. T. Sornborger, and P. J. Coles, “Quantum-assisted quantum compiling,” Quantum , vol. 3, p. 140, May 2019. [Online]. Available: https://quantum-journal.org/papers/q-2019-05-13-140/

2019

Cited alongside, same era.

I. Cong, S. Choi, and M. D. Lukin, “Quantum convolutional neural networks,” Nat. Phys. , vol. 15, no. 12, pp. 1273–1278, Dec. 2019. [Online]. Available: https://www.nature.com/articles/s41567-019-0648-8

2019

Cited alongside, same era.

T. Akiba, S. Sano, T. Yanase, T. Ohta, and M. Koyama, “Optuna: A Next-Generation Hyperparameter Optimization Framework,” in The 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining , 2019, pp. 2623–2631

2019

B. Kiani, R. Balestriero, Y. LeCun, and S. Lloyd, “projUNN: efficient method for training deep networks with unitary matrices,” Oct. 2022. [Online]. Available: http://arxiv.org/abs/2203.05483

Original

2022

Later among the works it cites.

V. Bergholm, J. Izaac, M. Schuld, C. Gogolin, S. Ahmed, V. Ajith, M. S. Alam, G. Alonso-Linaje, B. AkashNarayanan, A. Asadi, J. M. Arrazola, U. Azad, S. Banning, C. Blank, T. R. Bromley, B. A. Cordier, J. Ceroni, A. Delgado, O. Di Matteo, A. Dusko, T. Garg, D. Guala, A. Hayes, R. Hill, A. Ijaz, T. Isacsson, D. Ittah, S. Jahangiri, P. Jain, E. Jiang, A. Khandelwal, K. Kottmann, R. A. Lang, C. Lee, T. Loke, A. Lowe, K. McKiernan, J. J. Meyer, J. A. Monta√±ez-Barrera, R. Moyard, Z. Niu, L. J. O’Riordan, S. Oud, A. Panigrahi, C.-Y. Park, D. Polatajko, N. Quesada, C. Roberts, N. S√°, I. Schoch, B. Shi, S. Shu, S. Sim, A. Singh, I. Strandberg, J. Soni, A. Sz√°va, S. Thabet, R. A. Vargas-Hern√°ndez, T. Vincent, N. Vitucci, M. Weber, D. Wierichs, R. Wiersema, M. Willmann, V. Wong, S. Zhang, and N. Killoran, “PennyLane: Automatic differentiation of hybrid quantum-classical computations,” Jul. 2022. [Online]. Available: http://arxiv.org/abs/1811.04968

Original

2022

Later among the works it cites.

M. C. Caro, H.-Y. Huang, M. Cerezo, K. Sharma, A. Sornborger, L. Cincio, and P. J. Coles, “Generalization in quantum machine learning from few training data,” Nat Commun , vol. 13, no. 1, p. 4919, Aug. 2022. [Online]. Available: https://www.nature.com/articles/s41467-022-32550-3

2022