2021

Acquisition of Chess Knowledge in AlphaZero

McGrath, Thomas, Kapishnikov, Andrei, Tomašev, Nenad et al.

Understand

What is learned by sophisticated neural network agents such as AlphaZero? This question is of both scientific and practical interest.

If the representations of strong neural networks bear no resemblance to human concepts, our ability to understand faithful explanations of their decisions will be restricted, ultimately limiting what we can achieve with neural network interpretability.
In this work we provide evidence that human knowledge is acquired by the AlphaZero neural network as it trains on the game of chess.
By probing for a broad range of human chess concepts we show when and where these concepts are represented in the AlphaZero network.

Built on

Decoupling feature extraction from policy learning: assessing benefits of state representation learning in goal based robotics (2019)
Raffin, A. et al · 1901
Earlier work this paper cites.
Human-centered tools for coping with imperfect algorithms during medical decision-making
Original
Cai, C. J. et al · 1902
Earlier work this paper cites.
Explainable reinforcement learning through a causal lens (2019)
Madumal, P., Miller, T., Sonenberg, L. & Vetere, F · 1905
Earlier work this paper cites.
Discorl: Continual reinforcement learning via policy distillation (2019)
Traoré, R. et al · 1907
Earlier work this paper cites.
Sentimate: Learning to play chess through natural language processing (2019)
Kamlish, I., Chocron, I. B. & McCarthy, N · 1907
Earlier work this paper cites.
Efficient saliency maps for explainable ai (2020)
Mundhenk, T. N., Chen, B. Y. & Friedland, G · 1911
Earlier work this paper cites.
Explain your move: Understanding agent actions using specific and relevant feature attribution (2020)
Puri, N. et al · 1912
Earlier work this paper cites.
Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values
Paatero, P. & Tapper, U · 1994
Earlier work this paper cites.
Case-based evaluation in computer chess
Kerner, Y · 1995
Earlier work this paper cites.
Learning strategies for explanation patterns: Basic game patterns with application to chess
HaCohen-Kerner, Y · 1995
Earlier work this paper cites.
Learning the parts of objects by non-negative matrix factorization
Lee, D. D. & Seung, H. S · 1999
Earlier work this paper cites.
Dream architecture: a developmental approach to open-ended learning in robotics (2020)
Doncieux, S. et al · 2005
Earlier work this paper cites.
Gradient alignment in deep neural networks
Original
Srinivas, S. & Fleuret, F · 2006
Earlier work this paper cites.
Explainability in deep reinforcement learning (2020)
Heuillet, A., Couthouis, F. & Díaz-Rodríguez, N · 2008
Earlier work this paper cites.
Learning personalized models of human behavior in chess (2021)
McIlroy-Young, R., Wang, R., Sen, S., Kleinberg, J. & Anderson, A · 2008
Earlier work this paper cites.
Visualizing data using t-sne
Van der Maaten, L. & Hinton, G · 2008
Earlier work this paper cites.
Assessing game balance with alphazero: Exploring alternative rule sets in chess (2020)
Tomašev, N., Paquet, U., Hassabis, D. & Kramnik, V · 2009
Earlier work this paper cites.
Domain-level explainability – a challenge for creating trust in superhuman ai strategies (2020)
Andrulis, J., Meyer, O., Schott, G., Weinbach, S. & Gruhn, V · 2011
Earlier work this paper cites.
Debugging tests for model explanations
Original
Adebayo, J., Muelly, M., Liccardi, I. & Kim, B · 2011
Earlier work this paper cites.
Hierarchical sub-task decomposition for reinforcement learning of multi-robot delivery mission
Kawano, H · 2013
Earlier work this paper cites.
Chess q & a : Question answering on chess games (2015)
Cirik, V., Morency, L.-P. & Hovy, E · 2015
Earlier work this paper cites.
Understanding intermediate layers using linear classifier probes
Original
Alain, G. & Bengio, Y · 2016
Earlier work this paper cites.
Towards deep symbolic reinforcement learning (2016)
Garnelo, M., Arulkumaran, K. & Shanahan, M · 2016
Earlier work this paper cites.
Graying the black box: Understanding DQNs
Zahavy, T., Ben-Zrihem, N. & Mannor, S · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
He, K., Zhang, X., Ren, S. & Sun, J · 2016
Earlier work this paper cites.
Network dissection: Quantifying interpretability of deep visual representations
Bau, D., Zhou, B., Khosla, A., Oliva, A. & Torralba, A · 2017
Earlier work this paper cites.
Towards a rigorous science of interpretable machine learning
Original
Doshi-Velez, B., Finale; Kim · 2017
Earlier work this paper cites.
A unified approach to interpreting model predictions
Lundberg, S. M. & Lee, S.-I · 2017
Earlier work this paper cites.
Axiomatic attribution for deep networks
Sundararajan, M., Taly, A. & Yan, Q · 2017
Earlier work this paper cites.
Interpretable explanations of black boxes by meaningful perturbation
Original
Fong, R. & Vedaldi, A · 2017
Earlier work this paper cites.
Grad-cam: Visual explanations from deep networks via gradient-based localization
Selvaraju, R. R. et al · 2017
Earlier work this paper cites.

Similar

Real Time Image Saliency for Black Box Classifiers
Dabkowski, P. & Gal, Y · 2017
Cited alongside, same era.
The (un) reliability of saliency methods
Original
Kindermans, P.-J. et al · 2017
Cited alongside, same era.
Hybrid reward architecture for reinforcement learning (2017)
van Seijen, H. et al · 2017
Cited alongside, same era.
Understanding Chess with Explainable AI
DecodeChess · 2017
Cited alongside, same era.
Axiomatic attribution for deep networks
Sundararajan, M., Taly, A. & Yan, Q · 2017
Cited alongside, same era.
A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play
Silver, D. et al · 2018
Cited alongside, same era.
Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav) (2018)
Kim, B. et al · 2018
Cited alongside, same era.
Towards robust interpretability with self-explaining neural networks
Alvarez-Melis, D. & Jaakkola, T. S · 2018
Cited alongside, same era.
The building blocks of interpretability
Olah, C. et al · 2018
Cited alongside, same era.
The mythos of model interpretability: In machine learning, the concept of interpretability is both important and slippery
Lipton, Z. C · 2018
Cited alongside, same era.
Regression concept vectors for bidirectional explanations in histopathology
Graziani, M., Andrearczyk, V. & Müller, H · 2018
Cited alongside, same era.
Sanity checks for saliency maps
Adebayo, J. et al · 2018
Cited alongside, same era.

Then

Grad-cam: Visual explanations from deep networks via gradient-based localization
Selvaraju, R. R. et al · 2019
Later among the works it cites.
Leela Chess Zero
LCZero Development Community · 2019
Later among the works it cites.
Deep learning investigation for chess player attention prediction using eye-tracking and game data
Louedec, J. L., Guntz, T., Crowley, J. L. & Vaufreydaz, D · 2019
Later among the works it cites.
Game Changer: AlphaZero’s Groundbreaking Chess Strategies and the Promise of AI (New In Chess, 2019)
Sadler, M. & Regan, N · 2019
Later among the works it cites.
A theory of usable information under computational constraints
Xu, Y., Zhao, S., Song, J., Stewart, R. & Ermon, S · 2019
Later among the works it cites.
Gated linear networks
Original
Veness, J. et al · 2019
Later among the works it cites.
Understanding the role of individual units in a deep neural network
Bau, D. et al · 2020
Later among the works it cites.
Emergent linguistic structure in artificial neural networks trained by self-supervision
Manning, C. D., Clark, K., Hewitt, J., Khandelwal, U. & Levy, O · 2020
Later among the works it cites.
A primer in bertology: What we know about how bert works
Rogers, A., Kovaleva, O. & Rumshisky, A · 2020
Later among the works it cites.
Concept bottleneck models
Original
Koh, P. W. et al · 2020
Later among the works it cites.
Concept whitening for interpretable image recognition
Chen, Z., Bei, Y. & Rudin, C · 2020
Later among the works it cites.
Explainable artificial intelligence (xai): Concepts, taxonomies, opportunities and challenges toward responsible ai
Arrieta, A. B. et al · 2020
Later among the works it cites.
Bridging the gap: Providing post-hoc symbolic explanations for sequential decision-making problems with black box simulators
Original
Sreedharan, S., Soni, U., Verma, M., Srivastava, S. & Kambhampati, S · 2020
Later among the works it cites.
Concept enforcement and modularization as methods for the iso 26262 safety argumentation of neural networks (2020)
Schwalbe, G. & Schels, M · 2020
Later among the works it cites.
Zoom in: An introduction to circuits
Olah, C. et al · 2020
Later among the works it cites.
Thread: Circuits
Cammarata, N. et al · 2020
Later among the works it cites.
Tldr: Policy summarization for factored ssp problems using temporal abstractions
Sreedharan, S., Srivastava, S. & Kambhampati, S · 2020
Later among the works it cites.
Interestingness elements for explainable reinforcement learning: Understanding agents’ capabilities and limitations
Sequeira, P. & Gervasio, M · 2020
Later among the works it cites.
Aligning superhuman ai with human behavior: Chess as a model system
McIlroy-Young, R., Sen, S., Kleinberg, J. & Anderson, A · 2020
Later among the works it cites.
Learning to play the chess variant Crazyhouse above world champion level with deep neural networks and human data
Czech, J., Willig, M., Beyer, A., Kersting, K. & Fürnkranz, J · 2020
Later among the works it cites.
Restricting the flow: Information bottlenecks for attribution
Original
Schulz, K., Sixt, L., Tombari, F. & Landgraf, T · 2020
Later among the works it cites.
Information-theoretic probing with minimum description length
Original
Voita, E. & Titov, I · 2020
Later among the works it cites.
Debiasing concept-based explanations with causal analysis
Bahadori, M. T. & Heckerman, D · 2020
Later among the works it cites.
Understanding rl vision
Hilton, J., Cammarata, N., Carter, S., Goh, G. & Olah, C · 2020
Later among the works it cites.
Towards falsifiable interpretability research
Original
Leavitt, M. L. & Morcos, A · 2020
Later among the works it cites.
Multimodal neurons in artificial neural networks
Goh, G. et al · 2021
Closest in time.
Concept-based model explanations for electronic health records
Mincu, D. et al · 2021
Closest in time.
Promises and pitfalls of black-box concept learning models
Original
Mahinpei, A., Clark, J., Lage, I., Doshi-Velez, F. & Pan, W · 2021
Closest in time.
A bayesian framework for information-theoretic probing
Original
Pimentel, T. & Cotterell, R · 2021
Closest in time.
Chessbase Mega Database
ChessBase · 2021
Closest in time.
Where, when & which concepts does AlphaZero learn? Lessons from the game of Hex
Forde, J. Z., Lovering, C., Konidaris, G., Pavlick, E. & Littman, M. L · 2022
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…