The Chess Transformer: Mastering Play using Generative Language Models

Understand

This work demonstrates that natural language transformers can support more generic strategic modeling, particularly for text-archived games.

• In addition to learning natural language skills, the abstract transformer architecture can generate meaningful moves on a chessboard.
• With further fine-tuning, the transformer learns complex gameplay by training on 2.8 million chess games in Portable Game Notation.
• After 30,000 training steps, OpenAI's Generative Pre-trained Transformer (GPT-2) optimizes weights for 774 million parameters.