2015

Using PCA to Efficiently Represent State Spaces

Curran, William, Brys, Tim, Taylor, Matthew et al.

Understand

Reinforcement learning algorithms need to deal with the exponential growth of states and actions when exploring optimal control in high-dimensional spaces.

  • This is known as the curse of dimensionality.
  • By projecting the agent's state onto a low-dimensional manifold, we can represent the state space in a smaller and more efficient representation.
  • By using this representation during learning, the agent can converge to a good policy much faster.

Built on

  • Face recognition using eigenfaces

    M.A. Turk and A.P. Pentland · 1991

    Earlier work this paper cites.

  • A tutorial on principal component analysis

    Jonathon Shlens · 2005

    Earlier work this paper cites.

  • Dimensional reduction for reward-based learning, 2005

    Christian D. Swinehart and L. F. Abbott · 2005

    Earlier work this paper cites.

  • A survey of robot learning from demonstration

    Brenna D. Argall, Sonia Chernova, Manuela Veloso, and Brett Browning · 2008

    Earlier work this paper cites.

Similar

  • A survey on transfer learning

    Sinno Jialin Pan and Qiang Yang · 2009

    Cited alongside, same era.

  • Compressive reinforcement learning with oblique random projections, 2011

    Bo Liu and Sridhar Mahadevan · 2011

    Cited alongside, same era.

  • The Mario AI benchmark and competitions

    Sergey Karakovskiy and Julian Togelius · 2012

    Cited alongside, same era.

  • Cs229 final report reinforcement learning to play mario

    Yizheng Liao, Kun Yi, and Zhe Yang · 2012

    Cited alongside, same era.

  • Reinforcement learning with reward shaping and mixed resolution function approximation

    M. Grzes and D. Kudenko

    Cited in the paper.

Then

  • Multi-objectivization of reinforcement learning problems by reward shaping

    Tim Brys, Anna Harutyunyan, Peter Vrancx, Matthew E Taylor, Daniel Kudenko, and Ann Nowé · 2014

    Later among the works it cites.

  • Learning a super mario controller from examples of human play

    Geoffrey Lee, Min Luo, Fabio Zambetta, and Xiaodong Li · 2014

    Later among the works it cites.

  • Reinforcement learning from demonstration through shaping

    Tim Brys, Anna Harutyunyan, Halit Bener Suay, Sonia Chernova, Matthew E. Taylor, and Ann Nowé · 2015

    Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…