Fetching the paper…
Reading the bibliography…
This article provides the first survey of computational models of emotion in reinforcement learning (RL) agents.
Hull CL (1943) Principles of behavior: an introduction to behavior theory. Appleton-Century
1943
Earlier work this paper cites.
Osgood CE, Suci GJ, Tannenbaum PH (1964) The measurement of meaning. University of Illinois Press
1964
Earlier work this paper cites.
Russell JA (1978) Evidence of convergent validity on the dimensions of affect. Journal of personality and social psychology 36(10):1152
1978
Earlier work this paper cites.
Kahneman D, Tversky A (1979) Prospect theory: An analysis of decision under risk. Econometrica: Journal of the Econometric Society pp 263–291
1979
Earlier work this paper cites.
Bozinovski S (1982) A self-learning system using secondary reinforcement. Cybernetics and Systems Research pp 397–402
1982
Earlier work this paper cites.
Thorpe S, Rolls E, Maddison S (1983) The orbitofrontal cortex: neuronal activity in the behaving monkey. Experimental Brain Research 49(1):93–115
1983
Earlier work this paper cites.
Ekman P, Friesen WV, O’Sullivan M, Chan A, Diacoyanni-Tarlatzis I, Heider K, Krause R, LeCompte WA, Pitcairn T, Ricci-Bitti PE, et al (1987) Universals and cultural differences in the judgments of facial expressions of emotion. Journal of personality and social psychology 53(4):712
1987
Earlier work this paper cites.
Sutton RS (1988) Learning to predict by the methods of temporal differences. Machine learning 3(1):9–44
1988
Earlier work this paper cites.
Frijda NH, Kuipers P, Ter Schure E (1989) Relations among emotion, appraisal, and emotional action readiness. Journal of personality and social psychology 57(2):212
1989
Earlier work this paper cites.
Watkins CJCH (1989) Learning from delayed rewards. PhD thesis, University of Cambridge England
1989
Earlier work this paper cites.
Ortony A, Clore GL, Collins A (1990) The cognitive structure of emotions. Cambridge university press
1990
Earlier work this paper cites.
Lazarus RS (1991) Cognition and motivation in emotion. American psychologist 46(4):352
1991
Earlier work this paper cites.
Johnson-Laird PN, Oatley K (1992) Basic emotions, rationality, and folk theory. Cognition and Emotion 6(3-4):201–223
1992
Earlier work this paper cites.
Damasio AR (1994) Descartes’ Error: Emotion, Reason and the Human Brain. Grosset/Putnam
1994
Earlier work this paper cites.
Rolls ET, Baylis LL (1994) Gustatory, olfactory, and visual convergence within the primate orbitofrontal cortex. The Journal of neuroscience 14(9):5437–5452
1994
Earlier work this paper cites.
Rummery GA, Niranjan M (1994) On-line Q-learning using connectionist systems. Tech. rep., University of Cambridge, Department of Engineering
1994
Earlier work this paper cites.
Russell S, Norvig P, Intelligence A (1995) A modern approach. Artificial Intelligence Prentice-Hall, Egnlewood Cliffs 25:27
1995
Earlier work this paper cites.
Bozinovski S, Stojanov G, Bozinovska L (1996) Emotion, embodiment, and consequence driven systems. In: Proc AAAI Fall Symposium on Embodied Cognition and Action, pp 12–17
1996
Earlier work this paper cites.
Shibata T, Yoshida M, Yamato J (1997) Artificial emotional creature for human-machine interaction. In: Systems, Man, and Cybernetics, 1997. Computational Cybernetics and Simulation., 1997 IEEE International Conference on, IEEE, vol 3, pp 2269–2274
1997
Earlier work this paper cites.
Balkenius C, Morén J (1998) A computational model of emotional conditioning in the brain. In: Proceedings of Workshop on Grounding Emotions in Adaptive Systems, Zurich
1998
Earlier work this paper cites.
Gadanho S, Hallam J (1998) Emotion triggered learning for autonomous robots. DAI Research Paper
1998
Earlier work this paper cites.
Sutton RS, Barto AG (1998) Reinforcement learning: An introduction. MIT press Cambridge
1998
Earlier work this paper cites.
Velasquez J (1998) Modeling emotion-based decision-making. Emotional and intelligent: The tangled knot of cognition pp 164–169
1998
Earlier work this paper cites.
Ng AY, Harada D, Russell S (1999) Policy invariance under reward transformations: Theory and application to reward shaping. In: ICML, vol 99, pp 278–287
1999
Earlier work this paper cites.
Russell JA, Barrett LF (1999) Core affect, prototypical emotional episodes, and other things called emotion: dissecting the elephant. Journal of personality and social psychology 76(5):805
1999
Earlier work this paper cites.
Scherer KR (1999) Appraisal theory. Handbook of cognition and emotion pp 637–663
1999
Earlier work this paper cites.
Doya K (2000) Metalearning, neuromodulation, and emotion. Affective minds 101
2000
Earlier work this paper cites.
El-Nasr MS, Yen J, Ioerger TR (2000) Flame—fuzzy logic adaptive model of emotions. Autonomous Agents and Multi-agent systems 3(3):219–257
2000
Earlier work this paper cites.
Moren J, Balkenius C (2000) A computational model of emotional learning in the amygdala. From animals to animats 6:115–124
2000
Earlier work this paper cites.
Gadanho SC, Hallam J (2001) Robot learning driven by emotions. Adaptive Behavior 9(1):42–64
2001
Earlier work this paper cites.
Scherer KR, Schorr A, Johnstone T (2001) Appraisal processes in emotion: Theory, methods, research. Oxford University Press
2001
Earlier work this paper cites.
Doya K (2002) Metalearning and neuromodulation. Neural Networks 15(4):495–506
2002
Earlier work this paper cites.
Gmytrasiewicz PJ, Lisetti CL (2002) Emotions and personality in agent design and modeling. In: Game theory and decision theory in agent-based systems, Springer, pp 81–95
2002
Earlier work this paper cites.
Michaud F (2002) EMIB—Computational Architecture Based on Emotion and Motivation for In-tentional Selection and Configuration of Behaviour-Producing Modules. Cognitive Science Quarterly 3-4:340–361
2002
Earlier work this paper cites.
Murphy RR, Lisetti CL, Tardif R, Irish L, Gage A (2002) Emotion-based control of cooperating heterogeneous mobile robots. Robotics and Automation, IEEE Transactions on 18(5):744–757
2002
Earlier work this paper cites.
Tsankova DD (2002) Emotionally influenced coordination of behaviors for autonomous mobile robots. In: Intelligent Systems, 2002. Proceedings. 2002 First International IEEE Symposium, IEEE, vol 1, pp 92–97
2002
Earlier work this paper cites.
2002
Earlier work this paper cites.
Breazeal C (2003) Emotion and sociable humanoid robots. International Journal of Human-Computer Studies 59(1):119–155
2003
Earlier work this paper cites.
Cañamero D (2003) Designing emotions for activity selection in autonomous agents. Emotions in humans and artifacts 115:148
2003
Earlier work this paper cites.
Fong T, Nourbakhsh I, Dautenhahn K (2003) A survey of socially interactive robots. Robotics and autonomous systems 42(3):143–166
2003
Earlier work this paper cites.
Gadanho SC (2003) Learning behavior-selection by emotions and cognition in a multi-goal robot task. The journal of machine learning research 4:385–412
2003
Earlier work this paper cites.
LeDoux J (2003) The emotional brain, fear, and the amygdala. Cellular and molecular neurobiology 23(4-5):727–738
2003
Earlier work this paper cites.
Schweighofer N, Doya K (2003) Meta-learning in reinforcement learning. Neural Networks 16(1):5–9
2003
Earlier work this paper cites.
Ayesh A (2004) Emotionally motivated reinforcement learning based controller. In: Systems, Man and Cybernetics, 2004 IEEE International Conference on, IEEE, vol 1, pp 874–878
2004
Cited alongside, same era.
Belavkin RV (2004) On relation between emotion and entropy. In: Proceedings of the AISB’04 Symposium on Emotion, Cognition and Affective Computing., AISB Press, pp 1–8
2004
Cited alongside, same era.
Chentanez N, Barto AG, Singh SP (2004) Intrinsically motivated reinforcement learning. In: Advances in neural information processing systems, pp 1281–1288
2004
Cited alongside, same era.
Doshi P, Gmytrasiewicz P (2004) Towards affect-based approximations to rational planning: a decision-theoretic perspective to emotions. In: Working Notes of the Spring Symposium on Architectures for Modeling Emotion: Cross-Disciplinary Foundations
2004
Cited alongside, same era.
Bratman J, Singh S, Sorg J, Lewis R (2012) Strong mitigation: Nesting search for good policies within search for good reward. In: Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems-Volume 1, International Foundation for Autonomous Agents and Multiagent Systems, pp 407–414
2012
Later among the works it cites.
Hester T, Stone P (2012a) Intrinsically motivated model learning for a developing curious agent. In: 2012 IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL), IEEE, pp 1–6
2012
Later among the works it cites.
Huang X, Du C, Peng Y, Wang X, Liu J (2012) Goal-oriented action planning in partially observable stochastic domains. In: Cloud Computing and Intelligent Systems (CCIS), 2012 IEEE 2nd International Conference on, IEEE, vol 3, pp 1381–1385
2012
Later among the works it cites.
Knox WB, Glass B, Love B, Maddox WT, Stone P (2012) How Humans Teach Agents. International Journal of Social Robotics 4(4):409–421, DOI
2012
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Tanaka F, Noda K, Sawada T, Fujita M (2004) Associated emotion and its expression in an entertainment robot QRIO. In: Entertainment Computing–ICEC 2004, Springer, pp 499–504
2004
Cited alongside, same era.
Blanchard AJ, Canamero L (2005) From imprinting to adaptation: Building a history of affective interaction. In: Proceedings of the 5th International Workshop on Epigenetic Robotics, Lund University Cognitive Studies, pp 23–30
2005
Cited alongside, same era.
Coutinho E, Miranda ER, Cangelosi A (2005) Towards a model for embodied emotions. In: Artificial intelligence, 2005. epia 2005. portuguese conference on, IEEE, pp 54–63
2005
Cited alongside, same era.
Dunn BD, Dalgleish T, Lawrence AD (2006) The somatic marker hypothesis: A critical evaluation. Neuroscience & Biobehavioral Reviews 30(2):239–271, DOI
2005
Cited alongside, same era.
Lahnstein M (2005) The emotive episode is a composition of anticipatory and reactive evaluations. In: Proc. of the AISB’05 Symposium on Agents that Want and Like, pp 62–69
2005
Cited alongside, same era.
Ahn H, Picard RW (2006) Affective cognitive learning and decision making: The role of emotions. In: EMCSR 2006: The 18th Europ. Meet. on Cyber. and Syst. Res
2006
Cited alongside, same era.
Cañamero D (1997a) A hormonal model of emotions for behavior control. VUB AI-Lab Memo 2006
2006
Cited alongside, same era.
Goerke N (2006) EMOBOT: a robot control architecture based on emotion-like internal values. INTECH Open Access Publisher
2006
Cited alongside, same era.
Later among the works it cites.
Kober J, Peters J (2012) Reinforcement learning in robotics: A survey. In: Reinforcement Learning, Springer, pp 579–610
2012
Later among the works it cites.
Obayashi M, Takuno T, Kuremoto T, Kobayashi K (2012) An emotional model embedded reinforcement learning system. In: Systems, Man, and Cybernetics (SMC), 2012 IEEE International Conference on, IEEE, pp 1058–1063
2012
Later among the works it cites.
Rumbell T, Barnden J, Denham S, Wennekers T (2012) Emotions in autonomous agents: comparative analysis of mechanisms and functions. Autonomous Agents and Multi-Agent Systems 25(1):1–45
2012
Later among the works it cites.
Salichs MA, Malfaz M (2012) A new approach to modeling emotions and their use on a decision-making system for artificial agents. Affective Computing, IEEE Transactions on 3(1):56–68
2012
Later among the works it cites.
Shi X, Wang Z, Zhang Q (2012) Artificial Emotion Model Based on Neuromodulators and Q-learning. In: Future Control and Automation, Springer, pp 293–299
2012
Later among the works it cites.
Vinciarelli A, Pantic M, Heylen D, Pelachaud C, Poggi I, D’Errico F, Schröder M (2012) Bridging the gap between social animal and unsocial machine: A survey of social signal processing. Affective Computing, IEEE Transactions on 3(1):69–87
2012
Later among the works it cites.
Von Haugwitz R, Kitamura Y, Takashima K (2012) Modulating reinforcement-learning parameters using agent emotions. In: Soft Computing and Intelligent Systems (SCIS) and 13th International Symposium on Advanced Intelligent Systems (ISIS), 2012 Joint 6th International Conference on, IEEE, pp 1281–1285
2012
Later among the works it cites.
Wiering M, Van Otterlo M (2012) Reinforcement learning. Adaptation, Learning, and Optimization 12
2012
Later among the works it cites.
Broekens J, Bosse T, Marsella SC (2013) Challenges in Computational Modeling of Affective Processes. Affective Computing, IEEE Transactions on 4(3):242–245
2013
Later among the works it cites.
Castro-González Á, Malfaz M, Salichs MA (2013) An autonomous social robot in fear. Autonomous Mental Development, IEEE Transactions on 5(2):135–151
2013
Later among the works it cites.
Cos I, Cañamero L, Hayes GM, Gillies A (2013) Hedonic value: enhancing adaptation for motivated agents. Adaptive Behavior p 1059712313486817
2013
Later among the works it cites.
Hoey J, Schroder T, Alhothali A (2013) Bayesian affect control theory. In: Affective Computing and Intelligent Interaction (ACII), 2013 Humaine Association Conference on, IEEE, pp 166–172
2013
Later among the works it cites.
Joffily M, Coricelli G (2013) Emotional valence and the free-energy principle. PLOS Computational Biology 9(6)
2013
Later among the works it cites.
Knox WB, Stone P, Breazeal C (2013) Training a Robot via Human Feedback: A Case Study, Lecture Notes in Computer Science, vol 8239, Springer International Publishing, book section 46, pp 460–470
2013
Later among the works it cites.
Kuremoto T, Tsurusaki T, Kobayashi K, Mabu S, Obayashi M (2013) An improved reinforcement learning system using affective factors. Robotics 2(3):149–164
2013
Later among the works it cites.
Moussa MB, Magnenat-Thalmann N (2013) Toward socially responsible agents: integrating attachment and learning in emotional decision-making. Computer Animation and Virtual Worlds 24(3-4):327–334
2013
Later among the works it cites.
Yu C, Zhang M, Ren F (2013) Emotional multiagent reinforcement learning in social dilemmas. In: PRIMA 2013: Principles and Practice of Multi-Agent Systems, Springer, pp 372–387
2013
Later among the works it cites.
Calvo R, D’Mello S, Gratch J, Kappas A (2014) Handbook of affective computing
2014
Later among the works it cites.
Franklin S, Madl T, D’mello S, Snaider J (2014) LIDA: A systems-level architecture for cognition, emotion, and learning. Autonomous Mental Development, IEEE Transactions on 6(1):19–41
2014
Later among the works it cites.
Gratch J, Marsella S (2014) Appraisal models. Oxford University Press
2014
Later among the works it cites.
Jacobs E, Broekens J, Jonker CM (2014) Emergent dynamics of joy, distress, hope and fear in reinforcement learning agents. In: Adaptive Learning Agents workshop at AAMAS2014
2014
Later among the works it cites.
Lisetti C, Hudlicka E (2014) Why and How to Build Emotion-based Agent Architectures
2014
Later among the works it cites.
Sequeira P, Melo FS, Paiva A (2014) Learning by appraising: an emotion-based approach to intrinsic reward design. Adaptive Behavior p 1059712314543837
2014
Later among the works it cites.
Ficocelli M, Terao J, Nejat G (2015) Promoting Interactions Between Humans and Robots Using Robotic Emotional Behavior. IEEE Transactions on Cybernetics pp 1–13
2015
Later among the works it cites.
Hoey J, Schröder T (2015) Bayesian Affect Control Theory of Self. In: AAAI, Citeseer, pp 529–536
2015
Later among the works it cites.
Kim KH, Cho SB (2015) A Group Emotion Control System based on Reinforcement Learning. SoCPaR 2015
2015
Later among the works it cites.
Lhommet M, Marsella SC (2015) Expressing Emotion Through Posture and Gesture, Oxford University Press, pp 273–285
2015
Later among the works it cites.
Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G, et al (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529–533
2015
Later among the works it cites.
Ochs M, Niewiadomski R, Pelachaud C (2015) Facial Expressions of Emotions for Virtual Characters, Oxford University Press, pp 261–272
2015
Later among the works it cites.
Oh J, Guo X, Lee H, Lewis RL, Singh S (2015) Action-conditional video prediction using deep networks in atari games. In: Advances in Neural Information Processing Systems, pp 2863–2871
2015
Later among the works it cites.
Paiva A, Leite I, Ribeiro T (2015) Emotion modeling for social robots, Oxford University Press, pp 296–308
2015
Later among the works it cites.
Williams H, Lee-Johnson C, Browne WN, Carnegie DA (2015) Emotion inspired adaptive robotic path planning. In: Evolutionary Computation (CEC), 2015 IEEE Congress on, IEEE, pp 3004–3011
2015
Later among the works it cites.
Yu C, Zhang M, Ren F, Tan G (2015) Emotional Multiagent Reinforcement Learning in Spatial Social Dilemmas. IEEE Transactions on Neural Networks and Learning Systems 26(12):3083–3096
2015
Later among the works it cites.
Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press
2016
Later among the works it cites.
Houthooft R, Chen X, Duan Y, Schulman J, De Turck F, Abbeel P (2016) Curiosity-driven Exploration in Deep Reinforcement Learning via Bayesian Neural Networks. arXiv preprint arXiv:160509674
2016
Later among the works it cites.
Moerland TM, Broekens J, Jonker CM (2016) Fear and Hope Emerge from Anticipation in Model-based Reinforcement Learning. In: Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), pp 848–854
2016
Later among the works it cites.
Silver D, Huang A, Maddison CJ, Guez A, Sifre L, Van Den Driessche G, Schrittwieser J, Antonoglou I, Panneershelvam V, Lanctot M, et al (2016) Mastering the game of Go with deep neural networks and tree search. Nature 529(7587):484–489
2016
Later among the works it cites.
Moerland TM, Broekens J, Jonker CM (2017) Learning Multimodal Transition Dynamics for Model-Based Reinforcement Learning. arXiv preprint arXiv:170500470
2017
Closest in time.