Fetching the paper…
Reading the bibliography…
We reframe the analysis of progress in AI by incorporating into an overall framework both the task performance of a system, and the time and resource costs incurred in the development and deployment of the system.
A general empirical solution to the macro software sizing and estimating problem
Lawrence H. Putnam · 1978
Earlier work this paper cites.
Gradient-based learning applied to document recognition
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner · 1998
Earlier work this paper cites.
Embodied conversational agents
Justine Cassell, Joseph Sullivan, Elizabeth Churchill, and Scott Prevost · 2000
Earlier work this paper cites.
The foundations of cost-sensitive learning
Charles Elkan · 2001
Earlier work this paper cites.
Types of cost in inductive concept learning
Peter D Turney · 2002
Earlier work this paper cites.
A detailed cost model for concurrent use with hardware/software co-design
Daniel Ragan, Peter Sandborn, and Paul Stoaks · 2002
Earlier work this paper cites.
Customer value propositions in business markets
James C Anderson, James A Narus, and Wouter Van Rossum · 2006
Earlier work this paper cites.
Universal intelligence: A definition of machine intelligence
Shane Legg and Marcus Hutter · 2007
Earlier work this paper cites.
Self-driving cars and the urban challenge
Chris Urmson et al · 2008
Earlier work this paper cites.
Replicability is not reproducibility: nor is it good science
C. Drummond · 2009
Earlier work this paper cites.
What the no free lunch theorems really mean; how to improve search algorithms
David H Wolpert · 2012
Earlier work this paper cites.
Imagenet classification with deep convolutional neural networks
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton · 2012
Earlier work this paper cites.
Introduction to "this is watson"
D. A. Ferrucci · 2012
Earlier work this paper cites.
Investigating contingency awareness using atari 2600 games
Marc G Bellemare, Joel Veness, and Michael Bowling · 2012
Earlier work this paper cites.
The arcade learning environment: An evaluation platform for general agents
M. G. Bellemare, Y. Naddaf, J. Veness, and M. Bowling · 2013
Earlier work this paper cites.
Min Lin, Qiang Chen, and Shuicheng Yan · 2013
Earlier work this paper cites.
Towards a viable autonomous driving research platform
Junqing Wei, Jarrod M Snider, Junsung Kim, John M Dolan, Raj Rajkumar, and Bakhtiar Litkouhi · 2013
Earlier work this paper cites.
The arcade learning environment: An evaluation platform for general agents
Marc G Bellemare, Yavar Naddaf, Joel Veness, and Michael Bowling · 2013
Earlier work this paper cites.
Playing atari with deep reinforcement learning
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller · 2013
Earlier work this paper cites.
The economics of Information Systems and software
Richard Veryard · 2014
Earlier work this paper cites.
Diannao: A small-footprint high-throughput accelerator for ubiquitous machine learning
Tianshi Chen, Zidong Du, Ninghui Sun, Jia Wang, Chengyong Wu, Yunji Chen, and Olivier Temam · 2014
Earlier work this paper cites.
Very deep convolutional networks for large-scale image recognition
Karen Simonyan and Andrew Zisserman · 2014
Earlier work this paper cites.
The google car: driving toward a better future?
Sharon L Poczter and Luka M Jankovic · 2014
Earlier work this paper cites.
Toward replicable and measurable robotics research
Fabio Bonsignorio and Angel P Del Pobil · 2015
Earlier work this paper cites.
Hidden technical debt in machine learning systems
D Sculley, Gary Holt, Daniel Golovin, Eugene Davydov, Todd Phillips, Dietmar Ebner, Vinay Chaudhary, Michael Young, Jean-Francois Crespo, and Dan Dennison · 2015
Earlier work this paper cites.
Software engineering
Ian Sommerville · 2015
Earlier work this paper cites.
Imagenet large scale visual recognition challenge
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al · 2015
Earlier work this paper cites.
Going deeper with convolutions
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich · 2015
Earlier work this paper cites.
Automatic online evaluation of intelligent assistants
Jiepu Jiang, Ahmed Hassan Awadallah, Rosie Jones, Umut Ozertem, Imed Zitouni, Ranjitha Gurunath Kulkarni, and Omar Zia Khan · 2015
Earlier work this paper cites.
Road safety with self-driving vehicles: General limitations and road sharing with conventional vehicles
Michael Sivak and Brandon Schoettle · 2015
Earlier work this paper cites.
Human-level control through deep reinforcement learning
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al · 2015
Earlier work this paper cites.
Massively parallel methods for deep reinforcement learning
Arun Nair, Praveen Srinivasan, Sam Blackwell, Cagdas Alcicek, Rory Fearon, Alessandro De Maria, Vedavyas Panneershelvam, Mustafa Suleyman, Charles Beattie, Stig Petersen, et al · 2015
Earlier work this paper cites.
Tom Schaul, John Quan, Ioannis Antonoglou, and David Silver · 2015
Earlier work this paper cites.
Dueling network architectures for deep reinforcement learning
Ziyu Wang, Tom Schaul, Matteo Hessel, Hado Van Hasselt, Marc Lanctot, and Nando De Freitas · 2015
Earlier work this paper cites.
Ai results for the atari 2600 games: difficulty and discrimination using irt
Fernando Martınez-Plumed and José Hernández-Orallo · 2016
Earlier work this paper cites.
Making sense of item response theory in machine learning
Fernando Martínez-Plumed, Ricardo BC Prudêncio, Adolfo Martínez-Usó, and José Hernández-Orallo · 2016
Earlier work this paper cites.
Miles Brundage · 2016
Earlier work this paper cites.
Mastering the game of go with deep neural networks and tree search
David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al · 2016
Earlier work this paper cites.
imagenet-validation.torch
Sergey Zagoruyko · 2016
Earlier work this paper cites.
Enet: A deep neural network architecture for real-time semantic segmentation
Adam Paszke, Abhishek Chaurasia, Sangpil Kim, and Eugenio Culurciello · 2016
Earlier work this paper cites.
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun · 2016
Cited alongside, same era.
Rethinking the inception architecture for computer vision
Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna · 2016
Cited alongside, same era.
Squeezenet: Alexnet-level accuracy with 50x fewer parameters and< 0.5 mb model size
Forrest N Iandola, Song Han, Matthew W Moskewicz, Khalid Ashraf, William J Dally, and Kurt Keutzer · 2016
Cited alongside, same era.
An analysis of deep neural network models for practical applications
Alfredo Canziani, Adam Paszke, and Eugenio Culurciello · 2016
Cited alongside, same era.
Learning to learn by gradient descent by gradient descent
Marcin Andrychowicz, Misha Denil, Sergio Gomez Colmenarejo, Matthew W. Hoffman, David Pfau, Tom Schaul, and Nando de Freitas · 2016
Advances in AI require progress across all of computer science
Gregory D Hager, Randal Bryant, Eric Horvitz, Maja Mataric, and Vasant Honavar · 2017
Later among the works it cites.
# exploration: A study of count-based exploration for deep reinforcement learning
Haoran Tang, Rein Houthooft, Davis Foote, Adam Stooke, OpenAI Xi Chen, Yan Duan, John Schulman, Filip DeTurck, and Pieter Abbeel · 2017
Later among the works it cites.
Count-based exploration with neural density models
Georg Ostrovski, Marc G Bellemare, Aaron van den Oord, and Rémi Munos · 2017
Later among the works it cites.
Evolution strategies as a scalable alternative to reinforcement learning
Tim Salimans, Jonathan Ho, Xi Chen, Szymon Sidor, and Ilya Sutskever · 2017
Later among the works it cites.
Count-based exploration in feature space for reinforcement learning
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Deep reinforcement learning with double q-learning
Hado Van Hasselt, Arthur Guez, and David Silver · 2016
Cited alongside, same era.
Asynchronous methods for deep reinforcement learning
Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu · 2016
Cited alongside, same era.
Learning values across many orders of magnitude
Hado P van Hasselt, Arthur Guez, Matteo Hessel, Volodymyr Mnih, and David Silver · 2016
Cited alongside, same era.
Unifying count-based exploration and intrinsic motivation
Marc Bellemare, Sriram Srinivasan, Georg Ostrovski, Tom Schaul, David Saxton, and Remi Munos · 2016
Cited alongside, same era.
Measuring the progress of AI research, 2017
P Eckersley and N Yomna · 2017
Cited alongside, same era.
AI Index, 2017
Yoav Shoham, Raymond Perrault, Erik Brynjolfsson, Jack Clark, and Calvin LeGassick · 2017
Cited alongside, same era.
A new ai evaluation cosmos: Ready to play the game?
Jose Hernández-Orallo, Marco Baroni, Jordi Bieger, Nader Chmait, David L Dowe, Katja Hofmann, Fernando Martínez-Plumed, Claes Strannegård, and Kristinn R Thórisson · 2017
Cited alongside, same era.
Jarryd Martin, Suraj Narayanan Sasikumar, Tom Everitt, and Marcus Hutter · 2017
Later among the works it cites.
A distributional perspective on reinforcement learning
Marc G Bellemare, Will Dabney, and Rémi Munos · 2017
Later among the works it cites.
Dual indicators to analyze ai benchmarks: Difficulty, discrimination, ability, and generality
Fernando Martinez-Plumed and Jose Hernandez-Orallo · 2018
Closest in time.
Deep learning: A critical appraisal
Gary Marcus · 2018
Closest in time.
Computational power and the social impact of artificial intelligence
Tim Hwang · 2018
Closest in time.
Benchmark suite
MLPerf · 2018
Closest in time.
The facets of artificial intelligence: A framework to track the evolution of ai
Fernando Martínez-Plumed, Bao Sheng Loe, Peter Flach, Seán Ó hÉigeartaigh, Karina Vold, and José Hernández-Orallo · 2018
Closest in time.
Estimate computation costs
MathWorks · 2018
Closest in time.
Planning with pixels in (almost) real time
Wilmer Bandres, Blai Bonet, and Hector Geffner · 2018
Closest in time.
Mobilenetv2: Inverted residuals and linear bottlenecks
Mark Sandler, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang-Chieh Chen · 2018
Closest in time.
Fd-mobilenet: Improved mobilenet with a fast downsampling strategy
Zheng Qin, Zhaoning Zhang, Xiaotao Chen, Changjian Wang, and Yuxing Peng · 2018
Closest in time.
Regularized evolution for image classifier architecture search
Esteban Real, Alok Aggarwal, Yanping Huang, and Quoc V Le · 2018
Closest in time.
Squeeze-and-excitation networks
Jie Hu, Li Shen, and Gang Sun · 2018
Closest in time.
Shufflenet v2: Practical guidelines for efficient cnn architecture design
Ningning Ma, Xiangyu Zhang, Hai-Tao Zheng, and Jian Sun · 2018
Closest in time.
Gpipe: Efficient training of giant neural networks using pipeline parallelism
Yanping Huang, Yonglong Cheng, Dehao Chen, HyoukJoong Lee, Jiquan Ngiam, Quoc V Le, and Zhifeng Chen · 2018
Closest in time.
Alexa, siri, cortana, and more: An introduction to voice assistants
Matthew B Hoy · 2018
Closest in time.
What’s up with privacy?: User preferences and privacy concerns in intelligent personal assistants
Lydia Manikonda, Aditya Deotale, and Subbarao Kambhampati · 2018
Closest in time.
Rating the smarts of the digital personal assistants in 2018
Eric Enge · 2018
Closest in time.
How much electricity does the amazon echo use?
Craig Lloyd · 2018
Closest in time.
How much power does your smart home tech really use?
Andrew Williams · 2018
Closest in time.
Spread of self-driving cars could cause more pollution – unless the electric grid transforms radically
Jennifer Hatch Peter Fox-Penner and Will Gorman · 2018
Closest in time.
Reshaping urban mobility with autonomous vehicles lessons from the city of boston
World Economic Forum · 2018
Closest in time.
The reactor: A fast and sample-efficient actor-critic agent for reinforcement learning
Audrunas Gruslys, Will Dabney, Mohammad Gheshlaghi Azar, Bilal Piot, Marc Bellemare, and Remi Munos · 2018
Closest in time.
Item response theory in ai: Analysing machine learning classifiers at the instance level
Fernando Martínez-Plumed, Ricardo BC Prudêncio, Adolfo Martínez-Usó, and José Hernández-Orallo · 2019
Closest in time.
URL https://dl.acm.org/citation.cfm?id=3287588
Proceedings of the Conference on Fairness, Accountability, and Transparency, FAT* 2019, Atlanta, GA, USA, January 29-31, 2019 , 2019. ACM · 2019
Closest in time.
Google assistant
Google · 2019
Closest in time.
Surveying safety-relevant ai characteristics
José Hernández-Orallo, Fernando Martínez-Plumed, Shahar Avin, et al · 2019
Closest in time.
Tracking ai: The capability is (not) near
Fernando Martínez-Plumed, Jose Hernández-Orallo, and Emilia Gómez · 2020
Closest in time.
Ai paradigms and ai safety: mapping artefacts and techniques to safety issues
Jose Hernández-Orallo, Fernando Martínez-Plumed, Shahar Avin, Jess Whittlestone, et al · 2020
Closest in time.
General intelligence disentangled via a generality metric for natural and artificial intelligence
José Hernández-Orallo, Bao Sheng Loe, Lucy Cheke, Fernando Martínez-Plumed, and Seán Ó hÉigeartaigh · 2021
Closest in time.
Compute and energy consumption trends in deep learning inference
Radosvet Desislavov, Fernando Martínez-Plumed, and José Hernández-Orallo · 2021
Closest in time.
Measuring the occupational impact of ai: tasks, cognitive abilities and ai benchmarks
Songül Tolan, Annarosa Pesole, Fernando Martínez-Plumed, Enrique Fernández-Macías, José Hernández-Orallo, and Emilia Gómez · 2021
Closest in time.
Missing the missing values: The ugly duckling of fairness in machine learning
Martínez-Plumed Fernando, Ferri Cèsar, Nieves David, and Hernández-Orallo José · 2021
Closest in time.
Training on the test set: Mapping the system-problem space in ai
José Hernández-Orallo, Wout Schellaert, and Fernando Martınez-Plumed · 2022
Closest in time.
When ai difficulty is easy: The explanatory power of predicting irt difficulty
Fernando Martınez-Plumed, David Castellano-Falcón, Carlos Monserrat, and José Hernández-Orallo · 2022
Closest in time.