Fetching the paper…
Reading the bibliography…
Conversational agents show the promise to allow users to interact with mobile devices using language.
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019 · 1910
Earlier work this paper cites.
“Put-that-there” Voice and gesture at the graphics interface. In Proceedings of the 7th annual conference on Computer graphics and interactive techniques . 262–270
Richard A Bolt. 1980 · 1980
Earlier work this paper cites.
Principles of mixed-initiative user interfaces. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems . 159–166
Eric Horvitz. 1999 · 1999
Earlier work this paper cites.
Conversational interface technologies
Clare-Marie Karat, John Vergo, and David Nahamoo. 2002 · 2002
Earlier work this paper cites.
Language Models are Few-Shot Learners
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020 · 2005
Earlier work this paper cites.
Big Bird: Transformers for Longer Sequences
Manzil Zaheer, Guru Guruganesh, Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontanon, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, and Amr Ahmed. 2020 · 2007
Earlier work this paper cites.
Vqa: Visual question answering. In Proceedings of the IEEE international conference on computer vision . 2425–2433
Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zitnick, and Devi Parikh. 2015 · 2015
Earlier work this paper cites.
Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
Song Han, Huizi Mao, and William J. Dally. 2015 · 2015
Earlier work this paper cites.
Distilling the Knowledge in a Neural Network
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015 · 2015
Earlier work this paper cites.
Squad: 100,000+ questions for machine comprehension of text
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016 · 2016
Earlier work this paper cites.
Rico: A Mobile App Dataset for Building Data-Driven Design Applications. In Proceedings of the 30th Annual Symposium on User Interface Software and Technology (UIST ’17)
Biplab Deka, Zifeng Huang, Chad Franzen, Joshua Hibschman, Daniel Afergan, Yang Li, Jeffrey Nichols, and Ranjitha Kumar. 2017 · 2017
Earlier work this paper cites.
Chatbots and the new world of HCI
Asbjørn Følstad and Petter Bae Brandtzæg. 2017 · 2017
Earlier work this paper cites.
SUGILITE: creating multimodal smartphone automation by demonstration. In Proceedings of the 2017 CHI conference on human factors in computing systems . 6038–6049
Toby Jia-Jun Li, Amos Azaria, and Brad A Myers. 2017 · 2017
Earlier work this paper cites.
Challenges of situational impairments during interaction with mobile devices. In Proceedings of the 29th Australian Conference on Computer-Human Interaction . 477–481
Zhanna Sarsenbayeva, Niels van Berkel, Chu Luo, Vassilis Kostakos, and Jorge Goncalves. 2017 · 2017
Earlier work this paper cites.
APPINITE: A Multi-Modal Interface for Specifying Data Descriptions in Programming by Demonstration Using Natural Language Instructions. In 2018 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) . 105–114
Toby Jia-Jun Li, Igor Labutov, Xiaohan Nancy Li, Xiaoyi Zhang, Wenze Shi, Wanling Ding, Tom M. Mitchell, and Brad A. Myers. 2018 · 2018
Earlier work this paper cites.
Kite: Building Conversational Bots from Mobile Apps. In Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services (Munich, Germany) (MobiSys ’18) . Association for Computing Machinery, New York, NY, USA, 96–109
Toby Jia-Jun Li and Oriana Riva. 2018 · 2018
Earlier work this paper cites.
Reinforcement Learning on Web Interfaces Using Workflow-Guided Exploration
Evan Zheran Liu, Kelvin Guu, Panupong Pasupat, Tianlin Shi, and Percy Liang. 2018 · 2018
Earlier work this paper cites.
Mapping natural language commands to web elements. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing . Association for Computational Linguistics, Brussels, Belgium, 4970–4976
Panupong Pasupat, Tian-Shun Jiang, Evan Liu, Kelvin Guu, and Percy Liang. 2018 · 2018
Earlier work this paper cites.
PUMICE: A Multi-Modal Agent That Learns Concepts and Conditionals from Natural Language and Demonstrations. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology (New Orleans, LA, USA) (UIST ’19) . Association for Computing Machinery, New York, NY, USA, 577–589
Toby Jia-Jun Li, Marissa Radensky, Justin Jia, Kirielle Singarajah, Tom M. Mitchell, and Brad A. Myers. 2019 · 2019
Cited alongside, same era.
Situationally aware mobile devices for overcoming situational impairments. In Proceedings of the ACM SIGCHI Symposium on Engineering Interactive Computing Systems . 1–18
Jacob O Wobbrock. 2019 · 2019
Cited alongside, same era.
Mapping Natural Language Instructions to Mobile UI Action Sequences. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics . Association for Computational Linguistics, Online, 8198–8210
Yang Li, Jiacong He, Xin Zhou, Yuan Zhang, and Jason Baldridge. 2020b · 2020
Cited alongside, same era.
Widget Captioning: Generating Natural Language Description for Mobile User Interface Elements
Yang Li, Gang Li, Luheng He, Jingjie Zheng, Hong Li, and Zhiwei (Wei) Guan. 2020c · 2020
ScreenQA: Large-Scale Question-Answer Pairs over Mobile App Screenshots
Yu-Chung Hsiao, Fedir Zubach, Maria Wang, and Chen Jindong. 2022 · 2022
Closest in time.
Survey of Hallucination in Natural Language Generation
Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Yejin Bang, Andrea Madotto, and Pascale Fung. 2022 · 2022
Closest in time.
PromptMaker: Prompt-Based Prototyping with Large Language Models. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI EA ’22) . Association for Computing Machinery, New York, NY, USA, Article 35, 8 pages
Ellen Jiang, Kristen Olson, Edwin Toh, Alejandra Molina, Aaron Donsbach, Michael Terry, and Carrie J Cai. 2022a · 2022
Closest in time.
Discovering the Syntax and Strategies of Natural Language Programming with Generative Language Models. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22) . Association for Computing Machinery, New York, NY, USA, Article 386, 19 pages
Ellen Jiang, Edwin Toh, Alejandra Molina, Kristen Olson, Claire Kayacik, Aaron Donsbach, Carrie J Cai, and Michael Terry. 2022b · 2022
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Cited alongside, same era.
Screen2vec: Semantic embedding of gui screens and gui components. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems . 1–15
Toby Jia-Jun Li, Lindsay Popowski, Tom Mitchell, and Brad A Myers. 2021c · 2021
Cited alongside, same era.
VUT: Versatile UI Transformer for Multi-Modal Multi-Task User Interface Modeling
Yang Li, Gang Li, Xin Zhou, Mostafa Dehghani, and Alexey Gritsenko. 2021a · 2021
Cited alongside, same era.
Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm
Laria Reynolds and Kyle McDonell. 2021 · 2021
Cited alongside, same era.
Conversations with GUIs. In Proceedings of the ACM SIGCHI Conference on Designing Interactive Systems (DIS ’21’) . Association for Computing Machinery, New York, NY, USA
Kashyap Todi, Luis A. Leiva, Daniel Buschek, Pin Tian, and Antti Oulasvirta. 2021 · 2021
Cited alongside, same era.
Screen2Words: Automatic Mobile UI Summarization with Multimodal Learning. In The 34th Annual ACM Symposium on User Interface Software and Technology (Virtual Event, USA) (UIST ’21) . Association for Computing Machinery, New York, NY, USA, 498–510
Bryan Wang, Gang Li, Xin Zhou, Zhourong Chen, Tovi Grossman, and Yang Li. 2021 · 2021
Cited alongside, same era.
Screen Parsing: Towards Reverse Engineering of UI Models from Screenshots. In The 34th Annual ACM Symposium on User Interface Software and Technology (Virtual Event, USA) (UIST ’21) . Association for Computing Machinery, New York, NY, USA, 470–483
Jason Wu, Xiaoyi Zhang, Jeff Nichols, and Jeffrey P Bigham. 2021 · 2021
Cited alongside, same era.
Screen Recognition: Creating Accessibility Metadata for Mobile Applications from Pixels. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21) . Association for Computing Machinery, New York, NY, USA, Article 275, 15 pages
Xiaoyi Zhang, Lilian de Greef, Amanda Swearngin, Samuel White, Kyle Murray, Lisa Yu, Qi Shan, Jeffrey Nichols, Jason Wu, Chris Fleizach, Aaron Everitt, and Jeffrey P Bigham. 2021 · 2021
Cited alongside, same era.
ACT-1: Transformer for Actions
Adept. 2022 · 2022
Cited alongside, same era.
Closest in time.
Stylette: Styling the Web with Natural Language. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22) . Association for Computing Machinery, New York, NY, USA, Article 5, 17 pages
Tae Soo Kim, DaEun Choi, Yoonseo Choi, and Juho Kim. 2022 · 2022
Closest in time.
Large Language Models are Zero-Shot Reasoners
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022 · 2022
Closest in time.
CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22) . Association for Computing Machinery, New York, NY, USA, Article 388, 19 pages
Mina Lee, Percy Liang, and Qian Yang. 2022c · 2022
Closest in time.
Promptiverse: Scalable Generation of Scaffolding Prompts Through Human-AI Hybrid Knowledge Graph Annotation. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22) . Association for Computing Machinery, New York, NY, USA, Article 96, 18 pages
Yoonjoo Lee, John Joon Young Chung, Tae Soo Kim, Jean Y Song, and Juho Kim. 2022a · 2022
Closest in time.
Interactive Children’s Story Rewriting Through Parent-Children Interaction. In Proceedings of the First Workshop on Intelligent and Interactive Writing Assistants (In2Writing 2022) . 62–71
Yoonjoo Lee, Tae Soo Kim, Minsuk Chang, and Juho Kim. 2022b · 2022
Closest in time.
Describing UI Screenshots in Natural Language
Luis A. Leiva, Asutosh Hota, and Antti Oulasvirta. 2022 · 2022
Closest in time.
Learning to Denoise Raw Mobile UI Layouts for Improving Datasets at Scale. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22) . Association for Computing Machinery, New York, NY, USA, Article 67, 13 pages
Gang Li, Gilles Baechler, Manuel Tragut, and Yang Li. 2022a · 2022
Closest in time.
Will AI Console Me When I Lose My Pet? Understanding Perceptions of AI-Mediated Email Writing. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22) . Association for Computing Machinery, New York, NY, USA, Article 474, 13 pages
Yihe Liu, Anushk Mittal, Diyi Yang, and Amy Bruckman. 2022 · 2022
Closest in time.
CHATGPT: Optimizing language models for dialogue
OpenAI. 2022 · 2022
Closest in time.
Rationale-Augmented Ensembles in Language Models
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, and Denny Zhou. 2022 · 2022
Closest in time.
Emergent abilities of large language models
Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al · 2022
Closest in time.
AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans, LA, USA) (CHI ’22) . Association for Computing Machinery, New York, NY, USA, Article 385, 22 pages
Tongshuang Wu, Michael Terry, and Carrie Jun Cai. 2022 · 2022
Closest in time.
Least-to-Most Prompting Enables Complex Reasoning in Large Language Models
Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Olivier Bousquet, Quoc Le, and Ed Chi. 2022 · 2022
Closest in time.