Fetching the paper…
Reading the bibliography…
Recently, large language models (LLM) based generative AI has been gaining momentum for their impressive high-quality performances in multiple domains, particularly after the release of the ChatGPT.
Man-computer symbiosis
Joseph CR Licklider. 1960 · 1960
Earlier work this paper cites.
Statistical Principles in Experimental Design . Vol. 2
Ben James Winer, Donald R Brown, Kenneth M Michels, et al · 1971
Earlier work this paper cites.
Randomization analysis of experimental data: The Fisher randomization test comment
Donald B Rubin. 1980 · 1980
Earlier work this paper cites.
Experimentation in software engineering
Victor R Basili, Richard W Selby, and David H Hutchens. 1986 · 1986
Earlier work this paper cites.
No Silver Bullet–Essence and accidents of software engineering
Frederik P Brooks. 1987 · 1987
Earlier work this paper cites.
Long short-term memory
Sepp Hochreiter and Jürgen Schmidhuber. 1997 · 1997
Earlier work this paper cites.
NaturalJava: a natural language interface for programming in Java. In Proceedings of the 5th International Conference on Intelligent User Interfaces (IUI ’00) . Association for Computing Machinery, New York, NY, USA, 207–211
David Price, Ellen Rilofff, Joseph Zachary, and Brandon Harvey. 2000 · 2000
Earlier work this paper cites.
A critique and improvement of the CL common language effect size statistics of McGraw and Wong
András Vargha and Harold D Delaney. 2000 · 2000
Earlier work this paper cites.
A survey of controlled experiments in software engineering
Dag I. K. Sjoberg, Jo E. Hannay, Ove Hansen, Vigdis By Kampenes, Amela Karahasanovic, Nils-Kristian Liborg, and Anette C. Rekdal. 2005 · 2005
Earlier work this paper cites.
Programming With Unrestricted Natural Language. In Proceedings of the 2005 Australasian Language Technology Association Workshop (ATLA ’05) . Sydney, Australia, 191–199
David Vadas and James R. Curran. 2005 · 2005
Earlier work this paper cites.
NASA-task load index (NASA-TLX); 20 years later. In Proceedings of the 50th Human Factors and Ergonomics Society Annual Meeting (Ergonomics ’06) , Vol. 50. Sage Publications, Los Angeles, CA, USA, 904–908
Sandra G Hart. 2006 · 2006
Earlier work this paper cites.
Reporting Experiments in Software Engineering
Andreas Jedlitschka, Marcus Ciolkowski, and Dietmar Pfahl. 2008 · 2008
Earlier work this paper cites.
Unit Test Case Generation with Transformers
Michele Tufano, Dawn Drain, Alexey Svyatkovskiy, Shao Kun Deng, and Neel Sundaresan. 2020 · 2009
Earlier work this paper cites.
Recurrent neural network based language model. In Proceedings of the 11th Annual Conference of the International Speech Communication Association (Interspeech ’10), Makuhari, Chiba, Japan, September 26-30, 2010 , Takao Kobayashi, Keikichi Hirose, and Satoshi Nakamura (Eds.). ISCA, 1045–1048
Tomas Mikolov, Martin Karafiát, Lukas Burget, Jan Cernocký, and Sanjeev Khudanpur. 2010 · 2010
Earlier work this paper cites.
Wilcoxon-Signed-Rank Test
Denise Rey and Markus Neuhäuser. 2011 · 2011
Earlier work this paper cites.
Experimental Design: Procedures for the Behavioral Sciences (4. ed.)
Roger E Kirk. 2012 · 2012
Earlier work this paper cites.
Screening and treatment of subclinical hypothyroidism or hyperthyroidism
Bruin Rugge, Howard Balshem, Raj Sehgal, Rose Relevo, Paul Gorman, and Mark Helfand. 2012 · 2012
Earlier work this paper cites.
To stay or leave? the relationship of emotional and informational support to commitment in online health support groups. In Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work (CSCW ’12) . Association for Computing Machinery, New York, NY, USA, 833–842
Yi-Chia Wang, Robert Kraut, and John M. Levine. 2012 · 2012
Earlier work this paper cites.
Experimentation in Software Engineering
Claes Wohlin, Per Runeson, Martin Höst, Magnus C Ohlsson, Björn Regnell, and Anders Wesslén. 2012 · 2012
Earlier work this paper cites.
The principles of experimental design and their application in sociology
Michelle Jackson and David R Cox. 2013 · 2013
Earlier work this paper cites.
Basics of Software Engineering Experimentation
Natalia Juristo and Ana M Moreno. 2013 · 2013
Earlier work this paper cites.
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Junyoung Chung, Çaglar Gülçehre, KyungHyun Cho, and Yoshua Bengio. 2014 · 2014
Earlier work this paper cites.
Using Psycho-Physiological Measures to Assess Task Difficulty in Software Development. In Proceedings of the 36th International Conference on Software Engineering (Hyderabad, India) (ICSE 2014) . Association for Computing Machinery, New York, NY, USA, 402–413
Thomas Fritz, Andrew Begel, Sebastian C. Müller, Serap Yigit-Elliott, and Manuela Züger. 2014 · 2014
Earlier work this paper cites.
Are students representatives of professionals in software engineering experiments?. In Proceedings of the 37th International Conference on Software Engineering - Volume 1 (ICSE ’15) . IEEE Press, 666–676
Iflaah Salman, Ayse Tosun Misirli, and Natalia Juristo. 2015 · 2015
Cited alongside, same era.
Latent predictor networks for code generation. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . Association for Computational Linguistics, Berlin, Germany, 599–609
Wang Ling, Phil Blunsom, Edward Grefenstette, Karl Moritz Hermann, Tomáš Kočiský, Fumin Wang, and Andrew Senior. 2016 · 2016
Cited alongside, same era.
Programmers Are Users Too: Human-Centered Methods for Improving Programming Tools
Brad A. Myers, Andrew J. Ko, Thomas D. LaToza, and YoungSeok Yoon. 2016 · 2016
Cited alongside, same era.
Attention is all you need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017 · 2017
Cited alongside, same era.
Perfection Not Required? Human-AI Partnerships in Code Translation. In 26th International Conference on Intelligent User Interfaces (College Station, TX, USA) (IUI ’21) . Association for Computing Machinery, New York, NY, USA, 402–412
Justin D. Weisz, Michael Muller, Stephanie Houde, John Richards, Steven I. Ross, Fernando Martinez, Mayank Agarwal, and Kartik Talamadupula. 2021 · 2021
Later among the works it cites.
Reliance and Automation for Human-AI Collaborative Data Labeling Conflict Resolution
Michelle Brachman, Zahra Ashktorab, Michael Desmond, Evelyn Duesterwald, Casey Dugan, Narendra Nath Joshi, Qian Pan, and Aabhas Sharma. 2022 · 2022
Later among the works it cites.
Considerations and Pitfalls for Reducing Threats to the Validity of Controlled Experiments on Code Comprehension
Dror G. Feitelson. 2022 · 2022
Later among the works it cites.
CoAuthor: Designing a human-AI collaborative writing dataset for exploring language model capabilities. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI ’22) . Association for Computing Machinery, New York, NY, USA, Article 388, 19 pages
Mina Lee, Percy Liang, and Qian Yang. 2022 · 2022
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
Characterizing developer behavior in cloud based IDEs. In Proceedings of the 11th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM ’17) . IEEE Press, Markham, Ontario, Canada, 48–57
Yi Wang. 2017 · 2017
Cited alongside, same era.
A syntactic neural model for general-purpose code generation. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . Association for Computational Linguistics, Vancouver, Canada, 440–450
Pengcheng Yin and Graham Neubig. 2017 · 2017
Cited alongside, same era.
Empirical software engineering experts on the use of students and professionals in experiments
Davide Falessi, Natalia Juristo, Claes Wohlin, Burak Turhan, Jürgen Münch, Andreas Jedlitschka, and Markku Oivo. 2018 · 2018
Cited alongside, same era.
Human-AI collaboration in data science: Exploring data scientists’ perceptions of automated AI
Dakuo Wang, Justin D. Weisz, Michael Muller, Parikshit Ram, Werner Geyer, Casey Dugan, Yla Tausczik, Horst Samulowitz, and Alexander Gray. 2019 · 2019
Cited alongside, same era.
A research agenda for hybrid intelligence: augmenting human intellect with collaborative, adaptive, responsible, and explainable artificial intelligence
Zeynep Akata, Dan Balliet, Maarten de Rijke, Frank Dignum, Virginia Dignum, Guszti Eiben, Antske Fokkens, Davide Grossi, Koen Hindriks, Holger Hoos, Hayley Hung, Catholijn Jonker, Christof Monz, Mark Neerincx, Frans Oliehoek, Henry Prakken, Stefan Schlobach, Linda van der Gaag, Frank van Harmelen, Herke van Hoof, Birna van Riemsdijk, Aimee van Wynsberghe, Rineke Verbrugge, Bart Verheij, Piek Vossen, and Max Welling. 2020 · 2020
Cited alongside, same era.
Human-centered AI: The role of human-centered design research in the development of AI. In Synergy-DRS International Conference 2020 . Online
Jan Auernhammer. 2020 · 2020
Cited alongside, same era.
Wrex: A Unified Programming-by-Example Interaction for Synthesizing Readable Code for Data Scientists. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI ’20) . Association for Computing Machinery, New York, NY, USA, 1–12
Ian Drosos, Titus Barik, Philip J. Guo, Robert DeLine, and Sumit Gulwani. 2020 · 2020
Cited alongside, same era.
CodeBERT: A pre-trained model for programming and natural languages. In Findings of the Association for Computational Linguistics: EMNLP 2020 , Trevor Cohn, Yulan He, and Yang Liu (Eds.). Association for Computational Linguistics, Online, 1536–1547
Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, and Ming Zhou. 2020 · 2020
Cited alongside, same era.
Later among the works it cites.
Training language models to follow instructions with human feedback
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al · 2022
Later among the works it cites.
Transformers for Natural Language Processing: Build, train, and fine-tune deep neural network architectures for NLP with Python, PyTorch, TensorFlow, BERT, and GPT-3 (2. ed.)
Denis Rothman and Antonio Gulli. 2022 · 2022
Later among the works it cites.
Expectation vs. Experience: Evaluating the usability of code generation tools powered by large language models. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems (CHI EA ’22) . Association for Computing Machinery, New York, NY, USA, Article 332, 7 pages
Priyan Vaithilingam, Tianyi Zhang, and Elena L. Glassman. 2022 · 2022
Later among the works it cites.
Better Together? An Evaluation of AI-Supported Code Translation. In 27th International Conference on Intelligent User Interfaces (Helsinki, Finland) (IUI ’22) . Association for Computing Machinery, New York, NY, USA, 369–391
Justin D. Weisz, Michael Muller, Steven I. Ross, Fernando Martinez, Stephanie Houde, Mayank Agarwal, Kartik Talamadupula, and John T. Richards. 2022 · 2022
Later among the works it cites.
The End of Programming
Matt Welsh. 2022 · 2022
Later among the works it cites.
AI chains: Transparent and controllable human-AI interaction by chaining large language model prompts. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI ’22) . Association for Computing Machinery, New York, NY, USA, Article 385, 22 pages
Tongshuang Wu, Michael Terry, and Carrie Jun Cai. 2022 · 2022
Later among the works it cites.
In-IDE Code Generation from Natural Language: Promise and Challenges
Frank F. Xu, Bogdan Vasilescu, and Graham Neubig. 2022 · 2022
Later among the works it cites.
Off to a Good Start: Dynamic Contribution Patterns and Technical Success in an OSS Newcomer’s Early Career
Yang Yue, Yi Wang, and David Redmiles. 2023 · 2022
Later among the works it cites.
Productivity assessment of neural code completion. In Proceedings of the 6th ACM SIGPLAN International Symposium on Machine Programming (MAPS ’22) . Association for Computing Machinery, New York, NY, USA, 21–29
Albert Ziegler, Eirini Kalliamvakou, X. Alice Li, Andrew Rice, Devon Rifkin, Shawn Simister, Ganesh Sittampalam, and Edward Aftandilian. 2022 · 2022
Later among the works it cites.
Generated faces in the wild: Quantitative comparison of stable diffusion, Midjourney and DALL-E 2
Ali Borji. 2023 · 2023
Later among the works it cites.
Studying the Effect of AI Code Generators on Supporting Novice Learners in Introductory Programming. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI ’23) . Association for Computing Machinery, New York, NY, USA, Article 455, 23 pages
Majeed Kazemitabaar, Justin Chow, Carl Ka To Ma, Barbara J. Ericson, David Weintrop, and Tovi Grossman. 2023 · 2023
Later among the works it cites.
“What It Wants Me To Say”: Bridging the Abstraction Gap Between End-User Programmers and Code-Generating Large Language Models. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (Hamburg, Germany) (CHI ’23) . Association for Computing Machinery, New York, NY, USA, Article 598, 31 pages
Michael Xieyang Liu, Advait Sarkar, Carina Negreanu, Benjamin Zorn, Jack Williams, Neil Toronto, and Andrew D. Gordon. 2023 · 2023
Later among the works it cites.
Reading Between the Lines: Modeling User Behavior and Costs in AI-Assisted Programming
Hussein Mozannar, Gagan Bansal, Adam Fourney, and Eric Horvitz. 2023 · 2023
Later among the works it cites.
Do Users Write More Insecure Code with AI Assistants?. In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security (CCS ’23) . Association for Computing Machinery, New York, NY, USA, 2785–2799
Neil Perry, Megha Srivastava, Deepak Kumar, and Dan Boneh. 2023 · 2023
Later among the works it cites.
Supporting human-AI collaboration in auditing LLMs with LLMs. In Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society (AIES ’23) . Association for Computing Machinery, New York, NY, USA, 913–926
Charvi Rastogi, Marco Tulio Ribeiro, Nicholas King, Harsha Nori, and Saleema Amershi. 2023 · 2023
Later among the works it cites.
Lost at C: A user study on the security implications of large language model code assistants. In Proceedings of the 32nd USENIX Security Symposium (USENIX Security ’23) . USENIX Association, Anaheim, CA, 2205–2222
Gustavo Sandoval, Hammond Pearce, Teo Nys, Ramesh Karri, Siddharth Garg, and Brendan Dolan-Gavitt. 2023 · 2023
Later among the works it cites.
ChatGPT: five priorities for research
Eva AM van Dis, Johan Bollen, Willem Zuidema, Robert van Rooij, and Claudi L Bockting. 2023 · 2023
Later among the works it cites.
A prompt pattern catalog to enhance prompt engineering with ChatGPT
Jules White, Quchen Fu, Sam Hays, Michael Sandborn, Carlos Olea, Henry Gilbert, Ashraf Elnashar, Jesse Spencer-Smith, and Douglas C. Schmidt. 2023 · 2023
Later among the works it cites.
A Large-Scale Survey on the Usability of AI Programming Assistants: Successes and Challenges. In Proceedings of the 46th IEEE/ACM International Conference on Software Engineering (ICSE ’24) . Association for Computing Machinery, New York, NY, USA, Article 52, 13 pages
Jenny T. Liang, Chenyang Yang, and Brad A. Myers. 2024 · 2024
Closest in time.