2020

Opportunities and Challenges in Code Search Tools

Liu, Chao, Xia, Xin, Lo, David et al.

Understand

Code search is a core software engineering task.

Effective code search tools can help developers substantially improve their software development efficiency and effectiveness.
In recent years, many code search studies have leveraged different techniques, such as deep learning and information retrieval approaches, to retrieve expected code from a large-scale codebase.
However, there is a lack of a comprehensive comparative summary of existing code search approaches.

Built on

Some quick sign tests for trend in location and dispersion
David Roxbee Cox and Alan Stuart. 1955 · 1955
Earlier work this paper cites.
A coefficient of agreement for nominal scales
Jacob Cohen. 1960 · 1960
Earlier work this paper cites.
Binary codes capable of correcting deletions, insertions, and reversals. In Soviet physics doklady , Vol. 10. 707–710
Vladimir I Levenshtein. 1966 · 1966
Earlier work this paper cites.
Extended Boolean information retrieval
Gerard Salton, Edward A Fox, and Harry Wu. 1983 · 1983
Earlier work this paper cites.
Dependency analysis tools: reusable components for software maintenance. In Proceedings. Conference on Software Maintenance . IEEE, 126–131
Norman Wilde, Ross Huitt, and Scott Huitt. 1989 · 1989
Earlier work this paper cites.
The R*-tree: an efficient and robust access method for points and rectangles. In ACM SIGMOD international conference on Management of data . 322–331
Norbert Beckmann, Hans-Peter Kriegel, Ralf Schneider, and Bernhard Seeger. 1990 · 1990
Earlier work this paper cites.
Similarity search in high dimensions via hashing. In VLDB , Vol. 99. 518–529
Aristides Gionis, Piotr Indyk, Rajeev Motwani, et al · 1999
Earlier work this paper cites.
Latent semantic indexing: An overview
Barbara Rosario. 2000 · 2000
Earlier work this paper cites.
CCFinder: a multilinguistic token-based code clone detection system for large scale source code
Toshihiro Kamiya, Shinji Kusumoto, and Katsuro Inoue. 2002 · 2002
Earlier work this paper cites.
Simian-similarity analyser
Simon Harris. 2003 · 2003
Earlier work this paper cites.
S pearman Correlation Coefficients, Differences between
Leann Myers and Maria J Sirois. 2004 · 2004
Earlier work this paper cites.
BIDE: Efficient mining of frequent closed sequences. In Proceedings. 20th international conference on data engineering . IEEE, 79–90
Jianyong Wang and Jiawei Han. 2004 · 2004
Earlier work this paper cites.
XSnippet: mining for sample code. In ACM SIGPLAN conference on Object-oriented programming systems, languages, and applications . 413–430
Naiyana Sahavechaphan and Kajal Claypool. 2006 · 2006
Earlier work this paper cites.
The Kendall rank correlation coefficient
Hervé Abdi. 2007 · 2007
Earlier work this paper cites.
Clustering by passing messages between data points
Brendan J Frey and Delbert Dueck. 2007 · 2007
Earlier work this paper cites.
Deckard: Scalable and accurate tree-based detection of code clones. In International Conference on Software Engineering . IEEE, 96–105
Lingxiao Jiang, Ghassan Misherghi, Zhendong Su, and Stephane Glondu. 2007 · 2007
Earlier work this paper cites.
Guidelines for performing systematic literature reviews in software engineering
Staffs Keele et al · 2007
Earlier work this paper cites.
A survey on software clone detection research
Chanchal Kumar Roy and James R Cordy. 2007 · 2007
Earlier work this paper cites.
Parseweb: a programmer assistant for reusing open source code on the web. In International Conference on Automated Software Engineering . 204–213
Suresh Thummalapenta and Tao Xie. 2007 · 2007
Earlier work this paper cites.
Novelty and diversity in information retrieval evaluation. In Proceedings of the 31st annual international ACM SIGIR conference on Research and development in information retrieval . 659–666
Charles LA Clarke, Maheedhar Kolla, Gordon V Cormack, Olga Vechtomova, Azin Ashkan, Stefan Büttcher, and Ian MacKinnon. 2008 · 2008
Earlier work this paper cites.
Z3: An efficient SMT solver. In International conference on Tools and Algorithms for the Construction and Analysis of Systems . Springer, 337–340
Leonardo De Moura and Nikolaj Bjørner. 2008 · 2008
Earlier work this paper cites.
Interpreting tf-idf term weights as making relevance decisions
Ho Chung Wu, Robert Wing Pong Luk, Kam Fai Wong, and Kui Lam Kwok. 2008 · 2008
Earlier work this paper cites.
Mining search topics from a code search engine usage log. In Working Conference on Mining Software Repositories . IEEE, 111–120
Sushil Bajracharya and Cristina Lopes. 2009 · 2009
Earlier work this paper cites.
Pearson correlation coefficient
Jacob Benesty, Jingdong Chen, Yiteng Huang, and Israel Cohen. 2009 · 2009
Earlier work this paper cites.
Two studies of opportunistic programming: interleaving web foraging, learning, and writing code. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems . 1589–1598
Joel Brandt, Philip J Guo, Joel Lewenstein, Mira Dontcheva, and Scott R Klemmer. 2009 · 2009
Earlier work this paper cites.
Kernelized locality-sensitive hashing for scalable image search. In 2009 IEEE 12th international conference on computer vision . IEEE, 2130–2137
Brian Kulis and Kristen Grauman. 2009 · 2009
Earlier work this paper cites.
Sourcerer: mining and searching internet-scale software repositories
Erik Linstead, Sushil Bajracharya, Trung Ngo, Paul Rigor, Cristina Lopes, and Pierre Baldi. 2009 · 2009
Earlier work this paper cites.
A survey on transfer learning
Sinno Jialin Pan and Qiang Yang. 2009 · 2009
Earlier work this paper cites.
Creating and evolving software by searching, selecting and synthesizing relevant source code. In International Conference on Software Engineering-Companion Volume . IEEE, 283–286
Denys Poshyvanyk and Mark Grechanik. 2009 · 2009
Earlier work this paper cites.
Semantics-based code search. In Proceedings of the 31st International Conference on Software Engineering . IEEE Computer Society, 243–253
Steven P Reiss. 2009 · 2009
Earlier work this paper cites.
The probabilistic relevance framework: BM25 and beyond
Stephen Robertson and Hugo Zaragoza. 2009 · 2009
Earlier work this paper cites.
Searching and skimming: An exploratory study. In IEEE International Conference on Software Maintenance and Evolution . IEEE, 157–166
Jamie Starke, Chris Luce, and Jonathan Sillito. 2009 · 2009
Earlier work this paper cites.
Alattin: Mining alternative patterns for detecting neglected conditions. In International Conference on Automated Software Engineering . IEEE, 283–294
Suresh Thummalapenta and Tao Xie. 2009 · 2009
Earlier work this paper cites.
MAPO: Mining and recommending API usage patterns. In European Conference on Object-Oriented Programming . Springer, 318–343
Hao Zhong, Tao Xie, Lu Zhang, Jian Pei, and Hong Mei. 2009 · 2009
Earlier work this paper cites.
On indexing error-tolerant set containment. In Proceedings of the 2010 ACM SIGMOD International Conference on Management of data . 927–938
Parag Agrawal, Arvind Arasu, and Raghav Kaushik. 2010 · 2010
Earlier work this paper cites.
Leveraging usage similarity for effective retrieval of examples in code repositories. In International Symposium on Foundations of Software Engineering . 157–166
Sushil K Bajracharya, Joel Ossher, and Cristina V Lopes. 2010 · 2010
Earlier work this paper cites.
Example-centric programming: integrating web search into the development environment. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems . ACM, 513–522
Joel Brandt, Mira Dontcheva, Marcos Weskamp, and Scott R Klemmer. 2010 · 2010
Earlier work this paper cites.
Towards an intelligent code search engine.. In AAAI
Jinhan Kim, Sanghoon Lee, Seung-won Hwang, and Sunghun Kim. 2010 · 2010
Earlier work this paper cites.
Instant code clone search. In International Symposium on Foundations of Software Engineering . 167–176
Mu-Woong Lee, Jong-Won Roh, Seung-won Hwang, and Sunghun Kim. 2010 · 2010
Earlier work this paper cites.
Lucene in action . Vol. 2
Michael McCandless, Erik Hatcher, Otis Gospodnetić, and O Gospodnetić. 2010 · 2010
Earlier work this paper cites.
Composition in distributional models of semantics
Jeff Mitchell and Mirella Lapata. 2010 · 2010
Earlier work this paper cites.
Augmented bug localization using past bug information. In Annual Southeast Regional Conference . 1–6
Brent D Nichols. 2010 · 2010
Earlier work this paper cites.
My repository runneth over: an empirical study on diversifying data sources to improve feature search. In International Conference on Program Comprehension . IEEE, 206–215
Sukanya Ratanotayanon, Hye Jung Choi, and Susan Elliott Sim. 2010 · 2010
Earlier work this paper cites.
An examination of software engineering work practices. In CASCON First Decade High Impact Papers . IBM Corp., 174–188
Janice Singer, Timothy Lethbridge, Norman Vinson, and Nicolas Anquetil. 2010 · 2010
Earlier work this paper cites.
Transfer learning
Lisa Torrey and Jude Shavlik. 2010 · 2010
Earlier work this paper cites.
Data mining: concepts and techniques
Jiawei Han, Jian Pei, and Micheline Kamber. 2011 · 2011
Earlier work this paper cites.
A test-driven approach to code search and its application to the reuse of auxiliary functionality
Otávio Augusto Lazzarini Lemos, Sushil Bajracharya, Joel Ossher, Paulo Cesar Masiero, and Cristina Lopes. 2011 · 2011
Earlier work this paper cites.
Recommending proper API code examples for documentation purpose. In Asia-Pacific Software Engineering Conference . IEEE, 331–338
Lee Wei Mar, Ye-Chi Wu, and Hewijin Christine Jiau. 2011 · 2011
Earlier work this paper cites.
Portfolio: finding relevant functions and their usage. In International Conference on Software Engineering . 111–120
Collin McMillan, Mark Grechanik, Denys Poshyvanyk, Qing Xie, and Chen Fu. 2011 · 2011
Earlier work this paper cites.

Similar

How well do search engines support code retrieval on the web?
Susan Elliott Sim, Medha Umarji, Sukanya Ratanotayanon, and Cristina V Lopes. 2011 · 2011
Cited alongside, same era.
Alattin: mining alternative patterns for defect detection
Suresh Thummalapenta and Tao Xie. 2011 · 2011
Cited alongside, same era.
Analyzing and mining a code search engine usage log
Sushil Krishna Bajracharya and Cristina Videira Lopes. 2012 · 2012
Cited alongside, same era.
Recommending source code for use in rapid software prototypes. In Proceedings of the 34th International Conference on Software Engineering . IEEE Press, 848–858
Collin McMillan, Negar Hariri, Denys Poshyvanyk, Jane Cleland-Huang, and Bamshad Mobasher. 2012 · 2012
Cited alongside, same era.
Feature location in source code: a taxonomy and survey
Bogdan Dit, Meghan Revelle, Malcom Gethers, and Denys Poshyvanyk. 2013 · 2013
Cited alongside, same era.
Automatic query reformulations for text retrieval in software engineering. In Proceedings of the 2013 International Conference on Software Engineering . IEEE Press, 842–851
Sonia Haiduc, Gabriele Bavota, Andrian Marcus, Rocco Oliveto, Andrea De Lucia, and Tim Menzies. 2013 · 2013
Cited alongside, same era.
Rendezvous: A search engine for binary code. In Working Conference on Mining Software Repositories . IEEE, 329–338
Wei Ming Khoo, Alan Mycroft, and Ross Anderson. 2013 · 2013
Cited alongside, same era.
What help do developers seek, when and how?. In IEEE Conference on Software Maintenance, Reengineering, and Reverse Engineering . IEEE, 142–151
Hongwei Li, Zhenchang Xing, Xin Peng, and Wenyun Zhao. 2013 · 2013
Cited alongside, same era.
Portfolio: Searching for relevant functions and their usages in millions of lines of code
Collin Mcmillan, Denys Poshyvanyk, Mark Grechanik, Qing Xie, and Chen Fu. 2013 · 2013
Cited alongside, same era.
Efficient estimation of word representations in vector space
Original
Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013 · 2013
Cited alongside, same era.
A survey of feature location techniques
Julia Rubin and Marsha Chechik. 2013 · 2013
Cited alongside, same era.
Assisting code search with automatic query reformulation for bug localization. In Working Conference on Mining Software Repositories . IEEE, 309–318
Bunyamin Sisman and Avinash C Kak. 2013 · 2013
Cited alongside, same era.

Then

Deep code search. In International Conference on Software Engineering . IEEE, 933–944
Xiaodong Gu, Hongyu Zhang, and Sunghun Kim. 2018 · 2018
Later among the works it cites.
Query expansion based on statistical learning from code changes
Qing Huang, Yangrui Yang, Xue Zhan, Hongyan Wan, and Guoqing Wu. 2018 · 2018
Later among the works it cites.
Semantics-Based Code Search Using Input/Output Examples. In International Working Conference on Source Code Analysis and Manipulation . IEEE, 92–102
Renhe Jiang, Zhengzhao Chen, Zejun Zhang, Yu Pei, Minxue Pan, and Tian Zhang. 2018 · 2018
Later among the works it cites.
FaCoY: a code-to-code search engine. In International Conference on Software Engineering . 946–957
Kisub Kim, Dongsun Kim, Tegawendé F Bissyandé, Eunjong Choi, Li Li, Jacques Klein, and Yves Le Traon. 2018 · 2018
Later among the works it cites.
A comment-driven approach to API usage patterns discovery and search
Shin-Jie Lee, Xavier Lin, Wu-Chen Su, and Hsi-Min Chen. 2018 · 2018
Later among the works it cites.
Cross-project change-proneness prediction. In 2018 IEEE 42nd Annual Computer Software and Applications Conference (COMPSAC) , Vol. 1. IEEE, 64–73
Chao Liu, Dan Yang, Xin Xia, Meng Yan, and Xiaohong Zhang. 2018b · 2018
Later among the works it cites.
Recommending github projects for developer onboarding
Chao Liu, Dan Yang, Xiaohong Zhang, Baishakhi Ray, and Md Masudur Rahman. 2018d · 2018
Later among the works it cites.
Evaluating how developers use general-purpose web-search for code retrieval. In Working Conference on Mining Software Repositories . 465–475
Md Masudur Rahman, Jed Barson, Sydney Paul, Joshua Kayani, Federico Andrés Lois, Sebastián Fernandez Quezada, Christopher Parnin, Kathryn T Stolee, and Baishakhi Ray. 2018 · 2018
Later among the works it cites.
Seeking the user interface
Steven P Reiss, Yun Miao, and Qi Xin. 2018 · 2018
Later among the works it cites.
Retrieval on source code: a neural code search. In ACM SIGPLAN International Workshop on Machine Learning and Programming Languages . 31–41
Saksham Sachdev, Hongyu Li, Sifei Luan, Seohyun Kim, Koushik Sen, and Satish Chandra. 2018 · 2018
Later among the works it cites.
Augmenting and structuring user queries to support efficient free-form code search
Raphael Sirres, Tegawendé F Bissyandé, Dongsun Kim, David Lo, Jacques Klein, Kisub Kim, and Yves Le Traon. 2018 · 2018
Later among the works it cites.
Accurate and scalable cross-architecture cross-os binary code search with emulation
Yinxing Xue, Zhengzi Xu, Mahinthan Chandramohan, and Yang Liu. 2018 · 2018
Later among the works it cites.
An overview of multi-task learning
Yu Zhang and Qiang Yang. 2018 · 2018
Later among the works it cites.
SOSRepair: Expressive Semantic Search for Real-World Program Repair
Afsoon Afzal, Manish Motwani, Kathryn Stolee, Yuriy Brun, and Claire Le Goues. 2019 · 2019
Later among the works it cites.
SCOR: source code retrieval with semantics and order. In Working Conference on Mining Software Repositories . IEEE, 1–12
Shayan Akbar and Avinash Kak. 2019 · 2019
Later among the works it cites.
When deep learning met code search. In International Symposium on Foundations of Software Engineering . 964–974
Jose Cambronero, Hongyu Li, Seohyun Kim, Koushik Sen, and Satish Chandra. 2019 · 2019
Later among the works it cites.
Capturing source code semantics via tree-based convolution over API-enhanced AST. In ACM International Conference on Computing Frontiers . 174–182
Long Chen, Wei Ye, and Shikun Zhang. 2019 · 2019
Later among the works it cites.
Asm2vec: Boosting static representation robustness for binary clone search against code obfuscation and compiler optimization. In IEEE Symposium on Security and Privacy . IEEE, 472–489
Steven HH Ding, Benjamin CM Fung, and Philippe Charland. 2019 · 2019
Later among the works it cites.
CodeKernel: a graph kernel based approach to the selection of API usage examples. In International Conference on Automated Software Engineering . IEEE, 590–601
Xiaodong Gu, Hongyu Zhang, and Sunghun Kim. 2019 · 2019
Later among the works it cites.
Enhance code search via reformulating queries with evolving contexts
Qing Huang and Guoqing Wu. 2019a · 2019
Later among the works it cites.
QE-integrating framework based on Github knowledge and SVM ranking
Qing Huang and Huaiguang Wu. 2019b · 2019
Later among the works it cites.
Deep learning the semantics of change sequences for query expansion
Qing Huang, Yang Yang, and Ming Cheng. 2019 · 2019
Later among the works it cites.
A Query Expansion Method Based on Evolving Source Code
Huan Jin and Lei Xiong. 2019 · 2019
Later among the works it cites.
Features and how to find them: a survey of manual feature location
Jacob Krüger, Thorsten Berger, and Thomas Leich. 2019 · 2019
Later among the works it cites.
Reinforcement Learning of Code Search Sessions. In Asia-Pacific Software Engineering Conference . IEEE, 458–465
Wei Li, Shuhan Yan, Beijun Shen, and Yuting Chen. 2019 · 2019
Later among the works it cites.
A two-phase transfer learning model for cross-project defect prediction
Chao Liu, Dan Yang, Xin Xia, Meng Yan, and Xiaohong Zhang. 2019c · 2019
Later among the works it cites.
Aroma: code recommendation via structural code search
Sifei Luan, Di Yang, Celeste Barnaby, Koushik Sen, and Satish Chandra. 2019 · 2019
Later among the works it cites.
Siamese: scalable and incremental code clone search via multiple code representations
Chaiyong Ragkhitwetsagul and Jens Krinke. 2019 · 2019
Later among the works it cites.
Slicing Based Code Recommendation for Type Based Instance Retrieval. In International Conference on Software and Systems Reuse . Springer, 149–167
Rui Sun, Hui Liu, and Leping Li. 2019 · 2019
Later among the works it cites.
Multi-modal attention network learning for semantic source code retrieval. In International Conference on Automated Software Engineering . IEEE, 13–25
Yao Wan, Jingdong Shu, Yulei Sui, Guandong Xu, Zhou Zhao, Jian Wu, and Philip Yu. 2019 · 2019
Later among the works it cites.
Code search based on alteration intent
Huaiguang Wu and Yang Yang. 2019 · 2019
Later among the works it cites.
User Interface Code Retrieval: A Novel Visual-Representation-Aware Approach
Yingtao Xie, Tao Lin, and Hongyan Xu. 2019 · 2019
Later among the works it cites.
CoaCor: code annotation for code retrieval with reinforcement learning. In The World Wide Web Conference . 2203–2214
Ziyu Yao, Jayavardhan Reddy Peddamail, and Huan Sun. 2019 · 2019
Later among the works it cites.
Enriching API Documentation with Code Samples and Usage Scenarios from Crowd Knowledge
Jingxuan Zhang, He Jiang, Zhilei Ren, Tao Zhang, and Zhiqiu Huang. 2019a · 2019
Later among the works it cites.
A novel neural source code representation based on abstract syntax tree. In 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE) . IEEE, 783–794
Jian Zhang, Xu Wang, Hongyu Zhang, Hailong Sun, Kaixuan Wang, and Xudong Liu. 2019b · 2019
Later among the works it cites.
S3: a scalable in-memory skip-list index for key-value store
Jingtian Zhang, Sai Wu, Zeyuan Tan, Gang Chen, Zhushi Cheng, Wei Cao, Yusong Gao, and Xiaojie Feng. 2019c · 2019
Later among the works it cites.
psc2code: Denoising Code Extraction from Programming Screencasts
Lingfeng Bao, Zhenchang Xing, Xin Xia, David Lo, Minghui Wu, and Xiaohu Yang. 2020 · 2020
Closest in time.
Enhancing example-based code search with functional semantics
Zhengzhao Chen, Renhe Jiang, Zejun Zhang, Yu Pei, Minxue Pan, Tian Zhang, and Xuandong Li. 2020 · 2020
Closest in time.
Unsupervised software repositories mining and its application to code search
Gang Hu, Min Peng, Yihan Zhang, Qianqian Xie, Wang Gao, and Mengting Yuan. 2020 · 2020
Closest in time.
A Code-Description Representation Learning Model Based on Attention. In International Conference on Software Analysis, Evolution and Reengineering . IEEE, 447–455
Qing Huang, An Qiu, Maosheng Zhong, and Yuan Wang. 2020 · 2020
Closest in time.
On the Replicability and Reproducibility of Deep Learning in Software Engineering
Original
Chao Liu, Cuiyun Gao, Xin Xia, David Lo, John Grundy, and Xiaohu Yang. 2020a · 2020
Closest in time.
Simplifying Deep-Learning-Based Model for Code Search
Original
Chao Liu, Xin Xia, David Lo, Zhiwei Liu, Ahmed E Hassan, and Shanping Li. 2020b · 2020
Closest in time.
Improving Code Search with Co-Attentive Representation Learning. In International Conference on Program Comprehension
Jianhang Shuai, Ling Xu, Chao Liu, Meng Yan, Xin Xia, and Yan Lei. 2020 · 2020
Closest in time.
Multi-Dimension Convolutional Neural Network for Bug Localization
Bei Wang, Ling Xu, Meng Yan, Chao Liu, and Ling Liu. 2020 · 2020
Closest in time.
Are the Code Snippets What We Are Searching for? A Benchmark and an Empirical Study on Code Search with Natural-Language Queries. In International Conference on Software Analysis, Evolution and Reengineering . IEEE, 344–354
Shuhan Yan, Hang Yu, Yuting Chen, Beijun Shen, and Lingxiao Jiang. 2020 · 2020
Closest in time.
Leveraging Code Generation to Improve Code Retrieval and Summarization via Dual Learning. In The World Wide Web Conference . 2309–2319
Wei Ye, Rui Xie, Jinglei Zhang, Tianxiang Hu, Xiaoyin Wang, and Shikun Zhang. 2020 · 2020
Closest in time.
Query expansion via learning change sequences
Qun Zou and Changquan Zhang. 2020 · 2020
Closest in time.
Summarizing source code using a neural attention model. In AMACL . 2073–2083
Srinivasan Iyer, Ioannis Konstas, Alvin Cheung, and Luke Zettlemoyer. 2016 · 2083
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…