AcTracer: Active Testing of Large Language Model via Multi-Stage Sampling

Language models are few-shot learners

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al · 1901

Earlier work this paper cites.

Who belongs in the family?

Robert L Thorndike. 1953 · 1953

Earlier work this paper cites.

Kolmogorov-Smirnov two-sample tests

John W Pratt, Jean D Gibbons, John W Pratt, and Jean D Gibbons. 1981 · 1981

Earlier work this paper cites.

The dip test of unimodality

John A Hartigan and Pamela M Hartigan. 1985 · 1985

Earlier work this paper cites.

Auction algorithms for network flow problems: A tutorial introduction

Dimitri P Bertsekas. 1992 · 1992

Earlier work this paper cites.

Generalization of the Kolmogorov-Smirnov test

Erhard Reschenhofer. 1997 · 1997

Earlier work this paper cites.

Approximate nearest neighbors: towards removing the curse of dimensionality. In Proceedings of the thirtieth annual ACM symposium on Theory of computing . 604–613

Piotr Indyk and Rajeev Motwani. 1998 · 1998

Earlier work this paper cites.

Stratified Sampling Types

Garrett Glasgow. 2005 · 2005

Earlier work this paper cites.

Compressed sensing

David L Donoho. 2006 · 2006

Earlier work this paper cites.

Finding a" kneedle" in a haystack: Detecting knee points in system behavior. In 2011 31st international conference on distributed computing systems workshops . IEEE, 166–171

Ville Satopaa, Jeannie Albrecht, David Irwin, and Barath Raghavan. 2011 · 2011

Earlier work this paper cites.

Practical bayesian optimization of machine learning algorithms

Jasper Snoek, Hugo Larochelle, and Ryan P Adams. 2012 · 2012

Earlier work this paper cites.

Better mixing via deep representations. In International conference on machine learning . PMLR, 552–560

Yoshua Bengio, Grégoire Mesnil, Yann Dauphin, and Salah Rifai. 2013 · 2013

Earlier work this paper cites.

Balanced k-means for clustering. In Structural, Syntactic, and Statistical Pattern Recognition: Joint IAPR International Workshop, S+ SSPR 2014, Joensuu, Finland, August 20-22, 2014. Proceedings . Springer, 32–41

Mikko I Malinen and Pasi Fränti. 2014 · 2014

Earlier work this paper cites.

Adaptive strategy for stratified Monte Carlo sampling

Alexandra Carpentier, Remi Munos, and András Antos. 2015 · 2015

Earlier work this paper cites.

On the rate of convergence in Wasserstein distance of the empirical measure

Nicolas Fournier and Arnaud Guillin. 2015 · 2015

Earlier work this paper cites.

SQuAD: 100,000+ Questions for Machine Comprehension of Text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing , Jian Su, Kevin Duh, and Xavier Carreras (Eds.). Association for Computational Linguistics, Austin, Texas, 2383–2392

Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016 · 2016

Earlier work this paper cites.

Wasserstein GAN

Original

Martin Arjovsky, Soumith Chintala, and Léon Bottou. 2017 · 2017

Earlier work this paper cites.

Detecting adversarial samples from artifacts

Original

Reuben Feinman, Ryan R Curtin, Saurabh Shintre, and Andrew B Gardner. 2017 · 2017

Earlier work this paper cites.

TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , Regina Barzilay and Min-Yen Kan (Eds.). Association for Computational Linguistics, Vancouver, Canada, 1601–1611

Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017 · 2017

Earlier work this paper cites.

Why is random testing effective for partition tolerance bugs?

Rupak Majumdar and Filip Niksic. 2017 · 2017

Earlier work this paper cites.

Deepxplore: Automated whitebox testing of deep learning systems. In proceedings of the 26th Symposium on Operating Systems Principles . 1–18

Kexin Pei, Yinzhi Cao, Junfeng Yang, and Suman Jana. 2017 · 2017

Earlier work this paper cites.

Attention is all you need

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017 · 2017

Earlier work this paper cites.

Randomized testing of distributed systems with probabilistic guarantees

Burcu Kulahcioglu Ozkan, Rupak Majumdar, Filip Niksic, Mitra Tabaei Befrouei, and Georg Weissenbacher. 2018 · 2018

Earlier work this paper cites.

Active Learning for Convolutional Neural Networks: A Core-Set Approach. In International Conference on Learning Representations

Ozan Sener and Silvio Savarese. 2018 · 2018

Earlier work this paper cites.

Guiding deep learning system testing using surprise adequacy. In 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE) . IEEE, 1039–1049

Jinhan Kim, Robert Feldt, and Shin Yoo. 2019 · 2019

Earlier work this paper cites.

Hierarchical optimal transport for multimodal distribution alignment

John Lee, Max Dabagia, Eva Dyer, and Christopher Rozell. 2019b · 2019

Earlier work this paper cites.

Boosting operational dnn testing efficiency through conditioning. In Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering . 499–509

Zenan Li, Xiaoxing Ma, Chang Xu, Chun Cao, Jingwei Xu, and Jian Lü. 2019 · 2019

Earlier work this paper cites.

Adaptive: Parallel active learning of mathematical functions

Bas Nijholt, Joseph Weston, Jorn Hoofwijk, and Anton Akhmerov. 2019 · 2019

Earlier work this paper cites.

Language models are unsupervised multitask learners

Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al · 2019

Earlier work this paper cites.

Practical accuracy estimation for efficient deep neural network testing

Junjie Chen, Zhuo Wu, Zan Wang, Hanmo You, Lingming Zhang, and Ming Yan. 2020 · 2020

Earlier work this paper cites.

Deepgini: prioritizing massive tests to enhance the robustness of deep neural networks. In Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis . 177–188

Yang Feng, Qingkai Shi, Xinyu Gao, Jun Wan, Chunrong Fang, and Zhenyu Chen. 2020 · 2020

Earlier work this paper cites.

Program synthesis with large language models

Original

Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al · 2021

Earlier work this paper cites.

Evaluating large language models trained on code

Original

Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al · 2021

Earlier work this paper cites.

Training Verifiers to Solve Math Word Problems

Original

Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021 · 2021

Cited alongside, same era.

Are labels always necessary for classifier accuracy evaluation?. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition . 15069–15078

Weijian Deng and Liang Zheng. 2021 · 2021

Cited alongside, same era.

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In International Conference on Learning Representations

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021 · 2021

Cited alongside, same era.

Predicting with confidence on unseen distributions. In Proceedings of the IEEE/CVF international conference on computer vision . 1134–1144

Devin Guillory, Vaishaal Shankar, Sayna Ebrahimi, Trevor Darrell, and Ludwig Schmidt. 2021 · 2021

Cited alongside, same era.

Llama 2: Open foundation and fine-tuned chat models

Original

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al · 2023

Later among the works it cites.

DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models

Boxin Wang, Weixin Chen, Hengzhi Pei, Chulin Xie, Mintong Kang, Chenhui Zhang, Chejian Xu, Zidi Xiong, Ritik Dutta, Rylan Schaeffer, et al · 2023

Later among the works it cites.

CodeTransOcean: A Comprehensive Multilingual Benchmark for Code Translation. In Findings of the Association for Computational Linguistics: EMNLP 2023 , Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). Association for Computational Linguistics, Singapore, 5067–5089

Weixiang Yan, Yuchen Tian, Yunzhe Li, Qian Chen, and Wen Wang. 2023 · 2023

Later among the works it cites.

Foundations of human spatial problem solving

Noah Zarr and Joshua W Brown. 2023 · 2023

Later among the works it cites.