Fetching the paper…

Reading the bibliography…

2020

Limits of Detecting Text Generated by Large-Scale Language Models

Varshney, Lav R., Keskar, Nitish Shirish, Socher, Richard

Original
PDF
Post on X

Understand

Some consider large-scale language models that can generate long and coherent pieces of text as dangerous, since they may be used in misinformation campaigns.

Here we formulate large-scale language model output detection as a hypothesis testing problem to classify text as genuine or generated.
We show that error exponents for particular language models are bounded in terms of their perplexity, a standard measure of language generation performance.
Under the assumption that human language is stationary and ergodic, the formulation is extended from considering specific language models to considering maximum likelihood language models, among the class of k-order Markov approximations; error probabilities are characterized.

Limits of Detecting Text Generated by Large-Scale Language Models · Around

Built on

A. Wang and K. Cho, “BERT has a mouth, and it must speak: BERT as a Markov random field language model,” Feb. 2019, arXiv:1902.04094 [cs.CL]
Original
1902
Earlier work this paper cites.
A. Holtzman, J. Buys, M. Forbes, and Y. Choi, “The curious case of neural text degeneration,” Apr. 2019, arXiv:1904.09751 [cs.CL]
Original
1904
Earlier work this paper cites.
R. Zellers, A. Holtzman, H. Rashkin, Y. Bisk, A. Farhadi, F. Roesner, and Y. Choi, “Defending against neural fake news,” May 2019, arXiv:1905.12616 [cs.CL]
Original
1905
Earlier work this paper cites.
J. Bullock and M. Luengo-Oroz, “Automated speech generation from UN general assembly statements: Mapping risks in AI generated texts,” Jun. 2019, arXiv:1906.01946 [cs.CL]
Original
1906
Earlier work this paper cites.
A. Bakhtin, S. Gross, M. Ott, Y. Deng, M. Ranzato, and A. Szlam, “Real or fake? learning to discriminate machine from human generated text,” Jul. 2019, arXiv:1906.03351 [cs.LG]
Original
1906
Earlier work this paper cites.
I. Solaiman, M. Brundage, J. Clark, A. Askell, A. Herbert-Voss, J. Wu, A. Radford, G. Krueger, J. W. Kim, S. Kreps, M. McCain, A. Newhouse, J. Blazakis, K. McGuffie, and J. Wang, “Release strategies and the social impacts of language models,” Nov. 2019, arXiv:1908.09203v2 [cs.CL]
Original
1908
Earlier work this paper cites.
T. Schuster, R. Schuster, D. J. Shah, and R. Barzilay, “Are we safe yet? the limitations of distributional features for fake news detection,” Aug. 2019, arXiv:1908.09805 [cs.CL]
Original
1908
Earlier work this paper cites.
N. S. Keskar, B. McCann, L. R. Varshney, C. Xiong, and R. Socher, “CTRL: A conditional transformer language model for controllable generation,” Sep. 2019, arXiv:1909.05858 [cs.CL]
Original
1909
Earlier work this paper cites.
L. R. Varshney, N. S. Keskar, and R. Socher, “Pretrained AI models: Performativity, mobility, and change,” Sep. 2019, arXiv:1909.03290 [cs.CY]
Original
1909
Earlier work this paper cites.
D. Ippolito, D. Duckworth, C. Callison-Burch, and D. Eck, “Human and automatic detection of generated text,” Nov. 2019, arXiv:1911.00650 [cs.CL]
Original
1911
Earlier work this paper cites.
C. E. Shannon, “The redundancy of English,” in Transactions of the Seventh Conference on Cybernetics , Mar. 1950, pp. 123–158
1950
Earlier work this paper cites.
——, “Prediction and entropy of printed English,” Bell System Technical Journal , vol. 30, no. 1, pp. 50–64, Jan. 1951
1951
Earlier work this paper cites.
A. Chapanis, “The reconstruction of abbreviated printed messages,” Journal of Experimental Psychology , vol. 48, no. 6, pp. 496–510, Dec. 1954
1954
Earlier work this paper cites.

Similar

D. Jamison and K. Jamison, “A note on the entropy of partial-known languages,” Information and Control , vol. 12, no. 2, pp. 164–167, Feb. 1968
1968
Cited alongside, same era.
N. S. Tzannes, R. V. Spencer, and A. J. Kaplan, “On estimating the entropy of random fields,” Information and Control , vol. 16, no. 1, pp. 1–6, Mar. 1970
1970
Cited alongside, same era.
E. N. Gilbert, “Codes based on inaccurate source probabilities,” IEEE Transactions on Information Theory , vol. IT-17, no. 3, pp. 304–314, May 1971
1971
Cited alongside, same era.
T. M. Cover and R. C. King, “A convergent gambling estimate of the entropy of English,” IEEE Transactions on Information Theory , vol. IT-24, no. 4, pp. 413–421, Jul. 1978
1978
Cited alongside, same era.
T. M. Cover and J. A. Thomas, Elements of Information Theory . New York: John Wiley & Sons, 1991
1991

Then

S. Merity, C. Xiong, J. Bradbury, and R. Socher, “Pointer sentinel mixture models,” Sep. 2016, arXiv:1609.07843 [cs.CL]
Original
2016
Later among the works it cites.
I. Sason and S. Verdú, “ f f -divergence inequalities,” IEEE Transactions on Information Theory , vol. 62, no. 11, pp. 5973–6006, Nov. 2016
2016
Later among the works it cites.
C. Chelba, M. Norouzi, and S. Bengio, “ N N -gram language modeling using recurrent neural network estimation,” Mar. 2017, arXiv:1703.10724 [cs.CL]
Original
2017
Later among the works it cites.
A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever, “Language models are unsupervised multitask learners,” 2019
2019
Later among the works it cites.
A. Mitchell, J. Gottfried, G. Stocking, M. Walker, and S. Fedeli, Many Americans Say Made-Up News Is a Critical Problem That Needs To Be Fixed . Pew Research Center, Jun. 2019

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Cited alongside, same era.

H. Luschgy, A. L. Rukhin, and I. Vajda, “Adaptive tests for stochastic processes in the ergodic case,” Stochastic Processes and their Applications , vol. 45, no. 1, pp. 45–59, Mar. 1993

1993

Cited alongside, same era.

K. Marton, “Bounding d ¯ \bar{d} -distance by informational divergence: a method to prove measure concentration,” Annals of Probability , vol. 24, no. 2, pp. 857–866, Apr. 1996

1996

Cited alongside, same era.

——, “Measure concentration for a class of random processes,” Probability Theory and Related Fields , vol. 110, pp. 427–439, Mar. 1998

1998

Cited alongside, same era.

C. D. Manning and H. Schütze, Foundations of Statistical Natural Language Processing . Cambridge, MA, USA: MIT Press, 1999

1999

Cited alongside, same era.

U. M. Maurer, “Authentication theory and hypothesis testing,” IEEE Transactions on Information Theory , vol. 46, no. 4, pp. 1350–1356, Jul. 2000

2000

Cited alongside, same era.

Y. Sung, L. Tong, and H. V. Poor, “Neyman-Pearson detection of Gauss-Markov signals in noise: Closed-form error exponent and properties,” IEEE Transactions on Information Theory , vol. 52, no. 4, pp. 1354–1365, Apr. 2006

2006

Cited alongside, same era.

I. Csiszár and Z. Talata, “On rate of convergence of statistical estimation of stationary ergodic processes,” IEEE Transactions on Information Theory , vol. 56, no. 8, pp. 3637–3641, Aug. 2010

2010

Cited alongside, same era.

2019

Later among the works it cites.

S. Gehrmann, H. Strobelt, and A. Rush, “GLTR: Statistical detection and visualization of generated text,” in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL) , Jul. 2019, pp. 111–116

2019

Later among the works it cites.

S. Agarwal and L. R. Varshney, “Limits of deepfake detection: A robust estimation viewpoint,” in Proceedings of Synthetic Realities: Deep Learning for Detecting AudioVisual Fakes Workshop at ICML 2019 , Jun. 2019

2019

Later among the works it cites.

J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL) , Jun. 2019, pp. 4171–4186

2019

Later among the works it cites.

T. B. Hashimoto, H. Zhang, and P. Liang, “Unifying human and statistical evaluation for natural language generation,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL) , Jun. 2019, pp. 1689–1701

2019

Later among the works it cites.

C. Coupé, Y. M. Oh, D. Dediu, and F. Pellegrino, “Different languages, similar encoding efficiency: Comparable information rates across the human communicative niche,” Science Advances , vol. 5, no. 9, p. eaaw2594, Sep. 2019

2019

Later among the works it cites.

O. Binette, “A note on reverse Pinsker inequalities,” IEEE Transactions on Information Theory , vol. 65, no. 7, pp. 4094–4096, Jul. 2019

2019

Later among the works it cites.

D. Harwell, “Top AI researchers race to detect ‘deepfake’ videos: ‘We are outgunned’,” The Washington Post , Jun. 2019

2019

Later among the works it cites.