Training Optimus Prime, M.D.: Generating Medical Certification Items by Fine-Tuning OpenAI's gpt2 Transformer Model

Z. Dai, Z. Yang, Y. Yang, J. G. Carbonell, Q. V. Le, and R. Salakhutdinov, “Transformer-xl: Attentive language models beyond a fixed-length context,” CoRR , vol. abs/1901.02860, 2019. [Online]. Available: http://arxiv.org/abs/1901.02860

Original

1901

Earlier work this paper cites.

C. Shannon, “A mathematical theory of communication,” Bell System Technical Journal , vol. 27, pp. 379–423, 1948

1948

Earlier work this paper cites.

F. Rosenblatt, “The perceptron: A probabilistic model for information storage and organization in the brain,” Psychological Review , vol. 65, no. 6, pp. 386–408, 1958

1958

Earlier work this paper cites.

MIT, “Mit’s 1961 tv program: The thinking machine.” [Online]. Available: https://www.youtube.com/watch

1961

Earlier work this paper cites.

K. Hornik, “Approximation capabilities of multilayer feedforward networks,” Neural Networks , vol. 4, no. 2, pp. 251–257, 1991

1991

Earlier work this paper cites.

Z. Gilula, , and S. Haberman, “Models for analyzing categorical panel data,” Journal of the American Statistical Association , vol. 89, pp. 645–656, 1994

1994

Earlier work this paper cites.

S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Computation , vol. 9, no. 8, pp. 1735–1780, 1997

1997

Earlier work this paper cites.

L. Goldberg, “A broad-bandwidth, public domain, personality inventory measuring the lower-level facets of several five-factor models,” in Personality Psychology in Europe , and F. Ostendorf, Ed. Tilburg: The, 1999, vol. 7, pp. 7–28

1999

Earlier work this paper cites.

S. Embretson, “Generating items during testing: psychometric issues and models,” Psychometrika , vol. 64, no. 4, pp. 407–433, 1999

1999

Earlier work this paper cites.

I. Bejar, “Generative testing: from conception to implementation in item generation for test development,” in Item generation for test development , S.H.Irvine and P. C. Kyllonen, Eds. Mahwah, NJ: Lawrence Erlbaum Associates, 2002, pp. 199–217

2002

Earlier work this paper cites.

L. Goldberg, J. Johnson, H. Eber, R. Hogan, M. Ashton, C. Cloninger, , and H. Gough, “The international personality item pool and the future of public-domain personality measures,” Journal of Research in Personality , vol. 40, pp. 84–96, 2006

2006

Earlier work this paper cites.

A. Singh, M. Singh, A. Singh, D. Singh, P. Singh, and A. Sharma, “Free full text articles : where to search for them?” International Journal of Trichology , vol. 3, no. 2, pp. 75–79, 2011

2011

Earlier work this paper cites.

D. S. Chevitarese, D. Szwarcman, and M. Vellasco, “Speeding up the training of neural networks with cuda technology,” in Artificial Intelligence and Soft Computing , L. Rutkowski, M. Korytkowski, R. Scherer, R. Tadeusiewicz, L. A. Zadeh, and J. M. Zurada, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2012, pp. 30–38

2012

Cited alongside, same era.

E. Choi, M. T. Bahadori, and J. Sun, “Doctor AI: predicting clinical events via recurrent neural networks,” CoRR , vol. abs/1511.05942, 2015. [Online]. Available: http://arxiv.org/abs/1511.05942

Original

2015

Cited alongside, same era.

M. von Davier, “High-performance psychometrics: The parallel-e parallel-m algorithm for generalized latent variable models,” ETS Research Report Series , pp. 1–11, 2016

2016

Cited alongside, same era.

M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. J. Goodfellow, A. Harp, G. Irving, M. Isard, Y. Jia, R. Józefowicz, L. Kaiser, M. Kudlur, J. Levenberg, D. Mané, R. Monga, S. Moore, D. G. Murray, C. Olah, M. Schuster, J. Shlens, B. Steiner, I. Sutskever, K. Talwar, P. A. Tucker, V. Vanhoucke, V. Vasudevan, F. B. Viégas, O. Vinyals, P. Warden, M. Wattenberg, M. Wicke, Y. Yu, and X. Zheng, “Tensorflow: Large-scale machine learning on heterogeneous distributed systems,” CoRR , vol. abs/1603.04467, 2016. [Online]. Available: http://arxiv.org/abs/1603.04467

M. von Davier, “Automated item generation with recurrent neural networks,” Psychometrika , vol. 83, no. 4, pp. 847–857, 2018

2018

Later among the works it cites.

A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, , and I. Sutskever, “Language models are unsupervised multitask learners,” 2018. [Online]. Available: https://d4mucfpksywv.cloudfront.net/better-language-models/language

2018

Later among the works it cites.

J. Devlin, M. Chang, K. Lee, and K. Toutanova, “BERT: pre-training of deep bidirectional transformers for language understanding,” CoRR , vol. abs/1810.04805, 2018. [Online]. Available: http://arxiv.org/abs/1810.04805

Original

2018

Later among the works it cites.

C. Xiao, E. Choi, and J. Sun, “Opportunities and challenges in developing deep learning models using electronic health records data: a systematic review,” Journal of the American Medical Informatics Association , vol. 25, no. 10, pp. 1419–1428, 06 2018. [Online]. Available: https://doi.org/10.1093/jamia/ocy068

2018