2020

Prompt Agnostic Essay Scorer: A Domain Generalization Approach to Cross-prompt Automated Essay Scoring

Ridley, Robert, He, Liang, Dai, Xinyu et al.

Understand

Cross-prompt automated essay scoring (AES) requires the system to use non target-prompt essays to award scores to a target-prompt essay.

Since obtaining a large quantity of pre-graded essays to a particular prompt is often difficult and unrealistic, the task of cross-prompt AES is vital for the development of real-world AES systems, yet it remains an under-explored area of research.
Models designed for prompt-specific AES rely heavily on prompt-specific knowledge and perform poorly in the cross-prompt setting, whereas current approaches to cross-prompt AES either require a certain quantity of labelled target-prompt essays or require a large quantity of unlabelled target-prompt essays to perform transfer learning in a multi-step manner.
To address these issues, we introduce Prompt Agnostic Essay Scorer (PAES) for cross-prompt AES.

Built on

Page, E., B.: The imminence of… grading essays by computer. The Phi Delta Kappan, 47(5):238–243 (1966)
1966
Earlier work this paper cites.
Smith, M., Taffler, R.: Readability and understandability: Different measures of the textual complexity of accounting narrative. Accounting, Auditing & Accountability Journal, 5(4):0–0 (1992)
1992
Earlier work this paper cites.
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural computation 9(8):1735–1780 (1997)
1997
Earlier work this paper cites.
Larkey, L., S.: Automatic essay grading using text categorization techniques. In SIGIR. ACM, pages 90–95 (1998)
1998
Earlier work this paper cites.
Rudner, L., M.: Automated essay scoring using bayes’ theorem. National Council on Measurement in Education New Orleans La 1(2):3–21 (2002)
2002
Earlier work this paper cites.
Attali, Y., Burstein, J.: Automated essay scoring with e-raterⓇ v. 2. The Journal of Technology, Learning and Assessment 4(3) (2006)
2006
Earlier work this paper cites.
Granger, S., Dagneaux, E., Meunier, F., Paquot, M. International Corpus of Learner English (Version 2). Presses universitaires de Louvain (2009)
2009
Earlier work this paper cites.
Williamson, D., M.: A framework for implementing automated scoring. In Annual Meeting of the American Educational Research Association and the National Council on Measurement in Education, San Diego, CA. (2009)
2009
Earlier work this paper cites.
Yannakoudakis, H., Briscoe, T., Medlock, B.: A new dataset and method for automatically grading ESOL texts. In Proc. of ACL, pages 180–189 (2011)
Original
2011
Earlier work this paper cites.

Similar

Blanchard, D., Tetreault, J., Higgins, D., Cahill, A., Chodorow, M. TOEFL11: A corpus of non-native English. ETS Research Report Series, 2013(2):i–15 (2013)
2013
Cited alongside, same era.
Chen, H., He, B.: Automated essay scoring by maximizing human-machine agreement. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (2013)
2013
Cited alongside, same era.
Pascanu, R., Mikolov, T., Bengio, Y.: On the difficulty of training recurrent neural networks. ICML (3) 28:1310–1318 (2013)
2013
Cited alongside, same era.
Lei, C., Man, K., L., Ting, T.: Using learning analytics to analyze writing skills of students: A case study in a technological common core curriculum course. IAENG International Journal of Computer Science, 41(3) (2014)
2014
Cited alongside, same era.
Stab, C., Gurevych, I. Annotating argument components and relations in persuasive essays. In Proc. of COLING, pages 1501–1510, (2014)
2014
Cited alongside, same era.
Sutskever, I., Vinyals, O., Le, Q., V.: Sequence to sequence learning with neural networks. In Proc. of NIPS, pages 3104–3112 (2014)
2014
Cited alongside, same era.
Dauphin, Y., de Vries, H., Bengio, Y.: Equilibrated adaptive learning rates for nonconvex optimization. In Advances in Neural Information Processing Systems. pages 1504–1512 (2015)
2015
Cited alongside, same era.
Mcnamara, D., S., Crossley, S., A., Roscoe, R., D., Allen, L., K., Dai, J.: A hierarchical classification approach to automated essay scoring. Assessing Writing 23:35–59 (2015)
2015
Cited alongside, same era.
Phandi, P., Chai, K., M., A., Ng, H., T.: Flexible domain adaptation for automated essay scoring using correlated linear regression. In Proc. of EMNLP, pages 431–439 (2015)
2015
Cited alongside, same era.

Then

Alikaniotis, D., Yannakoudakis, H., Rei, M.: Automatic text scoring using neural networks. In Proc. of ACL, pages 715–725 (2016)
2016
Later among the works it cites.
Cummins, R., Zhang, M., Briscoe, T.: Constrained multi-task learning for automated essay scoring. In Proc. of ACL, pages 789–799 (2016)
2016
Later among the works it cites.
Dong, F., Zhang, Y.: Automatic features for essay scoring – An empirical study. In Proc. of EMNLP, pages 1072–1077 (2016)
2016
Later among the works it cites.
Taghipour, K., Ng, H., T.: A neural approach to automated essay scoring. In Proc. of EMNLP, pages 1882–1891, (2016)
2016
Later among the works it cites.
Dong, F., Zhang, Y., Yang, J.: Attention-based recurrent convolutional neural network for automatic essay scoring. In Proc. of CoNLL, pages 153–162, (2017)
2017
Later among the works it cites.
Jin, C., He, B., Hui, K., Sun, L.: TDNN: A two-stage deep neural network for promptindependent automated essay scoring. In Proc. of ACL, pages 1088–1097 (2018)
2018
Later among the works it cites.
Tay, Y., Phan, M., C., Tuan, L., A., Hui, S., C.: SkipFlow: Incorporating neural coherence features for end-to-end automatic text scoring. In Proc. of AAAI, pages 5948–5955 (2018)
2018
Later among the works it cites.
Ke, Z., Ng, V.: Automated essay scoring: A survey of the state of the art. In Proc. of the 28th IJCAI, pages 6300–6308. International Joint Conferences on Artificial Intelligence Organization (2019)
2019
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…