2023

ChatABL: Abductive Learning via Natural Language Interaction with ChatGPT

Zhong, Tianyang, Wei, Yaonai, Yang, Li et al.

Understand

Large language models (LLMs) such as ChatGPT have recently demonstrated significant potential in mathematical abilities, providing valuable reasoning paradigm consistent with human natural language.

However, LLMs currently have difficulty in bridging perception, language understanding and reasoning capabilities due to incompatibility of the underlying information flow among them, making it challenging to accomplish tasks autonomously.
On the other hand, abductive learning (ABL) frameworks for integrating the two abilities of perception and reasoning has seen significant success in inverse decipherment of incomplete facts, but it is limited by the lack of semantic understanding of logical reasoning rules and the dependence on complicated domain knowledge representation.
This paper presents a novel method (ChatABL) for integrating LLMs into the ABL framework, aiming at unifying the three abilities in a more user-friendly and understandable manner.

Built on

Zadeh, L.A.: Fuzzy sets. Information and control 8
1965
Earlier work this paper cites.
Pereira, F.C., Warren, D.H.: Definite clause grammars for language analysis—a survey of the formalism and a comparison with augmented transition networks. Artificial intelligence 13
1980
Earlier work this paper cites.
Abramson, H.: Definite clause translation grammars. University of British Columbia (1984)
1984
Earlier work this paper cites.
Biederman, I.: Human image understanding: Recent research and a theory. Computer vision, graphics, and image processing 32
1985
Earlier work this paper cites.
Humphrey, R.: How work roles influence perception: Structural-cognitive processes and organizational behavior. American Sociological Review pp. 242–252 (1985)
1985
Earlier work this paper cites.
Towell, G.G., Shavlik, J.W.: Knowledge-based artificial neural networks. Artificial intelligence 70
1994
Earlier work this paper cites.
Sloman, S.A.: The empirical case for two systems of reasoning. Psychological bulletin 119
1996
Earlier work this paper cites.
Chan, D., Schmitt, N., DeShon, R.P., Clause, C.S., Delbridge, K.: Reactions to cognitive ability tests: the relationships between race, test performance, face validity perceptions, and test-taking motivation. Journal of Applied Psychology 82
1997
Earlier work this paper cites.
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural computation 9
1997
Earlier work this paper cites.
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proceedings of the IEEE 86
1998
Earlier work this paper cites.
Houston, S.D., Mazariegos, O.F.C., Stuart, D.: The decipherment of ancient Maya writing. University of Oklahoma Press (2001)
2001
Earlier work this paper cites.
Garcez, A.S.d., Gabbay, D.M., Ray, O., Woods, J.: Abductive reasoning in neural-symbolic systems. Topoi 26
2007
Earlier work this paper cites.
Getoor, L., Taskar, B.: Introduction to statistical relational learning. MIT press (2007)
2007
Earlier work this paper cites.
De Raedt, L., Kersting, K.: Probabilistic inductive logic programming. Springer (2008)
2008
Earlier work this paper cites.
Zhu, Q., Song, Y., Hu, S., Li, X., Tian, M., Zhen, Z., Dong, Q., Kanwisher, N., Liu, J.: Heritability of the specific cognitive ability of face perception. Current Biology 20
Original
2010
Earlier work this paper cites.
De Raedt, L., Kimmig, A.: Probabilistic (logic) programming concepts. Machine Learning 100
2015
Earlier work this paper cites.
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. CoRR abs/1512.03385
Original
2015
Earlier work this paper cites.
Russell, S.: Unifying logic and probability. Communications of the ACM 58
2015
Earlier work this paper cites.
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (June 2015)
2015
Earlier work this paper cites.
Yu, Y., Qian, H., Hu, Y.Q.: Derivative-free optimization via classification. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 30 (2016)
2016
Earlier work this paper cites.
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Communications of the ACM 60
2017
Earlier work this paper cites.
Thoma, M.: The hasyv2 dataset. arXiv preprint arXiv:1701.08380 (2017)
Original
2017
Earlier work this paper cites.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30
2017
Earlier work this paper cites.
Dai, W.Z., Xu, Q.L., Yu, Y., Zhou, Z.H.: Tunneling neural perception and logic reasoning through abductive learning. arXiv preprint arXiv:1802.01173 (2018)
Original
2018
Earlier work this paper cites.

Similar

Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Original
2018
Cited alongside, same era.
Hossain, M.B., Naznin, F., Joarder, Y., Islam, M.Z., Uddin, M.J.: Recognition and solution for handwritten equation using convolutional neural network. In: 2018 Joint 7th International Conference on Informatics, Electronics & Vision (ICIEV) and 2018 2nd International Conference on Imaging, Vision & Pattern Recognition (icIVPR). pp. 250–255. IEEE (2018)
2018
Cited alongside, same era.
Radford, A., Narasimhan, K., Salimans, T., Sutskever, I., et al.: Improving language understanding by generative pre-training (2018)
2018
Cited alongside, same era.
Zhang, Z., Sabuncu, M.: Generalized cross entropy loss for training deep neural networks with noisy labels. Advances in neural information processing systems 31
2018
Cited alongside, same era.
Dai, W.Z., Xu, Q., Yu, Y., Zhou, Z.H.: Bridging machine learning and logical reasoning by abductive learning. Advances in Neural Information Processing Systems 32
2019
Cited alongside, same era.
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., Zettlemoyer, L.: Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461 (2019)
Original
2019
Cited alongside, same era.
Zhou, Z.H.: Abductive learning: towards bridging machine learning and logical reasoning. Science China Information Sciences 62
2019
Cited alongside, same era.
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J.D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al.: Language models are few-shot learners. Advances in neural information processing systems 33
2020
Cited alongside, same era.
Dai, W.Z., Muggleton, S.H.: Abductive knowledge induction from raw data. arXiv preprint arXiv:2010.03514 (2020)
Original
2020
Cited alongside, same era.
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)
Original
2020
Cited alongside, same era.
Fu, M., Liu, J., Zhang, H., Lu, S.: Multisensor fusion for magnetic flux leakage defect characterization under information incompletion. IEEE Transactions on Industrial Electronics 68
2020
Cited alongside, same era.
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., Liu, P.J.: Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research 21
2020
Cited alongside, same era.

Then

Rezayi, S., Dai, H., Liu, Z., Wu, Z., Hebbar, A., Burns, A.H., Zhao, L., Zhu, D., Li, Q., Liu, W., et al.: Clinicalradiobert: Knowledge-infused few shot learning for clinical notes named entity recognition. In: Machine Learning in Medical Imaging: 13th International Workshop, MLMI 2022, Held in Conjunction with MICCAI 2022, Singapore, September 18, 2022, Proceedings. pp. 269–278. Springer (2022)
2022
Later among the works it cites.
Sapir, M.: Machine learning is abduction inference. arXiv preprint arXiv:2206.07586 (2022)
Original
2022
Later among the works it cites.
Scao, T.L., Fan, A., Akiki, C., Pavlick, E., Ilić, S., Hesslow, D., Castagné, R., Luccioni, A.S., Yvon, F., Gallé, M., et al.: Bloom: A 176b-parameter open-access multilingual language model. arXiv preprint arXiv:2211.05100 (2022)
Original
2022
Later among the works it cites.
Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., Metzler, D., et al.: Emergent abilities of large language models. arXiv preprint arXiv:2206.07682 (2022)
Original
2022
Later among the works it cites.
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Chi, E., Le, Q., Zhou, D.: Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903 (2022)
Original
2022
Later among the works it cites.
Zhang, S., Roller, S., Goyal, N., Artetxe, M., Chen, M., Chen, S., Dewan, C., Diab, M., Li, X., Lin, X.V., et al.: Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068 (2022)
Original
2022
Later among the works it cites.
Aksyuk, V.: Consciousness is learning: predictive processing systems that learn by binding may perceive themselves as conscious. arXiv preprint arXiv:2301.07016 (2023)
Original
2023
Closest in time.
Blum, L., Blum, M.: A theoretical computer science perspective on consciousness and artificial general intelligence. arXiv preprint arXiv:2303.17075 (2023)
Original
2023
Closest in time.
Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P., Lee, Y.T., Li, Y., Lundberg, S., et al.: Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712 (2023)
Original
2023
Closest in time.
Dai, H., Liu, Z., Liao, W., Huang, X., Wu, Z., Zhao, L., Liu, W., Liu, N., Li, S., Zhu, D., et al.: Chataug: Leveraging chatgpt for text data augmentation. arXiv preprint arXiv:2302.13007 (2023)
Original
2023
Closest in time.
Ferilli, S.: Gear: A general inference engine for automated multistrategy reasoning. Electronics 12
2023
Closest in time.
Li, Y., Duan, Y.: The wisdom of artificial general intelligence: Experiments with gpt-4 for dikwp. arXiv preprint (2023)
2023
Closest in time.
Liu, Y., Han, T., Ma, S., Zhang, J., Yang, Y., Tian, J., He, H., Li, A., He, M., Liu, Z., et al.: Summary of chatgpt/gpt-4 research and perspective towards the future of large language models. arXiv preprint arXiv:2304.01852 (2023)
Original
2023
Closest in time.
Liu, Z., Yu, X., Zhang, L., Wu, Z., Cao, C., Dai, H., Zhao, L., Liu, W., Shen, D., Li, Q., et al.: Deid-gpt: Zero-shot medical text de-identification by gpt-4. arXiv preprint arXiv:2303.11032 (2023)
Original
2023
Closest in time.
Longpre, S., Hou, L., Vu, T., Webson, A., Chung, H.W., Tay, Y., Zhou, D., Le, Q.V., Zoph, B., Wei, J., et al.: The flan collection: Designing data and methods for effective instruction tuning. arXiv preprint arXiv:2301.13688 (2023)
Original
2023
Closest in time.
Ma, C., Wu, Z., Wang, J., Xu, S., Wei, Y., Liu, Z., Guo, L., Cai, X., Zhang, S., Zhang, T., et al.: Impressiongpt: An iterative optimizing framework for radiology report summarization with chatgpt. arXiv preprint arXiv:2304.08448 (2023)
Original
2023
Closest in time.
Nes, A., Sundberg, K., Watzl, S.: The perception/cognition distinction. Inquiry 66
2023
Closest in time.
Peng, K., Ding, L., Zhong, Q., Shen, L., Liu, X., Zhang, M., Ouyang, Y., Tao, D.: Towards making the most of chatgpt for machine translation. arXiv preprint arXiv:2303.13780 (2023)
Original
2023
Closest in time.
Wang, S., Zhao, Z., Ouyang, X., Wang, Q., Shen, D.: Chatcad: Interactive computer-aided diagnosis on medical image using large language models. arXiv preprint arXiv:2302.07257 (2023)
Original
2023
Closest in time.
Williams, D.: Hierarchical minds and the perception/cognition distinction. Inquiry 66
2023
Closest in time.
Wu, C., Yin, S., Qi, W., Wang, X., Tang, Z., Duan, N.: Visual chatgpt: Talking, drawing and editing with visual foundation models. arXiv preprint arXiv:2303.04671 (2023)
Original
2023
Closest in time.
Wu, Z., Zhang, L., Cao, C., Yu, X., Dai, H., Ma, C., Liu, Z., Zhao, L., Li, G., Liu, W., Li, Q., Shen, D., Li, X., Zhu, D., Liu, T.: Exploring the trade-offs: Unified large language models vs local fine-tuned models for highly-specific radiology nli task. arXiv preprint arXiv:2304.09138 (2023)
Original
2023
Closest in time.
Xu, F., Liu, J., Lin, Q., Zhao, T., Zhang, J., Zhang, L.: Mind reasoning manners: Enhancing type perception for generalized zero-shot logical reasoning over text. arXiv preprint arXiv:2301.02983 (2023)
Original
2023
Closest in time.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…