Fetching the paper…

Reading the bibliography…

2024

Towards Training A Chinese Large Language Model for Anesthesiology

Wang, Zhonghai, Jiang, Jie, Zhan, Yibing et al.

Original
PDF
Post on X

Understand

Medical large language models (LLMs) have gained popularity recently due to their significant practical utility.

However, most existing research focuses on general medicine, and there is a need for in-depth study of LLMs in specific fields like anesthesiology.
To fill the gap, we introduce Hypnos, a Chinese Anesthesia model built upon existing LLMs, e.g., Llama.
Hypnos' contributions have three aspects: 1) The data, such as utilizing Self-Instruct, acquired from current LLMs likely includes inaccuracies.

Towards Training A Chinese Large Language Model for Anesthesiology · Around

Built on

K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “Bleu: a method for automatic evaluation of machine translation,” in Proceedings of the 40th annual meeting of the Association for Computational Linguistics , 2002, pp. 311–318
2002
Earlier work this paper cites.
L. Chin-Yew, “Rouge: A package for automatic evaluation of summaries,” in Proceedings of the Workshop on Text Summarization Branches Out, 2004 , 2004
2004
Earlier work this paper cites.
A. Mutton, M. Dras, S. Wan, and R. Dale, “Gleu: Automatic evaluation of sentence-level fluency,” in Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics , 2007, pp. 344–351
2007
Earlier work this paper cites.
D. H. Robinson and A. H. Toledo, “Historical development of modern anesthesia,” Journal of Investigative Surgery , vol. 25, no. 3, pp. 141–149, 2012
2012
Earlier work this paper cites.
J. Li, M. Galley, C. Brockett, J. Gao, and B. Dolan, “A diversity-promoting objective function for neural conversation models,” arXiv preprint arXiv:1510.03055 , 2015
Original
2015
Earlier work this paper cites.
T. Kudo and J. Richardson, “Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing,” ArXiv , vol. abs/1808.06226, 2018. [Online]. Available: https://api.semanticscholar.org/CorpusID:52051958
Original
2018
Earlier work this paper cites.
A. E. Johnson, T. J. Pollard, S. J. Berkowitz, N. R. Greenbaum, M. P. Lungren, C.-y. Deng, R. G. Mark, and S. Horng, “Mimic-cxr, a de-identified publicly available database of chest radiographs with free-text reports,” Scientific data , vol. 6, no. 1, p. 317, 2019
2019
Earlier work this paper cites.
D. A. Hashimoto, E. Witkowski, L. Gao, O. Meireles, and G. Rosman, “Artificial intelligence in anesthesiology: current techniques, clinical applications, and limitations,” Anesthesiology , vol. 132, no. 2, pp. 379–394, 2020
2020
Earlier work this paper cites.
S. Rajbhandari, J. Rasley, O. Ruwase, and Y. He, “Zero: Memory optimizations toward training trillion parameter models,” in SC20: International Conference for High Performance Computing, Networking, Storage and Analysis . IEEE, 2020, pp. 1–16
2020
Earlier work this paper cites.
E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” arXiv preprint arXiv:2106.09685 , 2021
Original
2021
Earlier work this paper cites.
Z. Du, Y. Qian, X. Liu, M. Ding, J. Qiu, Z. Yang, and J. Tang, “Glm: General language model pretraining with autoregressive blank infilling,” arXiv preprint arXiv:2103.10360 , 2021
Original
2021
Earlier work this paper cites.
A. Komatsuzaki, J. Puigcerver, J. Lee-Thorp, C. R. Ruiz, B. Mustafa, J. Ainslie, Y. Tay, M. Dehghani, and N. Houlsby, “Sparse upcycling: Training mixture-of-experts from dense checkpoints,” arXiv preprint arXiv:2212.05055 , 2022
Original
2022
Earlier work this paper cites.
Y. Wang, Y. Kordi, S. Mishra, A. Liu, N. A. Smith, D. Khashabi, and H. Hajishirzi, “Self-instruct: Aligning language model with self generated instructions,” arXiv preprint arXiv:2212.10560 , 2022
Original
2022
Earlier work this paper cites.
Y. Bai, S. Kadavath, S. Kundu, A. Askell, J. Kernion, A. Jones, A. Chen, A. Goldie, A. Mirhoseini, C. McKinnon et al. , “Constitutional ai: Harmlessness from ai feedback,” arXiv preprint arXiv:2212.08073 , 2022
Original
2022
Earlier work this paper cites.
T. L. Scao, A. Fan, C. Akiki, E. Pavlick, S. Ilić, D. Hesslow, R. Castagné, A. S. Luccioni, F. Yvon, M. Gallé et al. , “Bloom: A 176b-parameter open-access multilingual language model,” arXiv preprint arXiv:2211.05100 , 2022
Original
2022
Earlier work this paper cites.
A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. W. Chung, C. Sutton, S. Gehrmann et al. , “Palm: Scaling language modeling with pathways,” arXiv preprint arXiv:2204.02311 , 2022
Original
2022
Earlier work this paper cites.
Anthropic, “Claude,” https://www.anthropic.com/, 12 2022
2022
Earlier work this paper cites.
Z. Liu, X. Yu, L. Zhang, Z. Wu, C. Cao, H. Dai, L. Zhao, W. Liu, D. Shen, Q. Li et al. , “Deid-gpt: Zero-shot medical text de-identification by gpt-4,” arXiv preprint arXiv:2303.11032 , 2023
Original
2023
Earlier work this paper cites.
Y. Cui, Z. Yang, and X. Yao, “Efficient and effective text encoding for chinese llama and alpaca,” arXiv preprint arXiv:2304.08177 , 2023
Original
2023
Earlier work this paper cites.

Similar

H. Wang, C. Liu, N. Xi, Z. Qiang, S. Zhao, B. Qin, and T. Liu, “Huatuo: Tuning llama model with chinese medical knowledge,” arXiv preprint arXiv:2304.06975 , 2023
Original
2023
Cited alongside, same era.
Y. Ji, Y. Deng, Y. Gong, Y. Peng, Q. Niu, L. Zhang, B. Ma, and X. Li, “Exploring the impact of instruction data scaling on large language models: An empirical study on real-world use cases,” arXiv preprint arXiv:2303.14742 , 2023
Original
2023
Cited alongside, same era.
Y. M. Y. W. Y. W. Hongcheng Liu, Yusheng Liao, “Medicalgpt-zh：中文医疗对话语言模型,” https://github.com/MediaBrain-SJTU/MedicalGPT-zh
2023
Cited alongside, same era.
Y. Cui, Z. Yang, and X. Yao, “Efficient and effective text encoding for chinese llama and alpaca,” arXiv preprint arXiv:2304.08177 , 2023. [Online]. Available: https://arxiv.org/abs/2304.08177
Original
2023
Cited alongside, same era.

Then

F. Muftić, M. Kadunić, A. Mušinbegović, and A. Abd Almisreb, “Exploring medical breakthroughs: A systematic review of chatgpt applications in healthcare,” Southeast Europe Journal of Soft Computing , vol. 12, no. 1, pp. 13–41, 2023
2023
Later among the works it cites.
T. Dave, S. A. Athaluri, and S. Singh, “Chatgpt in medicine: an overview of its applications, advantages, limitations, future prospects, and ethical considerations,” Frontiers in Artificial Intelligence , vol. 6, p. 1169595, 2023
2023
Later among the works it cites.
C. Xu, D. Guo, N. Duan, and J. McAuley, “Baize: An open-source chat model with parameter-efficient tuning on self-chat data,” arXiv preprint arXiv:2304.01196 , 2023
Original
2023
Later among the works it cites.
Y. Chen, Z. Wang, X. Xing, Z. Xu, K. Fang, S. Li, J. Wang, and X. Xu, “Bianque-1.0: Improving the ”question” ability of medical chat model through finetuning with hybrid instructions and multi-turn doctor qa datasets,” 2023. [Online]. Available: https://github.com/scutcyr/BianQue

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Y. Liu, T. Han, S. Ma, J. Zhang, Y. Yang, J. Tian, H. He, A. Li, M. He, Z. Liu et al. , “Summary of chatgpt/gpt-4 research and perspective towards the future of large language models,” arXiv preprint arXiv:2304.01852 , 2023

2023

Cited alongside, same era.

H. Dai, Z. Liu, W. Liao, X. Huang, Z. Wu, L. Zhao, W. Liu, N. Liu, S. Li, D. Zhu et al. , “Chataug: Leveraging chatgpt for text data augmentation,” arXiv preprint arXiv:2302.13007 , 2023

2023

Cited alongside, same era.

Z. Liu, A. Zhong, Y. Li, L. Yang, C. Ju, Z. Wu, C. Ma, P. Shu, C. Chen, S. Kim et al. , “Radiology-gpt: A large language model for radiology,” arXiv preprint arXiv:2306.08666 , 2023

2023

Cited alongside, same era.

M. Sallam, “The utility of chatgpt as an example of large language models in healthcare education, research and practice: Systematic review on the future perspectives and potential limitations,” medRxiv , pp. 2023–02, 2023

2023

Cited alongside, same era.

F. C. Kitamura, “Chatgpt is shaping the future of medical writing but still requires human judgment,” p. e230171, 2023

2023

Cited alongside, same era.

M. Angel, J. Rinehart, M. Canneson, and P. F. Baldi, “Clinical knowledge and reasoning abilities of ai large language models in anesthesiology: A comparative study on the aba exam,” medRxiv , pp. 2023–05, 2023

2023

Cited alongside, same era.

Y. Cui, Z. Yang, and X. Yao, “Efficient and effective text encoding for chinese llama and alpaca,” ArXiv , vol. abs/2304.08177, 2023. [Online]. Available: https://api.semanticscholar.org/CorpusID:258180548

2023

Cited alongside, same era.

J. Cui, Z. Li, Y. Yan, B. Chen, and L. Yuan, “Chatlaw: Open-source legal large language model with integrated external knowledge bases,” arXiv preprint arXiv:2306.16092 , 2023

2023

Cited alongside, same era.

2023

Later among the works it cites.

C. Wu, X. Zhang, Y. Zhang, Y. Wang, and W. Xie, “Pmc-llama: Further finetuning llama on medical papers,” arXiv preprint arXiv:2304.14454 , 2023

2023

Later among the works it cites.

W. Zhu and X. Wang, “Chatmed: A chinese medical large language model,” https://github.com/michael-wzhu/ChatMed

2023

Later among the works it cites.

H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar et al. , “Llama: Open and efficient foundation language models,” arXiv preprint arXiv:2302.13971 , 2023

2023

Later among the works it cites.

H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale et al. , “Llama 2: Open foundation and fine-tuned chat models,” arXiv preprint arXiv:2307.09288 , 2023

2023

Later among the works it cites.

Y. Li, Z. Li, K. Zhang, R. Dan, S. Jiang, and Y. Zhang, “Chatdoctor: A medical chat model fine-tuned on a large language model meta-ai (llama) using medical domain knowledge,” Cureus , vol. 15, no. 6, 2023

2023

Later among the works it cites.

R. Anil, A. M. Dai, O. Firat, M. Johnson, D. Lepikhin, A. Passos, S. Shakeri, E. Taropa, P. Bailey, Z. Chen et al. , “Palm 2 technical report,” arXiv preprint arXiv:2305.10403 , 2023

2023

Later among the works it cites.

K. Singhal, S. Azizi, T. Tu, S. S. Mahdavi, J. Wei, H. W. Chung, N. Scales, A. Tanwani, H. Cole-Lewis, S. Pfohl et al. , “Large language models encode clinical knowledge,” Nature , pp. 1–9, 2023

2023

Later among the works it cites.

K. Singhal, T. Tu, J. Gottweis, R. Sayres, E. Wulczyn, L. Hou, K. Clark, S. Pfohl, H. Cole-Lewis, D. Neal et al. , “Towards expert-level medical question answering with large language models,” arXiv preprint arXiv:2305.09617 , 2023

2023

Later among the works it cites.

baichuan inc, “Baichuan-7b,” https://github.com/baichuan-inc/Baichuan-7B, 6 2023

2023

Later among the works it cites.

Z. University, “Qizhen-chinese-llama-7b,” https://github.com/CMKRG/QiZhenGPT, 5 2023

2023

Later among the works it cites.

OpenAI, “Chatgpt-4,” https://openai.com/gpt-4, 3 2023

2023

Later among the works it cites.

R. Taori, I. Gulrajani, T. Zhang, Y. Dubois, X. Li, C. Guestrin, P. Liang, and T. B. Hashimoto, “Alpaca: A strong, replicable instruction-following model,” Stanford Center for Research on Foundation Models. https://crfm. stanford. edu/2023/03/13/alpaca. html , vol. 3, no. 6, p. 7, 2023

2023

Later among the works it cites.

W.-L. Chiang, Z. Li, Z. Lin, Y. Sheng, Z. Wu, H. Zhang, L. Zheng, S. Zhuang, Y. Zhuang, J. E. Gonzalez et al. , “Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality,” See https://vicuna. lmsys. org (accessed 14 April 2023) , 2023

2023

Later among the works it cites.

R. Taori, I. Gulrajani, T. Zhang, Y. Dubois, X. Li, C. Guestrin, P. Liang, and T. B. Hashimoto, “Stanford alpaca: An instruction-following llama model,” https://github.com/tatsu-lab/stanford_alpaca

2023

Later among the works it cites.

Y. G. Y. P. Q. N. B. M. Yunjie Ji, Yong Deng and X. Li, “Belle: Be everyone’s large language model engine,” https://github.com/LianjiaTech/BELLE

2023

Later among the works it cites.