ChemSafetyBench: Benchmarking LLM Safety on Chemistry Domain

7 revealing ways ais fail: Neural networks can be disastrously brittle, forgetful, and surprisingly bad at math

Charles Q Choi · 2021

Earlier work this paper cites.

Training verifiers to solve math word problems

Original

Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al · 2021

Earlier work this paper cites.

Measuring mathematical problem solving with the math dataset

Original

Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt · 2021

Earlier work this paper cites.

Learn to explain: Multimodal reasoning via thought chains for science question answering

Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan · 2022

Earlier work this paper cites.

Scienceqa: A novel resource for question answering on scholarly articles

Tanik Saikh, Tirthankar Ghosal, Amish Mittal, Asif Ekbal, and Pushpak Bhattacharyya · 2022

Earlier work this paper cites.

React: Synergizing reasoning and acting in language models

Original

Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao · 2022

Earlier work this paper cites.

Gpt-4 technical report

Original

Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al · 2023

Earlier work this paper cites.

Qwen technical report

Original

Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, Peng Wang, Shijie Wang, Wei Wang, Shengguang Wu, Benfeng Xu, Jin Xu, An Yang, Hao Yang, Jian Yang, Shusheng Yang, Yang Yao, Bowen Yu, Hongyi Yuan, Zheng Yuan, Jianwei Zhang, Xingxuan Zhang, Yichang Zhang, Zhenru Zhang, Chang Zhou, Jingren Zhou, Xiaohuan Zhou, and Tianhang Zhu · 2023

Earlier work this paper cites.

Autonomous chemical research with large language models

Daniil A Boiko, Robert MacKnight, Ben Kline, and Gabe Gomes · 2023

Earlier work this paper cites.

Bioinfo-bench: A simple benchmark framework for llm bioinformatics skills evaluation

Qiyuan Chen and Cheng Deng · 2023

Earlier work this paper cites.

What can large language models do in chemistry? a comprehensive benchmark on eight tasks

Taicheng Guo, Bozhao Nan, Zhenwen Liang, Zhichun Guo, Nitesh Chawla, Olaf Wiest, Xiangliang Zhang, et al · 2023

Earlier work this paper cites.

Testing llm performance on the physics gre: some observations, 2023

Pranav Gupta · 2023

Earlier work this paper cites.

A survey on large language models: Applications, challenges, limitations, and practical usage

Muhammad Usman Hadi, Rizwan Qureshi, Abbas Shah, Muhammad Irfan, Anas Zafar, Muhammad Bilal Shaikh, Naveed Akhtar, Jia Wu, Seyedali Mirjalili, et al · 2023

Earlier work this paper cites.

Control risk for potential misuse of artificial intelligence in science, 2023

Jiyan He, Weitao Feng, Yaosen Min, Jingwei Yi, Kunsheng Tang, Shuai Li, Jie Zhang, Kejiang Chen, Wenbo Zhou, Xing Xie, Weiming Zhang, Nenghai Yu, and Shuxin Zheng · 2023

Earlier work this paper cites.

Amortizing intractable inference in large language models

Original

Edward J Hu, Moksh Jain, Eric Elmoznino, Younesse Kaddar, Guillaume Lajoie, Yoshua Bengio, and Nikolay Malkin · 2023

Earlier work this paper cites.

Llama guard: Llm-based input-output safeguard for human-ai conversations

Original

Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, et al · 2023

Earlier work this paper cites.

Feasibility study on parameter adjustment for a humanoid using llm tailoring physical care

Tamon Miyake, Yushi Wang, Pin-chu Yang, and Shigeki Sugano · 2023

Earlier work this paper cites.

Llm-prop: Predicting physical and electronic properties of crystalline solids from their text descriptions

Original

Andre Niyongabo Rubungo, Craig Arnold, Barry P Rand, and Adji Bousso Dieng · 2023

Cited alongside, same era.

Large language models encode clinical knowledge

Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et al · 2023

Cited alongside, same era.

Scieval: A multi-level large language model evaluation benchmark for scientific research, 2023

Liangtai Sun, Yang Han, Zihan Zhao, Da Ma, Zhennan Shen, Baocai Chen, Lu Chen, and Kai Yu · 2023

Cited alongside, same era.

Llama: Open and efficient foundation language models, 2023

Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample · 2023

Cited alongside, same era.

Llama 2: Open foundation and fine-tuned chat models

Original

Llm based biological named entity recognition from scientific literature

Sung Jae Jung, Hajung Kim, and Kyoung Sang Jang · 2024

Closest in time.

Large language models in plant biology

Original

Hilbert Yuen In Lam, Xing Er Ong, and Marek Mutwil · 2024

Closest in time.

Salad-bench: A hierarchical and comprehensive safety benchmark for large language models

Original

Lijun Li, Bowen Dong, Ruohui Wang, Xuhao Hu, Wangmeng Zuo, Dahua Lin, Yu Qiao, and Jing Shao · 2024

Closest in time.

Augmenting large language models with chemistry tools

Andres M. Bran, Sam Cox, Oliver Schilter, Carlo Baldassari, Andrew D White, and Philippe Schwaller · 2024

Closest in time.

Ghs classification results, 2024

National Institute of Technology and Evaluation (NITE) · 2024

Closest in time.