Fetching the paper…
Reading the bibliography…
Since language models (LMs) now outperform average humans on many challenging tasks, it has become increasingly difficult to develop challenging, high-quality, and realistic evaluations.
Elements of physical biology
A. J. Lotka · 1925
Earlier work this paper cites.
Variations and fluctuations of the number of individuals in animal species living together
V. Volterra · 1928
Earlier work this paper cites.
Crystal statistics. i. a two-dimensional model with an order-disorder transition
L. Onsager · 1944
Earlier work this paper cites.
Crystal statistics. i. a two-dimensional model with an order-disorder transition
L. Onsager · 1944
Earlier work this paper cites.
Equation of state calculations by fast computing machines
N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller · 1953
Earlier work this paper cites.
Remarks on the unified model of elementary particles
Z. Maki, M. Nakagawa, and S. Sakata · 1962
Earlier work this paper cites.
Some Topics in the Theory of Fluids
B. Widom · 1963
Earlier work this paper cites.
Self-consistent equations including exchange and correlation effects
W. Kohn and L. J. Sham · 1965
Earlier work this paper cites.
Acceleration and trapping of particles by radiation pressure
A. Ashkin · 1970
Earlier work this paper cites.
Solvable model of a spin-glass
D. Sherrington and S. Kirkpatrick · 1975
Earlier work this paper cites.
A general method for numerically simulating the stochastic time evolution of coupled chemical reactions
D. T. Gillespie · 1976
Earlier work this paper cites.
Exact stochastic simulation of coupled chemical reactions
D. T. Gillespie · 1977
Earlier work this paper cites.
Hydrodynamic fluctuations at the convective instability
J. Swift and P. Hohenberg · 1977
Earlier work this paper cites.
A sequence of approximated solutions to the sk model for spin glasses
G. Parisi · 1980
Earlier work this paper cites.
Fixed-node quantum monte carlo for molecules
P. J. Reynolds, D. M. Ceperley, B. J. Alder, and W. A. Lester Jr · 1982
Earlier work this paper cites.
Dielectric response of a semi-infinite layered electron gas and raman scattering from its bulk and surface plasmons
J. K. Jain and P. B. Allen · 1985
Earlier work this paper cites.
Grand canonical Monte Carlo simulation of liquid argon
I. Ruff, A. Baranyai, G. Pálinkás, and K. Heinzinger · 1986
Earlier work this paper cites.
Spin glass theory and beyond: An Introduction to the Replica Method and Its Applications
M. Mézard, G. Parisi, and M. A. Virasoro · 1987
Earlier work this paper cites.
Model for a quantum hall effect without landau levels: Condensed-matter realization of the" parity anomaly"
F. D. M. Haldane · 1988
Earlier work this paper cites.
Pattern formation outside of equilibrium
M. Cross and P. Hohenberg · 1993
Earlier work this paper cites.
Numerical simulations of spin glass systems
E. Marinari, G. Parisi, and J. Ruiz-Lorenzo · 1998
Earlier work this paper cites.
Independent component analysis: algorithms and applications
A. Hyvärinen and E. Oja · 2000
Earlier work this paper cites.
Three flavor neutrino oscillations in matter
T. Ohlsson and H. Snellman · 2000
Earlier work this paper cites.
Quantum monte carlo simulations of solids
W. M. Foulkes, L. Mitas, R. Needs, and G. Rajagopal · 2001
Earlier work this paper cites.
Recursions for the computation of multipole translation and rotation coefficients for the 3-d helmholtz equation
N. A. Gumerov and R. Duraiswami · 2004
Earlier work this paper cites.
Computation of total capacity for discrete memoryless multiple-access channels
M. Rezaeian and A. Grant · 2004
Earlier work this paper cites.
Registry-dependent interlayer potential for graphitic systems
A. N. Kolmogorov and V. H. Crespi · 2005
Earlier work this paper cites.
Weak imposition of dirichlet boundary conditions in fluid mechanics
Y. Bazilevs and T. J. R. Hughes · 2007
Earlier work this paper cites.
Pattern Formation and Dynamics in Nonequilibrium Systems
M. Cross and H. Greenside · 2009
Earlier work this paper cites.
Measurement of the instantaneous velocity of a brownian particle
T. Li, S. Kheifets, D. Medellin, and M. G. Raizen · 2010
Earlier work this paper cites.
Energy spectrum and quantum hall effect in twisted bilayer graphene
P. Moon and M. Koshino · 2012
Earlier work this paper cites.
Brownian motion at short time scales
T. Li and M. G. Raizen · 2013
Earlier work this paper cites.
The fast azimuthal integration python library: pyfai
G. Ashiotis, A. Deschildre, Z. Nawaz, J. P. Wright, D. Karkoulis, F. E. Picca, and J. Kieffer · 2015
Earlier work this paper cites.
Measurement of the dynamic charge response of materials using low-energy, momentum-resolved electron energy-loss spectroscopy (m-eels)
S. Vig, A. Kogar, M. Mitrano, A. Husain, L. Venema, M. Rak, V. Mishra, P. Johnson, G. Gu, E. Fradkin, et al · 2017
Cited alongside, same era.
Bert: Pre-training of deep bidirectional transformers for language understanding, 2018
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova · 2018
Cited alongside, same era.
Diffuse Scattering and Local Order in Lead-Based Relaxor Ferroelectrics
M. J. Krogstad · 2018
Cited alongside, same era.
Nanoserpents: Graphene nanoribbon motion on two-dimensional hexagonal materials
W. Ouyang, D. Mandelli, M. Urbakh, and O. Hod · 2018
Cited alongside, same era.
Onset of natural selection in populations of autocatalytic heteropolymers
A. V. Tkachenko and S. Maslov · 2018
Cited alongside, same era.
The impact of large language models on scientific discovery: a preliminary study using gpt-4, 2023
M. R. AI4Science and M. A. Quantum · 2023
Later among the works it cites.
Qwen technical report, 2023
J. Bai, S. Bai, Y. Chu, Z. Cui, K. Dang, X. Deng, Y. Fan, W. Ge, Y. Han, F. Huang, B. Hui, L. Ji, M. Li, J. Lin, R. Lin, D. Liu, G. Liu, C. Lu, K. Lu, J. Ma, R. Men, X. Ren, X. Ren, C. Tan, S. Tan, J. Tu, P. Wang, S. Wang, W. Wang, S. Wu, B. Xu, J. Xu, A. Yang, H. Yang, J. Yang, S. Yang, Y. Yao, B. Yu, H. Yuan, Z. Yuan, J. Zhang, X. Zhang, Y. Zhang, Z. Zhang, C. Zhou, J. Zhou, X. Zhou, and T. Zhu · 2023
Later among the works it cites.
Meditron-70b: Scaling medical pretraining for large language models, 2023
Z. Chen, A. H. Cano, A. Romanou, A. Bonnet, K. Matoba, F. Salvi, M. Pagliardini, S. Fan, A. Köpf, A. Mohtashami, A. Sallinen, A. Sakhaeirad, V. Swamy, I. Krawczuk, D. Bayazit, A. Marmet, S. Montariol, M.-A. Hartley, M. Jaggi, and A. Bosselut · 2023
Later among the works it cites.
Crosscodeeval: A diverse and multilingual benchmark for cross-file code completion
Y. Ding, Z. Wang, W. U. Ahmad, H. Ding, M. Tan, N. Jain, M. K. Ramanathan, R. Nallapati, P. Bhatia, D. Roth, and B. Xiang · 2023
Later among the works it cites.
Pal: program-aided language models
L. Gao, A. Madaan, S. Zhou, U. Alon, P. Liu, Y. Yang, J. Callan, and G. Neubig · 2023
alphaXiv searches the wider corpus for related work and actual follow-ups.
alphaXiv is searching for related work…
GLUE: A multi-task benchmark and analysis platform for natural language understanding
A. Wang, A. Singh, J. Michael, F. Hill, O. Levy, and S. Bowman · 2018
Cited alongside, same era.
Learning to mine aligned code and natural language pairs from stack overflow
P. Yin, B. Deng, E. Chen, B. Vasilescu, and G. Neubig · 2018
Cited alongside, same era.
Codebert: A pre-trained model for programming and natural languages, 2020
Z. Feng, D. Guo, D. Tang, N. Duan, X. Feng, M. Gong, L. Shou, B. Qin, T. Liu, D. Jiang, and M. Zhou · 2020
Cited alongside, same era.
Information-theoretic aspects of the generalized amplitude-damping channel
S. Khatri, K. Sharma, and M. M. Wilde · 2020
Cited alongside, same era.
Reciprocal space imaging of ionic correlations in intercalation compounds
M. J. Krogstad, S. Rosenkranz, J. M. Wozniak, G. Jennings, J. P. Ruff, J. T. Vaughey, and R. Osborn · 2020
Cited alongside, same era.
Prethermalization and nonreciprocal phonon transport in a levitated optomechanical array
S. Liu, Z.-q. Yin, and T. Li · 2020
Cited alongside, same era.
A diverse corpus for evaluating and developing English math word problem solvers
S.-y. Miao, C.-C. Liang, and K.-Y. Su · 2020
Cited alongside, same era.
Later among the works it cites.
Pines’ demon observed as a 3d acoustic plasmon in sr2ruo4
A. A. Husain, E. W. Huang, M. Mitrano, M. S. Rak, S. I. Rubeck, X. Guo, H. Yang, C. Sow, Y. Maeno, B. Uchoa, et al · 2023
Later among the works it cites.
Registry-dependent potential energy and lattice corrugation of twisted bilayer graphene from quantum monte carlo
K. Krongchon, T. Rakib, S. Pathak, E. Ertekin, H. T. Johnson, and L. K. Wagner · 2023
Later among the works it cites.
Starcoder: may the source be with you!
R. Li, L. B. allal, Y. Zi, N. Muennighoff, D. Kocetkov, C. Mou, M. Marone, C. Akiki, J. LI, J. Chim, Q. Liu, E. Zheltonozhskii, T. Y. Zhuo, T. Wang, O. Dehaene, J. Lamy-Poirier, J. Monteiro, N. Gontier, M.-H. Yee, L. K. Umapathi, J. Zhu, B. Lipkin, M. Oblokulov, Z. Wang, R. Murthy, J. T. Stillerman, S. S. Patel, D. Abulkhanov, M. Zocca, M. Dey, Z. Zhang, U. Bhattacharyya, W. Yu, S. Luccioni, P. Villegas, F. Zhdanov, T. Lee, N. Timor, J. Ding, C. S. Schlesinger, H. Schoelkopf, J. Ebert, T. Dao, M. Mishra, A. Gu, C. J. Anderson, B. Dolan-Gavitt, D. Contractor, S. Reddy, D. Fried, D. Bahdanau, Y. Jernite, C. M. Ferrandis, S. Hughes, T. Wolf, A. Guha, L. V. Werra, and H. de Vries · 2023
Later among the works it cites.
Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation, 2023
J. Liu, C. S. Xia, Y. Wang, and L. Zhang · 2023
Later among the works it cites.
Codegen: An open large language model for code with multi-turn program synthesis, 2023
E. Nijkamp, B. Pang, H. Hayashi, L. Tu, H. Wang, Y. Zhou, S. Savarese, and C. Xiong · 2023
Later among the works it cites.
Gpt-4 technical report, 2023
OpenAI · 2023
Later among the works it cites.
Mapping structural correlations in real space
R. Osborn · 2023
Later among the works it cites.
Gpqa: A graduate-level google-proof q&a benchmark, 2023
D. Rein, B. L. Hou, A. C. Stickland, J. Petty, R. Y. Pang, J. Dirani, J. Michael, and S. R. Bowman · 2023
Later among the works it cites.
Large language models encode clinical knowledge
K. Singhal, S. Azizi, T. Tu, S. S. Mahdavi, J. Wei, H. W. Chung, N. Scales, A. Tanwani, H. Cole-Lewis, S. Pfohl, P. Payne, M. Seneviratne, P. Gamble, C. Kelly, A. Babiker, N. Schärli, A. Chowdhery, P. Mansfield, D. Demner-Fushman, B. Agüera y Arcas, D. Webster, G. S. Corrado, Y. Matias, K. Chou, J. Gottweis, N. Tomasev, Y. Liu, A. Rajkomar, J. Barral, C. Semturs, A. Karthikesalingam, and V. Natarajan · 2023
Later among the works it cites.
Mint: Evaluating llms in multi-turn interaction with tools and language feedback
X. Wang, Z. Wang, J. Liu, Y. Chen, L. Yuan, H. Peng, and H. Ji · 2023
Later among the works it cites.
The ecological consequences of microbial metabolic strategies in fluctuating environments
Z. Wang, A. Goyal, and S. Maslov · 2023
Later among the works it cites.
Pyqmc: An all-python real-space quantum monte carlo module in pyscf
W. A. Wheeler, S. Pathak, K. G. Kleiner, S. Yuan, J. N. Rodrigues, C. Lorsung, K. Krongchon, Y. Chang, Y. Zhou, B. Busemeyer, et al · 2023
Later among the works it cites.
Llama 3 model card
AI@Meta · 2024
Closest in time.
Introducing the next generation of claude, 2024
Anthropic · 2024
Closest in time.
Introducing the next generation of claude 3.5, 2024
Anthropic · 2024
Closest in time.
Consistency between reflection momentum-resolved electron energy loss spectroscopy and optical spectroscopy measurements of the long-wavelength density response of bi 2 sr 2 cacu 2 o 8+ x
J. Chen, X. Guo, C. Boyd, S. Bettler, C. Kengle, D. Chaudhuri, F. Hoveyda, A. Husain, J. Schneeloch, G. Gu, et al · 2024
Closest in time.
Deepseek-coder: When the large language model meets programming – the rise of code intelligence, 2024
D. Guo, Q. Zhu, D. Yang, Z. Xie, K. Dong, W. Zhang, G. Chen, X. Bi, Y. Wu, Y. K. Li, F. Luo, Y. Xiong, and W. Liang · 2024
Closest in time.
Mixtral of experts, 2024
A. Q. Jiang, A. Sablayrolles, A. Roux, A. Mensch, B. Savary, C. Bamford, D. S. Chaplot, D. de las Casas, E. B. Hanna, F. Bressand, G. Lengyel, G. Bour, G. Lample, L. R. Lavaud, L. Saulnier, M.-A. Lachaux, P. Stock, S. Subramanian, S. Yang, S. Antoniak, T. L. Scao, T. Gervet, T. Lavril, T. Wang, T. Lacroix, and W. E. Sayed · 2024
Closest in time.
Swe-bench: Can language models resolve real-world github issues?, 2024
C. E. Jimenez, J. Yang, A. Wettig, S. Yao, K. Pei, O. Press, and K. Narasimhan · 2024
Closest in time.
Q. Li, L. Cui, X. Zhao, L. Kong, and W. Bi · 2024
Closest in time.
Starcoder 2 and the stack v2: The next generation, 2024
A. Lozhkov, R. Li, L. B. Allal, F. Cassano, J. Lamy-Poirier, N. Tazi, A. Tang, D. Pykhtar, J. Liu, Y. Wei, T. Liu, M. Tian, D. Kocetkov, A. Zucker, Y. Belkada, Z. Wang, Q. Liu, D. Abulkhanov, I. Paul, Z. Li, W.-D. Li, M. Risdal, J. Li, J. Zhu, T. Y. Zhuo, E. Zheltonozhskii, N. O. O. Dade, W. Yu, L. Krauß, N. Jain, Y. Su, X. He, M. Dey, E. Abati, Y. Chai, N. Muennighoff, X. Tang, M. Oblokulov, C. Akiki, M. Marone, C. Mou, M. Mishra, A. Gu, B. Hui, T. Dao, A. Zebaze, O. Dehaene, N. Patry, C. Xu, J. McAuley, H. Hu, T. Scholak, S. Paquet, J. Robinson, C. J. Anderson, N. Chapados, M. Patwary, N. Tajbakhsh, Y. Jernite, C. M. Ferrandis, L. Zhang, S. Hughes, T. Wolf, A. Guha, L. von Werra, and H. de Vries · 2024
Closest in time.
Hello gpt-4o, 2024
OpenAI · 2024
Closest in time.
Code llama: Open foundation models for code, 2024
B. Roziere, J. Gehring, F. Gloeckle, S. Sootla, I. Gat, X. E. Tan, Y. Adi, J. Liu, R. Sauvestre, T. Remez, J. Rapin, A. Kozhevnikov, I. Evtimov, J. Bitton, M. Bhatt, C. C. Ferrer, A. Grattafiori, W. Xiong, A. Defossez, J. Copet, F. Azhar, H. Touvron, L. Martin, N. Usunier, T. Scialom, and G. Synnaeve · 2024
Closest in time.
Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024
Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, M. Zhang, Y. Li, Y. Wu, and D. Guo · 2024
Closest in time.
Introducing qwen2, June 2024
Q. Team · 2024
Closest in time.
Scibench: Evaluating college-level scientific problem-solving abilities of large language models, 2024
X. Wang, Z. Hu, P. Lu, Y. Zhu, J. Zhang, S. Subramaniam, A. R. Loomba, S. Zhang, Y. Sun, and W. Wang · 2024
Closest in time.
Emergent ecological advantage of sequential metabolic strategies in complex microbial communities
Z. Wang, Y. Fu, A. Goyal, and S. Maslov · 2024
Closest in time.
Deepseek-coder-v2: Breaking the barrier of closed-source models in code intelligence
Q. Zhu, D. Guo, Z. Shao, D. Yang, P. Wang, R. Xu, Y. Wu, Y. Li, H. Gao, S. Ma, et al · 2024
Closest in time.