Built on

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei, “Language models are few-shot learners,” in Advances in Neural Information Processing Systems , Vol. 33, edited by H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (Curran Associates, Inc., 2020) pp. 1877–1901
1901
Earlier work this paper cites.
Simone Marzani, Gregory Soyez, and Michael Spannowsky, Looking inside jets: an introduction to jet substructure and boosted-object phenomenology , Vol. 958 (Springer, 2019) arXiv:1901.10342 [hep-ph]
Original
1901
Earlier work this paper cites.
Anja Butter et al. , “The Machine Learning landscape of top taggers,” SciPost Phys. 7
Original
1902
Earlier work this paper cites.
Michael R. Zhang, James Lucas, Geoffrey Hinton, and Jimmy Ba, “Lookahead optimizer: k steps forward, 1 step back,” Advances in Neural Information Processing Systems 32
Original
1907
Earlier work this paper cites.
Liyuan Liu, Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, and Jiawei Han, “On the variance of the adaptive learning rate and beyond,” International Conference on Learning Representations (2021), arXiv:1908.03265 [cs.LG]
Original
1908
Earlier work this paper cites.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer, “Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension,” (2019), arXiv:1910.13461 [cs.CL]
Original
1910
Earlier work this paper cites.
Kyle Cranmer, Johann Brehmer, and Gilles Louppe, “The frontier of simulation-based inference,” Proceedings of the National Academy of Science 117
Original
1911
Earlier work this paper cites.
Matteo Cacciari, Gavin P. Salam, and Gregory Soyez, “The anti- k t k_{t} jet clustering algorithm,” JHEP 04
Original
2008
Earlier work this paper cites.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby, “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale,” arXiv e-prints , arXiv:2010.11929 (2020) , arXiv:2010.11929 [cs.CV]
Original
2010
Earlier work this paper cites.
Jesse Thaler and Ken Van Tilburg, “Identifying Boosted Objects with N-subjettiness,” JHEP 03
Original
2011
Earlier work this paper cites.
J. de Favereau, C. Delaere, P. Demin, A. Giammanco, V. Lemaître, A. Mertens, and M. Selvaggi (DELPHES 3), “DELPHES 3, A modular framework for fast simulation of a generic collider experiment,” JHEP 02
Original
2014
Earlier work this paper cites.
Andrew J. Larkoski, Simone Marzani, Gregory Soyez, and Jesse Thaler, “Soft drop,” Journal of High Energy Physics 2014
2014
Earlier work this paper cites.
Diederik P. Kingma and Jimmy Ba, “Adam: A Method for Stochastic Optimization,” arXiv e-prints , arXiv:1412.6980 (2014) , arXiv:1412.6980 [cs.LG]
Original
2014
Earlier work this paper cites.
Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Ruslan Salakhutdinov, and Alexander Smola, “Deep Sets,” arXiv e-prints , arXiv:1703.06114 (2017) , arXiv:1703.06114 [cs.LG]
Original
2017
Earlier work this paper cites.
Gilles Louppe, Kyunghyun Cho, Cyril Becot, and Kyle Cranmer, “QCD-Aware Recursive Neural Networks for Jet Physics,” JHEP 01
Original
2019
Earlier work this paper cites.
Pablo De Castro and Tommaso Dorigo, “INFERNO: Inference-Aware Neural Optimisation,” Comput. Phys. Commun. 244
Original
2019
Earlier work this paper cites.

Similar

Patrick T. Komiske, Eric M. Metodiev, and Jesse Thaler, “Energy Flow Networks: Deep Sets for Particle Jets,” JHEP 01
Original
2019
Cited alongside, same era.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” (2019), arXiv:1810.04805 [cs.CL]
Original
2019
Cited alongside, same era.
Duarte Javier, “Sample with jet, track and secondary vertex properties for hbb tagging ml studies. cern open data portal.” (2019), DOI:10.7483/OPENDATA.CMS.JGJX.MS7Q
2019
Cited alongside, same era.
Gage DeZoort, Savannah Thais, Javier Duarte, Vesal Razavimaleki, Markus Atkinson, Isobel Ojalvo, Mark Neubauer, and Peter Elmer, “Charged Particle Tracking via Edge-Classifying Interaction Networks,” Comput. Softw. Big Sci. 5
Original
2021
Cited alongside, same era.

Then

Rachel E. C. Smith, Inês Ochoa, Rúben Inácio, Jonathan Shoemaker, and Michael Kagan, “Differentiable Vertex Fitting for Jet Flavour Tagging,” (2023), arXiv:2310.12804 [hep-ex]
Original
2023
Later among the works it cites.
Nathan Simpson and Lukas Heinrich, “neos: End-to-End-Optimised Summary Statistics for High Energy Physics,” J. Phys. Conf. Ser. 2438
Original
2023
Later among the works it cites.
Michael Kagan and Lukas Heinrich, “Branches of a Tree: Taking Derivatives of Programs with Discrete and Branching Randomness in High Energy Physics,” (2023), arXiv:2308.16680 [stat.ML]
Original
2023
Later among the works it cites.
Lukas Heinrich and Michael Kagan, “Differentiable Matrix Elements with MadJax,” J. Phys. Conf. Ser. 2438
Original
2023
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Reading the bibliography…

2024

Finetuning Foundation Models for Joint Analysis Optimization

Vigl, Matthias, Hartman, Nicole, Heinrich, Lukas

Understand

In this work we demonstrate that significant gains in performance and data efficiency can be achieved in High Energy Physics (HEP) by moving beyond the standard paradigm of sequential optimization or reconstruction and analysis components.

We conceptually connect HEP reconstruction and analysis to modern machine learning workflows such as pretraining, finetuning, domain adaptation and high-dimensional embedding spaces and quantify the gains in the example usecase of searches of heavy resonances decaying via an intermediate di-Higgs system to four $b$-jets.

Built on

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei, “Language models are few-shot learners,” in Advances in Neural Information Processing Systems , Vol. 33, edited by H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (Curran Associates, Inc., 2020) pp. 1877–1901
1901
Earlier work this paper cites.
Simone Marzani, Gregory Soyez, and Michael Spannowsky, Looking inside jets: an introduction to jet substructure and boosted-object phenomenology , Vol. 958 (Springer, 2019) arXiv:1901.10342 [hep-ph]
Original
1901
Earlier work this paper cites.
Anja Butter et al. , “The Machine Learning landscape of top taggers,” SciPost Phys. 7
Original
1902
Earlier work this paper cites.
Michael R. Zhang, James Lucas, Geoffrey Hinton, and Jimmy Ba, “Lookahead optimizer: k steps forward, 1 step back,” Advances in Neural Information Processing Systems 32
Original
1907
Earlier work this paper cites.
Liyuan Liu, Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, and Jiawei Han, “On the variance of the adaptive learning rate and beyond,” International Conference on Learning Representations (2021), arXiv:1908.03265 [cs.LG]
Original
1908
Earlier work this paper cites.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer, “Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension,” (2019), arXiv:1910.13461 [cs.CL]
Original
1910
Earlier work this paper cites.
Kyle Cranmer, Johann Brehmer, and Gilles Louppe, “The frontier of simulation-based inference,” Proceedings of the National Academy of Science 117
Original
1911
Earlier work this paper cites.
Matteo Cacciari, Gavin P. Salam, and Gregory Soyez, “The anti- k t k_{t} jet clustering algorithm,” JHEP 04
Original
2008
Earlier work this paper cites.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby, “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale,” arXiv e-prints , arXiv:2010.11929 (2020) , arXiv:2010.11929 [cs.CV]
Original
2010
Earlier work this paper cites.
Jesse Thaler and Ken Van Tilburg, “Identifying Boosted Objects with N-subjettiness,” JHEP 03
Original
2011
Earlier work this paper cites.
J. de Favereau, C. Delaere, P. Demin, A. Giammanco, V. Lemaître, A. Mertens, and M. Selvaggi (DELPHES 3), “DELPHES 3, A modular framework for fast simulation of a generic collider experiment,” JHEP 02
Original
2014
Earlier work this paper cites.
Andrew J. Larkoski, Simone Marzani, Gregory Soyez, and Jesse Thaler, “Soft drop,” Journal of High Energy Physics 2014
2014
Earlier work this paper cites.
Diederik P. Kingma and Jimmy Ba, “Adam: A Method for Stochastic Optimization,” arXiv e-prints , arXiv:1412.6980 (2014) , arXiv:1412.6980 [cs.LG]
Original
2014
Earlier work this paper cites.
Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Ruslan Salakhutdinov, and Alexander Smola, “Deep Sets,” arXiv e-prints , arXiv:1703.06114 (2017) , arXiv:1703.06114 [cs.LG]
Original
2017
Earlier work this paper cites.
Gilles Louppe, Kyunghyun Cho, Cyril Becot, and Kyle Cranmer, “QCD-Aware Recursive Neural Networks for Jet Physics,” JHEP 01
Original
2019
Earlier work this paper cites.
Pablo De Castro and Tommaso Dorigo, “INFERNO: Inference-Aware Neural Optimisation,” Comput. Phys. Commun. 244
Original
2019
Earlier work this paper cites.

Similar

Patrick T. Komiske, Eric M. Metodiev, and Jesse Thaler, “Energy Flow Networks: Deep Sets for Particle Jets,” JHEP 01
Original
2019
Cited alongside, same era.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” (2019), arXiv:1810.04805 [cs.CL]
Original
2019
Cited alongside, same era.
Duarte Javier, “Sample with jet, track and secondary vertex properties for hbb tagging ml studies. cern open data portal.” (2019), DOI:10.7483/OPENDATA.CMS.JGJX.MS7Q
2019
Cited alongside, same era.
Gage DeZoort, Savannah Thais, Javier Duarte, Vesal Razavimaleki, Markus Atkinson, Isobel Ojalvo, Mark Neubauer, and Peter Elmer, “Charged Particle Tracking via Edge-Classifying Interaction Networks,” Comput. Softw. Big Sci. 5
Original
2021
Cited alongside, same era.

Then

Rachel E. C. Smith, Inês Ochoa, Rúben Inácio, Jonathan Shoemaker, and Michael Kagan, “Differentiable Vertex Fitting for Jet Flavour Tagging,” (2023), arXiv:2310.12804 [hep-ex]
Original
2023
Later among the works it cites.
Nathan Simpson and Lukas Heinrich, “neos: End-to-End-Optimised Summary Statistics for High Energy Physics,” J. Phys. Conf. Ser. 2438
Original
2023
Later among the works it cites.
Michael Kagan and Lukas Heinrich, “Branches of a Tree: Taking Derivatives of Programs with Discrete and Branching Randomness in High Energy Physics,” (2023), arXiv:2308.16680 [stat.ML]
Original
2023
Later among the works it cites.
Lukas Heinrich and Michael Kagan, “Differentiable Matrix Elements with MadJax,” J. Phys. Conf. Ser. 2438
Original
2023
Later among the works it cites.

Beyond the bibliography

alphaXiv searches the wider corpus for related work and actual follow-ups.

Open on alphaXiv

alphaXiv is searching for related work…

Joosep Pata, Javier Duarte, Jean-Roch Vlimant, Maurizio Pierini, and Maria Spiropulu, “MLPF: efficient machine-learned particle-flow reconstruction using graph neural networks,” The European Physical Journal C 81

2021

Cited alongside, same era.

Lukas Heinrich, Matthew Feickert, Giordon Stark, and Kyle Cranmer, “pyhf: pure-python implementation of histfactory statistical models,” Journal of Open Source Software 6

2021

Cited alongside, same era.

Adrien Bardes, Jean Ponce, and Yann LeCun, “VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning,” arXiv e-prints , arXiv:2105.04906 (2021) , arXiv:2105.04906 [cs.CV]