Transformers Can Do Bayesian Inference

Understand

Currently, it is hard to reap the benefits of deep learning for Bayesian methods, which allow the explicit specification of prior knowledge and accurately capture model uncertainty.

• We present Prior-Data Fitted Networks (PFNs).
• PFNs leverage in-context learning in large-scale machine learning techniques to approximate a large set of posteriors.
• The only requirement for PFNs to work is the ability to sample from a prior distribution over supervised learning tasks (or functions).