Scalable Streaming Tools for Analyzing $N$-body Simulations: Finding Halos and Investigating Excursion Sets in One Pass

Understand

Cosmological $N$-body simulations play a vital role in studying models for the evolution of the Universe.

• To compare to observations and make a scientific inference, statistic analysis on large simulation datasets, e.g., finding halos, obtaining multi-point correlation functions, is crucial.
• However, traditional in-memory methods for these tasks do not scale to the datasets that are forbiddingly large in modern simulations.
• Our prior paper proposes memory-efficient streaming algorithms that can find the largest halos in a simulation with up to $10^9$ particles on a small server or desktop.