Sound reconstruction from human brain activity via a generative model with brain-like auditory features

Understand

The successful reconstruction of perceptual experiences from human brain activity has provided insights into the neural representations of sensory experiences.

• However, reconstructing arbitrary sounds has been avoided due to the complexity of temporal sequences in sounds and the limited resolution of neuroimaging modalities.
• To overcome these challenges, leveraging the hierarchical nature of brain auditory processing could provide a path toward reconstructing arbitrary sounds.
• Previous studies have indicated a hierarchical homology between the human auditory system and deep neural network (DNN) models.