Contrast, Attend and Diffuse to Decode High-Resolution Images from Brain Activities

Understand

Decoding visual stimuli from neural responses recorded by functional Magnetic Resonance Imaging (fMRI) presents an intriguing intersection between cognitive neuroscience and machine learning, promising advancements in understanding human visual perception and building non-invasive brain-machine interfaces.

• However, the task is challenging due to the noisy nature of fMRI signals and the intricate pattern of brain visual representations.
• To mitigate these challenges, we introduce a two-phase fMRI representation learning framework.
• The first phase pre-trains an fMRI feature learner with a proposed Double-contrastive Mask Auto-encoder to learn denoised representations.