Mask-off: Synthesizing Face Images in the Presence of Head-mounted Displays

Understand

A head-mounted display (HMD) could be an important component of augmented reality system.

• However, as the upper face region is seriously occluded by the device, the user experience could be affected in applications such as telecommunication and multi-player video games.
• In this paper, we first present a novel experimental setup that consists of two near-infrared (NIR) cameras to point to the eye regions and one visible-light RGB camera to capture the visible face region.
• The main purpose of this paper is to synthesize realistic face images without occlusions based on the images captured by these cameras.