Neural Video Coding using Multiscale Motion Compensation and Spatiotemporal Context Model

Understand

Over the past two decades, traditional block-based video coding has made remarkable progress and spawned a series of well-known standards such as MPEG-4, H.264/AVC and H.265/HEVC.

• On the other hand, deep neural networks (DNNs) have shown their powerful capacity for visual content understanding, feature extraction and compact representation.
• Some previous works have explored the learnt video coding algorithms in an end-to-end manner, which show the great potential compared with traditional methods.
• In this paper, we propose an end-to-end deep neural video coding framework (NVC), which uses variational autoencoders (VAEs) with joint spatial and temporal prior aggregation (PA) to exploit the correlations in intra-frame pixels, inter-frame motions and inter-frame compensation residuals, respectively.