Frame Interpolation for Generative Video Coding

With the continuous development of generative models (diffusion), this project explores a new video encoding paradigm starting from ultra-low bit-rates: only keyframes are transmitted, and the frames in between are regenerated by diffusion-based interpolation.

• Bi-AGMI (Bidirectional Attention-Gated Motion Injection) connects forward and backward sampling trajectories via latent flipping and fuses them with a learnable attention gate, enabling stable keyframe interpolation over large temporal gaps with far fewer sampling steps. (IEEE GCCE 2025, Oral Presentation Award)
• FRS (Flow Residual Segmentation) adaptively divides a video into segments where optical flow stays consistent, and combines this segment selection with diffusion-driven interpolation for flow-consistent reconstruction at extremely low bitrates. (IEVC 2026)

This research was conducted in collaboration with NICT Japan (情報通信研究機構).