Generative Compression | Academic

GVCC: Zero-Shot Generative Video Compression

Fri, 27 Mar 2026 00:00:00 +0000

At ultra-low bitrates, high-fidelity reconstruction requires sampling plausible videos from the posterior rather than regressing to oversmoothed conditional means. GVCC (Generative Video Codebook Codec) turns a pretrained rectified-flow video generative model into a zero-shot decoder: the deterministic flow sampler is converted into an equivalent marginal-preserving stochastic process, and the transmitted bitstream encodes the per-step stochastic innovations that steer generation.

GVCC works in three practical modes: Text-to-Video (T2V), autoregressive Image-to-Video (I2V) with tail latent correction, and First-Last-Frame-to-Video (FLF2V) with boundary-sharing GOP chaining. (NeurIPS 2026)
GVCCTurbo is a BPP-driven scheduler that separates expensive prior refreshes from codebook corrections, turning bitrate into a schedule input and cutting prior evaluations from 20 to 9 (~44% decoding-time reduction).

Frame Interpolation for Generative Video Coding

Tue, 01 Oct 2024 00:00:00 +0000

With the continuous development of generative models (diffusion), this project explores a new video encoding paradigm starting from ultra-low bit-rates: only keyframes are transmitted, and the frames in between are regenerated by diffusion-based interpolation.

Bi-AGMI (Bidirectional Attention-Gated Motion Injection) connects forward and backward sampling trajectories via latent flipping and fuses them with a learnable attention gate, enabling stable keyframe interpolation over large temporal gaps with far fewer sampling steps. (IEEE GCCE 2025, Oral Presentation Award)
FRS (Flow Residual Segmentation) adaptively divides a video into segments where optical flow stays consistent, and combines this segment selection with diffusion-driven interpolation for flow-consistent reconstruction at extremely low bitrates. (IEVC 2026)

This research was conducted in collaboration with NICT Japan (情報通信研究機構).