Text-to-Edit: Controllable End-to-End Video Ad Creation via Multimodal LLMs

Understand

The exponential growth of short-video content has ignited a surge in the necessity for efficient, automated solutions to video editing, with challenges arising from the need to understand videos and tailor the editing according to user requirements.

• Addressing this need, we propose an innovative end-to-end foundational framework, ultimately actualizing precise control over the final video content editing.
• Leveraging the flexibility and generalizability of Multimodal Large Language Models (MLLMs), we defined clear input-output mappings for efficient video creation.
• To bolster the model's capability in processing and comprehending video content, we introduce a strategic combination of a denser frame rate and a slow-fast processing technique, significantly enhancing the extraction and understanding of both temporal and spatial video information.