The Diffusion Model and LLM Model have indeed attracted attention in the field of video generation and are considered emerging methods. The application of these models in video generation may be based on their success in other domains and their potential in handling sequential data and generating content.
The Diffusion Model is a generative model used to model data distributions and generate realistic samples. Its strength lies in its ability to handle high-dimensional data and complex data distributions, making it suitable for generating images and video content. By simulating the process of data points diffusing in a probabilistic field, the Diffusion Model can generate high-quality images and video sequences.
The Large Lauguage Model Model is a generative model based on language learning, commonly used in natural language processing tasks. Recent research has shown that LLM models excel in handling sequential data and can be used to generate text, audio, and video content. The strength of LLM models lies in their ability to capture long-term dependencies and contextual information, thereby generating coherent and semantically meaningful content.
It trains text-conditional diffusion models jointly on videos and images of variable durations, resolutions, and aspect ratios. Leveraging a transformer architecture that operates on spacetime patches of video and image latent codes(LLM). Sora is capable of generating a minute of high-fidelity video content now.