Tencent, one of China’s leading tech giants, has made a significant move in the text-to-video space by unveiling its new open source video generation model, DynamiCrafter. This development highlights the growing efforts of Chinese tech firms to make their mark in the field of generative videos, following the success of generative text and images.
DynamiCrafter, like other generative video tools, utilizes the diffusion method to transform captions and still images into short videos. Inspired by the natural phenomenon of diffusion in physics, this machine learning model can convert simple data into more complex and realistic content. It mimics how particles move from areas of high concentration to areas of low concentration.
The latest version of DynamiCrafter boasts an upgrade in pixel resolution, now producing videos at 640×1024, compared to its initial release with 320×512 videos in October. The team behind DynamiCrafter explains that their technology sets itself apart from competitors by expanding the applicability of image animation techniques to a wider range of visual content.
According to an academic paper published by the team, DynamiCrafter incorporates the motion prior of text-to-video diffusion models by integrating images into the generative process as guidance. In contrast, traditional techniques mainly focus on animating natural scenes or specific motions, such as clouds, fluid, human hair, or body movements.
In a demo comparing DynamiCrafter with Stable Video Diffusion and Pika Labs, Tencent’s model appears slightly more animated. However, it is important to note that the chosen samples may favor DynamiCrafter, and none of the models, even after several attempts, give the impression that AI will soon be capable of producing full-fledged movies.
Despite this limitation, generative videos are highly anticipated as the next frontier in the AI race, following the success of generative text and images. As a result, both startups and established tech companies are investing significant resources in this field. Tencent is not alone in its pursuit, as ByteDance (the parent company of TikTok), Baidu, and Alibaba have also released their video diffusion models.
ByteDance’s MagicVideo and Baidu’s UniVG have shared demos on GitHub, although they are not yet available to the public. Similarly, Alibaba has made its video generation model, VGen, open source, a strategy increasingly adopted by Chinese tech firms aiming to engage with the global developer community.
With the introduction of DynamiCrafter, Tencent has joined the ranks of China’s tech giants venturing into the text-to-video space. As the competition intensifies, it will be interesting to see how these companies continue to innovate and push the boundaries of generative video technology.
