NVIDIA introduced a neural network for generating video by description

NVIDIA Company announced a new VideoLDM AI model that creates short videos based on text. It was developed in collaboration with researchers at Cornell University.

VideoLDM takes into account up to 4.1 billion parameters, 2.7 billion of which are trained on video. Generated clips can be up to 2048×1280 pixels at 24 frames and have a duration of up to 4.7 seconds.

The neural network is capable of creating both simple scenes with a couple of words in the request, and something more complex. A few examples:

• Fireworks.

• A stormtrooper is vacuuming the beach.

• A traveler walks alone in a foggy forest at sunset.

More examples are on project website.

This NVIDIA neural network is not yet in the public domain. It was presented as a research paper within the framework of the Conference on Machine Vision and Pattern Recognition.

The developers noted impressive and rapid progress in learning, but did not talk about the possible future of the neural network. Nevertheless, we can assume that soon we will get a full-fledged video analogue