AI video-making: the good, the bad and the slightly creepy

Now that we’re all embracing AI for writing and image creation, will we soon be able to make videos with a prompt and a click? The technology is in its infancy and the videos made by some AI generators can be a bit…creepy. So maybe not quite yet. However, there are some great AI tools out there and, of course, there’s always a remake on the horizon.

Video content is a powerful way to communicate with your audience and an increasingly important part of any marketing strategy. But making videos the old-fashioned way is not cheap, easy or quick (on average it takes marketers two weeks to create a video from start to finish).

With AI video tools, you can – theoretically at least – make a video from scratch without the need for actors, film crew, equipment, video editing skills, or even a location. The technology still has a long way to go, however. The results you get from AI text-to-video tools (which generate videos in response to text prompts) have a particularly large amount of room for improvement. 

Jota Gonzalez, elespacio’s Motion & Graphic designer, says “For making video straight from scratch, the technology is not quite there yet. It’s a bit messy. If you want a creepy or weird video, you can create something pretty spectacular, but for good quality commercial videos, we have some way to go.”

There are some tools already out there, however, particularly in the pre-production and post-production space, that make the videographer’s life easier, from smart editing apps to compression technology. Here are just a few of them. 

Pre-production: selling the story

For now, most pre-production tasks are done by humans (finding the location, booking the models etc.). But one important and time-consuming step that AI can help with is writing the story and turning it into images.

“Sometimes you have the resources and capability to produce a video – but you don’t have the resources to sell the idea,” says Jota. “As a company or an individual, you might not have an artist to create a storyboard for you.”

One really exciting new addition to the AI video world is Adobe’s text-to-storyboard tool, part of its Firefly generative AI family, which it is integrating into its video and motion graphic design apps.  

This does just what the name implies - you upload your script, and Firefly turns it into a visual storyboard ready for presentation. “I think this tool could work wonders,” says Jota. “Anything that helps you to sell the idea – and do it faster – is great.”

Lights, camera, action: AI video generators

AI video generators create unique videos, based on nothing other than a script and/or a few prompts, images or other data that you provide. 

One of these is Kaiber,  self-described as ‘Made for artists, by artists’, which is mostly known for making music videos – upload your music and Kaiber will interpret it and create images that move with the beat. Linkin’ Park used it for their video for “Lost’, apparently.      

Another is Phenaki, which, according to Jota, does “literally what Midjourney does with images, but with video” and creates videos in response to text prompts. At the time of writing it is still in development and we found it to be user-unfriendly with results that are at the ‘creepy’ end of the spectrum. But the great thing is that it keeps the same look and feel throughout a project, even when the scene changes, so the storytelling is seamless.  

Example

Prompts used:

A photorealistic teddy bear is swimming in the ocean at San Francisco

The teddy bear keeps swimming under the water with colorful fishes

A panda bear is swimming under water

Lumiere3D is a tool that uses an intuitive video editor to create product animation videos. You scan your product with your smartphone and it recreates it in 3D, complete with all the details and shadows. You can then add effects, camera movements and even music.

Post-production: bringing it all together

There are lots of AI apps that streamline the editing and production process. Videos can often be edited automatically – colour correction, object removal, image stabilization and transcribing dialogue are all a breeze for AI.

One of the most powerful post-production tools is Runway. It has editing features such as rotoscoping and deleting objects, filters and motion tracking, which you can do on the fly in the app. You can generate music and sound effects to enhance videos and it has automatic translation and creative text generation. 

Here are some examples:

It also has text-to-video capabilities, like Phenaki, and while it is not as powerful as Phenaki and the integrated motion is a bit messy, it is much more user-friendly and it’s really fun to play with. 

A compressing question: DeepRender

Image and video data comprises most of the information transmitted on the internet and is expected to grow significantly in the coming years. This uses up bandwidth and energy and generates CO2 at an unsustainable rate.

An R&D start-up called DeepRender has developed a new approach to video compression, which uses AI, combined with statistics and information theory, to mimic neurological processes of the human eye. The technology has already been developed and is ready to come to the market, where it will help increase internet bandwidth supply and reduce energy usage. 

Grab the popcorn

We are still in the baby stages of AI video, and it’s a long way behind AI image or text generation. The tools mentioned here will either be unrecognisable or non-existent in a year’s time, and there will certainly be many more of them.

There will be ethical concerns (e.g. deepfakes) to navigate and there will be many twists in this plot. But if it takes on the more laborious and expensive tasks of video production, leaving creators to create – then AI will surely become a video star. Stay tuned.