It was a no-brainer that this year’s NAB show would have no shortage of big AI-related announcements. And so it goes: Adobe just shared a preview of the new video features they plan to include in Premiere Pro in 2024. Already this year, editors will be able to generate and add objects to shots, remove disturbing elements with a single click, and even zoom in the length of the footage. What’s interesting is that Adobe also anticipated its vision of integrating third-party generative models into editing software (yes, Open AI’s Sora is among them). Let’s take a closer look at what we can expect from the new generative AI in Adobe Premiere Pro.
Last year, during NAB 2023, Adobe only announced that it was working on one AI model: Firefly for video. Fast forward to today and we can glimpse the first experimental tests of this tool. Although the new technology is still considered to be in the research phase, what you see in the presentation below are real pixels generated with Adobe AI. Well, I can’t deny that the results look impressive!
Add or remove objects with generative AI in Adobe Premiere Pro
First, something we’ve been waiting for since this AI feature works great in Adobe Photoshop: Generative Fill. Editors will be able to select and track objects in shots and then replace them with automatically generated pixels. The preview shows Adobe Firefly removing elements from a scene entirely and adding something different in their place. The latter works based on text prompts, a familiar workflow as we know it from text-to-image generators.
In my opinion, this could become one of the most useful features to simplify and speed up editing workflows. Imagine: Quickly replacing a logo for a client that went through a rebrand, or removing a piece of equipment that crept into the final shot by mistake. Even the famous “Boom in the frame!” It may not be a problem anymore (just kidding!)
Extending the duration of shots with AI
Another feature that Adobe previewed is called Generative Extend and allows users to add frames to lengthen clips. Based on the demo, this workflow will work similarly to the Adobe Remix tool, another AI-enhanced feature we’re already familiar with. (However, this time not for soundtracks, but for video). Simply drag the right edge of your clip to extend it and let the AI do the rest:
The idea behind this seems plausible: editors often run into a problem when one or another shot is not long enough. However, when the video clip involves the actor’s performance, I fear that we could be entering a gray area. These are real people whose faces and actions will be generated by AI, even if only for a few frames. It still doesn’t seem right to me, so we’ll see what ethical solution Adobe offers.
Work with third-party generative AI models like Sora
A particularly exciting part of Adobe’s announcement is that the company decided not to compete in the video generator race. Instead, they previewed early explorations of bringing other famous AI models directly into Premiere Pro. Among them are the biggest ones on the market: Runway, Pika, and of course Sora from OpenAI.
While much of the initial conversation around generative AI has focused on a competition between companies to produce the “best” AI model, Adobe sees a much more diverse future.
A quote from Adobe’s press release
This way, users can choose between Adobe Firefly and other AI models directly in the editing software and choose the best result for their final cut. As we know, all video generators have their strengths. So, if you need a fully AI-generated shot for B-roll, Premiere Pro plans to offer it in different models as well:
Content Credentials for Generative AI in Adobe Premiere Pro
Of all the companies working on generative AI products, Adobe stands out for its ethical and transparent approach. You’ve probably already heard of “content credentials,” which the company developed to mark AI-generated footage. According to the demo preview, this tool is also coming to Premiere Pro. It will allow us to tag (and, as a result, the audience understand) when the AI was used and what model the creators applied in each shot.
AI-powered audio workflows in Premiere Pro
In addition to upcoming generative AI video tools, Adobe announced new audio workflows in Premiere Pro. The latest features include:
- Interactive fade controllers for simple custom audio transitions;
- New Essential Sound badge with AI audio category tagging (automatically tagging audio clips as dialogue, music, sound effects or ambience, and adding a new icon);
- Redesigned waveforms in the timeline that change size as the track height changes in clips.
…and more. These improvements will be generally available to Adobe Premiere Pro users starting in May.
Conclusion
Adobe did not announce exactly when it plans to release the video features shown. However, the developers mentioned that new generative AI tools will be available in Adobe Premiere Pro this year. We will keep you updated, so stay tuned!
What do you think of Adobe’s improvements and vision to integrate Firefly and other generative AI models directly into Premiere? Please share your thoughts with us in the comments below.
Featured image source: Adobe
#Adobe #Premiere #Pro #Introduce #Generative #Add #Objects #Zoom #Shots #AIEnhanced #Tools