www.cgchannel.com
html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd"Originally posted on 11 September 2024, and updated with details of the public beta.Adobe has rolled out the Firefly Video Model, a new AI model that will bring generative AI to the companys video production tools like Premiere Pro, and potentially After Effects.Its capabilities include generating additional audio or video frames from an existing video clip, and generating new video clips from reference images or text prompts.Below, weve compiled our FAQs on the Firefly Video Model, which is now available in public beta on the Firefly web app, and in beta builds of Premiere Pro.What can Fireflys generative AI models currently do?Released last year, Firefly is a suite of generative AI art tools available both as a web application and integrated into Adobe Creative Cloud tools.The underlying AI models were initially geared towards generating still images, and are now integrated into applications including Photoshop and the Substance 3D tools.What are the key features of the Firefly Video Model?The new Firefly Video Model extends those capabilities to moving images, synthesising new video footage from a range of sources in a similar way to platforms like Runway.According to Adobe, its capabilities include Generative Extend, which extends existing video clips, generating new frames at the start or end, or generating extra audio room tone.Image to Video generates clips based on reference stills, or even concept illustrations.Text to Video generates video from text prompts, with the additional option to specify camera properties like camera position, zoom extent and common camera motion types.What kinds of video and animation can the Firefly Video Model generate?Although the Firefly Video Model can generate 2D and 3D animation Adobes initial blog post showed examples of generated footage resembling 3D, hand-drawn and stop-motion animation its main initial use case is more naturalistic video.According to Adobe, it excels at generating videos of the natural world like landscapes, plants or animals.However, the firm also cites a range of other possible uses, including creating FX elements like fire, smoke and dust that could be composited into VFX or motion graphics projects.How good is the Firefly Video Model?Although the Firefly Video Model could potentially generate entire video sequences, current generative video tools struggle to maintain temporal and stylistic consistency in their output.Adobes blog post suggests that its initial use will be to generate supplementary material for projects shot conventionally essentially, generating AI stock footage citing use cases like generating B-roll and missing establishing shots.What user controls does the Firefly Video Model provide?As well as the source image and/or text prompt it supports up to 1,000 characters of text the implementation of the Video Model in the Firefly web app includes built-in controls.Users can select the aspect ratio of the video generated (currently only 16:9 or 9:16), and choose from a range of presets for shot size (ranging from extreme close up to extreme long shots), camera angle (including aerial shots), and camera moves (including handheld shots).What are the limitations of the Firefly Video Model?In the initial public beta, generated clips are limited to 5 seconds in length and 1080p resolution.https://www.cgchannel.com/wp-content/uploads/2024/09/240911_AdobeFireflyVideoModel_GenerativeAI_PremierePro.mp4The Firefly Video Models Generative Extend feature in use in a Premiere Pro project.Which Adobe applications will the Firefly Video Model be integrated in?Although the Firefly tools are available online, Adobe is also increasingly integrating them into its Creative Cloud apps.So far, the only one in which the Firefly Video Model will definitely be included is video editing software Premiere Pro, beta builds of which now include Generative Extend.However, Adobes initial blog post also namechecked compositing app After Effects. The firm has previously shown tech previews of generative AI tools suitable for After Effects.What data is the Firefly Video AI model trained on?Adobe describes the Firefly Video AI model as commercially safe and is only trained on content we have permission to use.Earlier in 2024, the firm updated its Terms of Use to explicitly rule out the use of users content aside from that uploaded to stock content marketplace Adobe Stock to train AI models, following a widely publicised artist backlash.How much does it cost to use the Firefly Video AI model?As with many of the other Firefly services, pricing for the Firefly Video Model is credit-based.Adobe has introduced new pricing to coincide with the public beta. Generating video initially uses 20 credits per second of footage, set to rise to 100 credits per second.The new $9.99/month Firefly Standard plans provide 2,000 video credits per month; $29.99/month Firefly Pro plans provide 7,000 credits per month. Credits are also included as part of Creative Cloud subscriptions.When will the Firefly Video Model become available?The Firefly Video Model is now available via the Firefly web app. Text to Video and Image to Video are both in beta.Read more about the Firefly Video model on Adobes blogVisit Adobes Firefly web app(Includes Text to Video and Image to Video)Have your say on this story by following CG Channel on Facebook, Instagram and X (formerly Twitter). As well as being able to comment on stories, followers of our social media accounts can see videos we dont post on the site itself, including making-ofs for the latest VFX movies, animations, games cinematics and motion graphics projects.