The era of text-only video generation is fading fast. Leading AI developers are introducing models that can interpret not just written prompts, but a wide range of reference inputs, from images and video clips to audio tracks, documents, and even web pages.
Vendors Expand Input Capabilities for AI Video Models
Kuaishou’s Kling 3.0, unveiled in February, allows creators to upload multiple images and reference video to preserve characters and scenes. ByteDance’s Seedance 2.5, released in July, supports up to 30 images, 10 video clips, and 10 audio clips in a single generation. Meanwhile, Alibaba Cloud’s Wan3.0 accepts images, video, audio, documents, and public web pages, and Google’s Veo 3.1 frames reference images as “ingredients” for building scenes and characters.
These specifications highlight a shift in workflow: creators must now decide which details belong in the prompt and which should be locked in via reference files.
Reference Discipline Matters
Industry experts caution that more inputs don’t always mean better results. Contradictions between references can confuse models, making it crucial to prioritize what must remain consistent. For character-driven clips, a clean portrait is often the most important reference. For product videos, shape, materials, and packaging should be preserved. Scene references help define layout, lighting, and palette, but they don’t replace clear instructions about action or camera movement.
Motion references, such as video clips, are particularly valuable for conveying timing, choreography, or camera behavior. Audio references, meanwhile, are best used when timing depends on sound cues, such as dance beats or dialogue synchronization.
Documents and Web Pages Enter the Mix
Wan3.0’s ability to accept documents and web pages opens new possibilities for explainer videos. However, this also introduces risks: inaccurate source material can be transformed just as efficiently as accurate content. Creators are advised to reduce documents to essential facts and terminology before uploading.
Building Shots in Passes
The emerging best practice is to build video generations in stages rather than uploading everything at once. A first pass might include only duration, aspect ratio, a subject image, and a simple action description. Scene, motion, and audio references can then be added one at a time, making it easier to identify which inputs improve the shot and which introduce errors.
This reference-first discipline applies whether creators work directly inside a model’s native interface or through broader creation platforms such as Whisk AI, which are designed to streamline multi-asset workflows. By keeping the first prompt and core image unchanged, then layering references step by step, creators gain clearer visibility into what fixes a shot and what causes conflicts.
Art Direction Evolves
Ultimately, reference-rich generation shifts the role of art direction. Instead of describing every detail in words, creators must decide which source controls each part of the shot. A shorter prompt paired with carefully chosen references often produces more reliable results, and makes troubleshooting far easier.
Pokdepinion: Is it just me or is AI moving way too fast now? I mean, I know technology evolves, and it already feels pretty fast with smartphones, but this one is just ridiculously fast.
