DomainShuttle: Freeform Open Domain Subject-driven Text-to-video Generation
arxiv:2606.26058
Copy markdown
DomainShuttle: Freeform Open Domain Subject-driven Text-to-video Generation
Published on Jun 24
· Submitted by 
Nan Chen on Jun 25
- +47
- +43
Authors:
Nan Chen ,
Yiyang Cai ,
Rongchang Xie ,
Junwen Pan ,
Cheng Chen ,
Weinan Jia ,
Zhuowei Chen ,
Wen Zhou ,
Zhenbang Sun ,
Wenhan Luo
Abstract
DomainShuttle enables open domain subject-driven text-to-video generation with high fidelity and flexibility across in-domain and cross-domain scenarios through domain-aware modeling and dual RoPE schemes.
Generated by Qwen/Qwen2.5-Coder-32B-Instruct
Open domain subject-driven text-to-video (S2V) generation has drawn significant interest in academia and industry. Open domain S2V mainly involves two scenarios: in-domain, which requires retaining the reference subject features as much as possible, and cross-domain, which preserves the intrinsic features of the subject while allowing subject-irrelevant properties to vary flexibly according to the text prompt. Existing methods primarily focus on maximizing subject fidelity in in-domain scenarios, which limits their editability and adaptability in cross-domain scenarios, such as novel styles, semantic combinations, or domain attributes. In this study, we propose that an ideal S2V method should flexibly shuttle between different domains, achieving strong performance in both in-domain and cross-domain scenarios. To this end, we propose DomainShuttle, which could achieve high fidelity and generative flexibility for open domain video personalization. Specifically, we introduce Domain-MoT, which decouples videos and reference features and introduces the domain-aware AdaLN for domain-specific modeling of reference images. We then introduce the Video-Reference DualRoPE scheme, which places reference image tokens and video tokens in separate RoPE spaces to enable precise subject-level spatial modeling, and Cross-Pair Consistent Loss, which aims to extract intrinsic subject features unaffected by irrelevant features. Extensive experiments demonstrate that DomainShuttle achieves significant performance improvements over existing methods, exhibiting high subject fidelity and generative flexibility across diverse open domain application scenarios.
View arXiv page View PDF Project page GitHub 66 Add to collection
Community
Paper author Paper submitter about 18 hours ago
We propose DomainShuttle, an open-domain subject-driven text-to-video method that flexibly handles both in-domain fidelity and cross-domain editability by decoupling reference and video features, modeling domain attributes, and learning intrinsic subject representations.
See translation
🔥
2
2
😎
1
1
🚀
1
1
🤗
1
1
+
Reply
EditPreview
Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.
Tap or paste here to upload images
Comment
· Sign up or log in to comment
Get this paper in your agent:
`hf papers read 2606.26058`
Don't have the latest CLI?
`curl -LsSf https://hf.co/cli/install.sh | bash`
Models citing this paper 0
No model linking this paper
Cite arxiv.org/abs/2606.26058 in a model README.md to link it from this page.
Datasets citing this paper 0
No dataset linking this paper
Cite arxiv.org/abs/2606.26058 in a dataset README.md to link it from this page.
Spaces citing this paper 0
No Space linking this paper
Cite arxiv.org/abs/2606.26058 in a Space README.md to link it from this page.
Collections including this paper 0
No Collection including this paper
Add this paper to a collection to link it from this page.