The challenge of generating human-like co-speech gestures represents a fascinating intersection of technology and creativity, offering ample opportunities for student research and innovation. Traditional methods in this domain have often been constrained by rigid style control mechanisms, utilizing predefined text labels or specific motion examples. Such limitations underscore the necessity for more adaptable solutions that can accurately capture and express a broader range of user intents.
This student project aims to explore advanced techniques for synthesizing realistic and stylized co-speech gestures, emphasizing the flexibility of style control beyond conventional boundaries. Leveraging the potential of large-scale contrastive language-image pre-training models, students are encouraged to develop methodologies that effectively extract and apply style representations from varied input sources, including text, motion clips, and videos. This initiative seeks to address and bridge the gap between user expectations and technological capabilities in gesture generation.
Key Areas of Focus
● Efficient Style Representation: Investigate methods for extracting detailed style representations from diverse inputs to capture nuanced user intents.
● Gesture Generation Techniques: Explore and develop models capable of producing high-quality, realistic gestures that align closely with intended styles.
● Dynamic Style Application: Implement adaptive techniques for infusing extracted style elements into the gesture generation process, ensuring accuracy and adaptability.
● Semantic Alignment: Devise strategies for aligning generated gestures with speech content semantically, using contrastive learning to ensure relevance and naturalness.
● Granular Style Control: Enable precise control over the styling of individual body parts, offering users the ability to fine-tune gestures to their specific preferences.
Undergraduate
None
Robotics (405.652, Lab)