About the role#
This internship focuses on generative models that produce realistic audio-visual data for multi-person conversations. You will help build the audio-visual perception systems that power future smart glasses. The project involves generating video and audio from inputs like text or reference images. These scenes must demonstrate natural social interaction, including turn-taking, gaze, and listener reactions. You will also ensure the scenes are controllable, allowing for precise 3D positioning and camera movement to create high-quality training data for downstream perception models.
What you'll do#
- Design and train video diffusion or flow-matching models to generate synchronized audio and video for multi-person conversations.
- Develop conditioning methods based on 3D layout, group structure, and egocentric camera trajectory.
- Maintain identity consistency across different frames and viewpoints using reference images.
- Model conversational behaviors such as speech-driven gestures, gaze, and turn-taking.
- Create evaluation metrics for realism, audio-visual sync, and the accuracy of 3D positions.
- Train downstream egocentric perception models using generated data to measure performance improvements.
- Collaborate with Meta researchers to publish findings at top-tier venues.
What you'll need#
- Current enrollment in a PhD program for Computer Science, Electrical Engineering, Machine Learning, or a related field.
- Research experience in generative modeling for video or images, with first-author publications at venues like CVPR, ICCV, ECCV, NeurIPS, ICLR, or SIGGRAPH.
- Hands-on experience training or fine-tuning video diffusion or flow-matching models.
- Proficiency with conditioning methods such as adapters, ControlNet, or LoRA.
- Strong foundation in 3D vision, including camera models and multi-view geometry.
- At least 3 years of experience with Python, PyTorch, and large-scale distributed GPU training.
- Ability to obtain and maintain work authorization in the United States.
- Intent to return to your degree program after the internship.
- Preferred skills include background in spatial audio, social interaction modeling, egocentric perception, or experience with 3D and 4D human representations.
Location & details#
- Location: Redmond, Washington.
- Modality: On-site.
- Role type: Full-time intern.
- Term: Rolling.
About Meta
Meta develops software and social media applications. It operates platforms such as Facebook, Messenger, Instagram, and WhatsApp. The company maintains headquarters in Menlo Park, California. Current work focuses on augmented and virtual reality alongside artificial intelligence and machine learning.
How to get in at Meta
Landing an internship at Meta requires a proactive approach to the application process. Intern Insider sends an instant alert the moment a role matching your target is published, allowing you to apply among the first. Early applicants often have their materials reviewed before the applicant pool grows significantly. You can use Intern Insider to surface the recruiters behind specific roles at Meta. Reaching out directly to these individuals to ask about the position or a referral often improves your response rates compared to applying into a queue.
