Research Intern, Inference (Fall 2026)

About the role#

The Inference Research team builds efficient, scalable, and reliable serving systems for large foundation models. This role focuses on the intersection of model architectures, systems engineering, and hardware optimization to lower the cost and latency of AI systems. Interns will work on distributed inference, compiler-aware optimization, and inference-time computation strategies like speculative decoding. Projects involve co-designing software and algorithms to support faster serving and larger model deployments.

What you'll do#

• Design and conduct rigorous experiments to validate hypotheses.
• Communicate plans, progress, and project results to the broader team.
• Document findings through scientific publications and blog posts.
• Co-design and implement cross-layer optimizations across models, systems, and hardware.

What you'll need#

• Currently pursuing a final year of a Bachelor's, Master's, or Ph.D. degree in Computer Science, Electrical Engineering, or a related field.
• Strong knowledge of Machine Learning and Deep Learning fundamentals.
• Experience with deep learning frameworks such as PyTorch or JAX.
• Strong programming skills in Python.
• Familiarity with Transformer architectures and recent developments in foundation models.
• Preferred: Prior research experience in foundation models, efficient ML, or ML systems; publications at conferences like MLSys or ICLR; experience with CUDA programming; and contributions to open-source machine learning projects.

Location & details#

• Location: San Francisco, California
• Term: Fall 2026 (September 14th to December 18th)
• Modality: On-site
• Compensation: $58–$63 per hour, plus housing stipends and benefits.