About the role#
The Applied Machine Learning Ark team builds and operates Large Language Model service platforms. We develop end-to-end solutions that include model training, fine-tuning, inference frameworks, and intelligent agent systems. This internship focuses on the design and evolution of agent evaluation systems. You will work across research, platform, and product teams to translate data into system improvements.
What you'll need#
- Currently pursuing a Bachelor's degree in Computer Science, Artificial Intelligence, Machine Learning, Data Science, or a related field.
- Solid foundation in machine learning and deep learning.
- Hands-on experience with LLM-based systems like agents, tool calling, retrieval, or multi-agent systems.
- Strong Python skills and experience with a mainstream ML or agent evaluation framework.
- Demonstrated research or engineering ability through publications, projects, internships, or open-source work.
- Familiarity with LLM post-training, reasoning and planning methods, or continual learning is helpful.
- Experience with feedback-driven optimization loops or large-scale log and trace analysis is a plus.
- Publications at top-tier ML or NLP venues, particularly in agent learning or evaluation, are preferred.
What you'll do#
- Design evaluation systems for LLM-based agents that measure task success, tool use, reasoning quality, and reliability.
- Build benchmarks and automated judging pipelines using a mix of rule-based checks, model-based judging, and human review.
- Analyze agent execution traces and user feedback to find failure patterns and drive system improvements.
- Support the closed loop from experience to capability to bring methods into production.
Location & details#
- Location: San Jose, California.
- Term: Summer 2027.
- Work Modality: On-site.
- Employment: Full-time intern.
- Compensation: $45 per hour.


