Learning to Interrupt in Language-based Multi-agent Communication
Conference on Language Modeling (COLM) (2026).
Ruta Desai
Principal Researcher, Microsoft Research
- ruta.p.desai [at] gmail.com
- CV
- Google Scholar
- GitHub
I am a Principal Researcher at Microsoft Research, where I lead research on agents that continually improve — for computer use and human-AI collaborative tasks.
Previously, I was a Staff Research Scientist in Fundamental AI Research (FAIR) at Meta, working on post-training agentic systems for human-AI collaboration and on self-improving social agents with collaborative reasoning capabilities. Before that, I was a Tech Lead Manager at Meta Reality Labs Research, where I led an embodied AI team solving perception and planning problems for Augmented Reality systems that extend their users' capabilities.
I obtained my PhD at the Robotics Institute, Carnegie Mellon University in 2018, working with Stelian Coros and Jim McCann. For my thesis, I devised human-AI systems that enabled casual users to build and program robots — towards increasing the accessibility of robotics. In the past, I have also had not-so-brief stints in human motor control and biomechanical simulation with Hartmut Geyer and Jessica K. Hodgins.
Research Interests
- LLM alignment: post-training for collaborative reasoning, preference learning, and real-time adaptation of agentic systems to human values
- Human-centric AI evaluation: benchmarks and evaluation frameworks for human-AI interaction
- Multimodal reasoning for social and physical interaction: vision-language systems for activity forecasting, embodied planning, and user mental models
Selected Publications
A full list of publications (40+) is available on Google Scholar.
Digital and Physical Agents
Ask-to-Act: Grounding Multimodal LLMs to Embodied Agents that Ask for Help with Reinforcement Learning
arXiv preprint (2025).
Collaborative Reasoner: Self-improving Social Agents with Synthetic Conversations
Advances in Neural Information Processing Systems (NeurIPS), San Diego, USA (2025).
ADAPT: Actively Discovering and Adapting to Preferences for any Task
Conference on Language Modeling (COLM), Montreal, Canada (2025).
PARTNR: A Benchmark for Planning and Reasoning in Embodied Multi-agent Tasks
International Conference on Learning Representations (ICLR), Singapore (2025).
Vision and Language
Enhancing Visual Planning with Auxiliary Tasks and Multi-token Prediction
IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Arizona, USA (2026).
Human-AI Interaction
XAIUI: User Belief-Driven Explainable AI for Context-Aware Adaptive Interfaces
ACM Transactions on Interactive Intelligent Systems (TiiS), 16(2), Article 10 (2026).
Interactive Program Synthesis for Modeling Collaborative Physical Activities from Narrated Demonstrations
arXiv preprint (2025).
Geppetto: Enabling Semantic Design of Expressive Robot Behaviors
ACM Conference on Human Factors in Computing Systems (CHI), Glasgow, UK (2019).
Best Paper Award — top 1%
Computational Abstractions for Interactive Design of Robotic Devices
IEEE International Conference on Robotics and Automation (ICRA), Singapore (2017).
Selected Preprints and Patents
A full list of patents (10+) is available on Google Patents.
Generative Design Techniques for Robot Behavior
US Patent 11,772,275 / US20200030988 A1 (2020–2023).
Outreach
Teaching and Talks
- Invited Talk, GRASP Seminar, University of Pennsylvania (2026).
- Part-time Lecturer, Computer Vision (CSE 455), University of Washington (2025).
- Guest Lecture, Generating Expressiveness in Intelligent Agents and Avatars, University of Florida (2022).
- Invited Talk, DUB Seminar, University of Washington (2020).
- Invited Talk, BID Seminar, University of California, Berkeley (2019).
- Invited Talk, GRASP Seminar, University of Pennsylvania (2019).
Mentoring and Advising
- Technical and strategy advising to two early-stage startups (2026–present).
- Advising 3+ PhD students at Microsoft (2026–present).
- Supervised 8+ PhD student interns at Meta (2019–2025).
- Led research collaborations with university partners including Yale, UC Berkeley, Georgia Tech, Oxford, and UT Austin.
Chairing and Awards
- Area Chair, Conference on Robot Learning (CoRL) (2026).
- Silver Reviewer Award, International Conference on Machine Learning (ICML) (2026).
- Organizer, CVPR Workshop on Causal and Object-centric Representations for Robotics (2024).
Women in Technology
I am always looking to support and contribute towards encouraging women in science and technology. In the past:
- I have led roadshows and volunteered in teaching middle school girls at Technights at CMU. We have smashed computers, fought combats in code, and have even gazed at stars through apps.
- I helped organize OurCS 2015, a conference to encourage undergraduate women in research. I was also part of the Google Anita Borg Scholars Alumni Planning Committee.
Selected Press
- TechCrunch, Meta is studying how humans and robots can collaborate on housework, 2025.
- SiliconANGLE, Meta's Habitat 3.0 simulates real-world environments for intelligent AI robot training, 2023.
- TechCrunch, Embodied AI spins a pen and helps clean the living room in new research, 2023.
- TechCrunch, New toolkit makes it easy to drag and drop your own robot, 2017.
- Communications of the ACM, Robot Design For Dummies, 2017.
- EurekAlert, CMU's interactive tool helps novices and experts make custom robots, 2017.
- NSF, Graduate Student Earns Prestigious Scholarships for Women — Ruta Desai, 2012.
- CMU SCS, Five SCS Students Named Siebel Scholars, 2012.