Charlie Ruan

CS PhD Student

UC Berkeley Sky Computing Lab

Biography

I am a first-year CS PhD student at UC Berkeley Sky Computing Lab, working on AI systems. I also have the fortune to work with Prof. Tianqi Chen as part of the Catalyst Group.

I obtained my M.S. degree in Computer Science at Carnegie Mellon University, advised by Prof. Tianqi Chen and collaborated with Prof. Zhihao Jia.

I focus on systems problem in emerging machine learning workloads, with various open-source development experience. I am the lead of WebLLM and a core contributor to MLC-LLM.

I obtained my B.S. degree in Computer Science and Operations Research from Cornell University, where I was fortunate to work with Prof. Christopher De Sa on distributed training and with Prof. Jim Dai on reinforcement learning and stochastic processes.

Interests

Machine Learning Systems
Distributed Systems

Education

PhD in Computer Science
UC Berkeley, 2025 - Present
MS in Computer Science
Carnegie Mellon University, 2023 - 2025
BS in Computer Science & Operations Research
Cornell University, 2019 - 2023

Publications

A System for Microserving of LLMs

Hongyi Jin, Ruihang Lai, Charlie F. Ruan, Yingcheng Wang, Todd Mowry, Xupeng Miao, Zhihao Jia, Tianqi Chen

WebLLM: A High-Performance In-Browser LLM Inference Engine

Charlie F. Ruan, Yucheng Qin, Xun Zhou, Ruihang Lai, Hongyi Jin, Yixin Dong, Bohan Hou, Meng-Shiun Yu, Yiyan Zhai, Sudeep Agarwal, Hangrui Cao, Siyuan Feng, Tianqi Chen

XGrammar: Flexible and Efficient Structured Generation Engine for Large Language Models

Yixin Dong, Charlie F. Ruan, Yaxing Cai, Ruihang Lai, Ziyi Xu, Yilong Zhao, Tianqi Chen

Local deployment of large-scale music AI models on commodity hardware

Xun Zhou, Charlie F. Ruan, Zihe Zhao, Tianqi Chen, Chris Donahue

Emerging Platforms Meet Emerging LLMs: A Year-Long Journey of Top-Down Development

Siyuan Feng, Jiawei Liu, Ruihang Lai, Charlie F. Ruan, Yong Yu, Lingming Zhang, Tianqi Chen

Coordinating Distributed Example Orders for Provably Accelerated Training

A. Feder Cooper, Wentao Guo, Khiem Pham, Tiancheng Yuan, Charlie F. Ruan, Yucheng Lu, Christopher De Sa

Projects

Core Contributor

MLC-LLM

June 2023 – Present Pittsburgh, PA

Enable universal native deployment for LLMs through machine learning compilation techniques. GitHub (19.3k stars)

Project Lead

WebLLM

June 2023 – Present Pittsburgh, PA

Leading the project to bring LLMs to run locally in client-side browser with WebGPU acceleration. GitHub (13.9k stars); talk at Google WebAI Summit ‘24

Research Experience

Graduate Student Researcher

Sky Computing Lab, UC Berkeley

June 2025 – Present Berkeley, CA

Research Assistant

Prof. Tianqi Chen & Prof. Zhihao Jia, Carnegie Mellon University

March 2024 – Present Pittsburgh, PA

Catalyst Group; LLM serving systems

Research Assistant

Prof. Christopher De Sa, Cornell University

September 2022 – June 2023 Ithaca, NY

Investigated finding provably better data permutations in distributed learning. CD-GraB was accepted by NeurIPS'23

Research Assistant

Prof. Jim Dai, Cornell University

November 2021 – September 2022 Pittsburgh, PA

Investigated using variance-reduction method approximating martingale-process in reinforcement learning with large state space

Industry Experience

Software Engineer Intern

Google

June 2023 – August 2023 Sunnyvale, CA

Worked on Core ML’s Distributed Runtime team, optimizing TensorFlow’s checkpoint to reduce wasted TPU cycles

Software Engineer Intern

Google Cloud

August 2022 – October 2022 Sunnyvale, CA

Worked on Technical Infrastructure’s Platform team, deploying accelerators in Google data centers using OpenBMC, implementing Linux daemon and firmware update APIs