Fengyu Gao

I'm a final-year PhD student in the Department of Computer Science at the University of Virginia, advised by Prof. Jing Yang. Previously, I received my Bachelor's degree in Computer Science and Technology from University of Science and Technology of China (USTC) in 2022.

My research focuses on building trustworthy AI systems, with an emphasis on enabling models to learn effectively from sensitive data and improving the reliability of increasingly capable AI agents. My work centers on two directions:

• Privacy-Preserving Machine Learning. I develop learning and data synthesis methods that extract useful information from sensitive and distributed data while providing rigorous privacy guarantees. My recent work focuses on differentially private synthesis for LLM preference alignment, in-context learning, and federated data, as well as the theoretical foundations of private federated online learning.
• LLM Agents. I study how LLM agents can improve their capabilities while remaining reliable, secure, and effectively evaluated. My recent work investigates co-evolving agent task solving and evaluation and learning from failure experience to improve agent security.

Email  /  Google Scholar  /  LinkedIn

Publications

(* indicates equal contribution)

DUET: Co-Evolving Solver and Grader Agents

Fengyu Gao, Sourav Pal, Austin Z. Henley, Arjun Radhakrishna, Gustavo Soares

TL;DR: DUET jointly optimizes a solver agent and a grader agent so that task solving and evaluation improve together.

Self-Reflection Fine-Tuning: Enhancing Agent Security against Prompt Injection Attacks from Failure Experience

Zixuan Wang*, Hao Li*, Fengyu Gao, G. Edward Suh, Yi Zeng, Yevgeniy Vorobeychik, Ning Zhang, Chaowei Xiao

TL;DR: We train LLM agents to learn from prompt-injection failures through self-reflection, improving robustness to unseen and adaptive attacks.

Differentially Private Preference Data Synthesis for Large Language Model Alignment

Fengyu Gao, Jing Yang

In International Conference on Machine Learning (ICML) 2026      [Code]

TL;DR: The first framework that generates differentially private synthetic preference data, enabling privacy-preserving preference alignment of large language models.

HeteroFedSyn: Differentially Private Tabular Data Synthesis for Heterogeneous Federated Settings

Xiaochen Li, Fengyu Gao, Xizixiang Wei, Tianhao Wang, Cong Shen, Jing Yang

In the ACM Special Interest Group on Management of Data (SIGMOD) 2026      [Code]

TL;DR: Differentially private tabular data synthesis for the horizontal federated setting, achieving utility comparable to centralized synthesis.

Data-Adaptive Differentially Private Prompt Synthesis for In-Context Learning

Fengyu Gao*, Ruida Zhou*, Tianhao Wang, Cong Shen, Jing Yang

In International Conference on Learning Representations (ICLR) 2025      [Code]

TL;DR: Differentially private synthetic few-shot example generation for in-context learning by leveraging data clustering patterns.

Federated Online Prediction from Experts with Differential Privacy: Separations and Regret Speed-ups

Fengyu Gao, Ruiquan Huang, Jing Yang

In Advances in Neural Information Processing Systems (NeurIPS) 2024

TL;DR: Differentially private federated online prediction from experts, achieving regret speed-up under stochastic and special oblivious adversaries, and establishing lower bounds.

Federated Q-Learning: Linear Regret Speedup with Low Communication Cost

Zhong Zheng, Fengyu Gao, Lingzhou Xue, Jing Yang

In International Conference on Learning Representations (ICLR) 2024

TL;DR: Model-free federated Q-learning for tabular MDPs, achieving linear regret speed-up with logarithmic communication cost.

Honors and Awards

Silver Reviewer, ICML, 2026

Top Reviewer, NeurIPS, 2025

Outstanding Student Scholarship, USTC, 2019 - 2021

Overseas Alumni Foundation Outstanding Student Scholarship, USTC, 2019

Outstanding Freshman Scholarship, USTC, 2018

Service

Conference Reviewer: ICLR'25, 26, 27; NeurIPS'25, 26; ICML'26; AAAI'27

Teaching Assistant: CS 6770 Natural Language Processing, Fall 2026, UVA; CS 4771 Reinforcement Learning, Spring 2026, UVA; CS4501 Law and AI, Fall 2025, UVA; Algebraic Structure, Spring 2021, USTC; Analog and Digital Circuits, Fall 2020, USTC

Hackathon Judge: Hack to the Beat, Women in Computing Sciences (WiCS), University of Virginia, 2026

Thank Dr. Jon Barron for sharing the source code of his homepage.