Rui Ye / 叶锐

I am a third-year PhD candidate at Shanghai Jiao Tong University (SJTU) in Shanghai, China. Before that, I received my Bachelor degree from SJTU, ranked the first out of 150.

I am currently advised by Prof. Siheng Chen, in the MediaBrain Lab. My research interests are in Trustworthy AI and Collaborative AI. Specifically, I am interested in trustworthy large language models (LLMs), multi-agent system, and federated learning. I have interned at Microsoft Research Asia (MSRA) and Shanghai AI Laboratory.

I am actively seeking collaborations and opportunities as a research intern or visiting student (could be one-year long), please feel free to contact me!!!

Email  /  Google Scholar  /  Github  /  LinkedIn  /  Twitter

^* denotes equal contribution, ^† denotes corresponding author, see full list in Google Scholar, some are highlighted.

This paper proposes the first realistic benchmark for federated learning of large language models, termed FedLLM-Bench. It encompasses 3 datasets for instruction tuning task and 1 dataset for preference alignment task, which exhibit diversities in language, quality, quantity, instruction, length, embedding, and preference.

This paper for the first time reveals the vulnerability of safety alignment during federated instruction tuning by proposing a simple safety attack method. While many existing FL defense methods fail to defend against such attack, we propose a post-hoc defense method that automatically and effectively enhances the safety alignment of LLMs.

This paper proposes OpenFedLLM for training large language models on decentralized private data via federated learning, which covers instruction tuning, value alignment, 7 FL algorithms, 8 training datasets, and 30+ evaluation metrics. Based on OpenFedLLM, we conduct a comprehensive empirical study, provide insights, and point out future directions.

This paper proposes to self-align large language models via social scene simulation, which is powered by our proposed simulator called MATRIX. Human evaluations show that our aligned 13/30B LLMs can outperform GPT-4 on value alignment.

This paper proposes inclusive and incentivized personalized federated learning (iPFL), which incentivizes data holders with diverse purposes to collaboratively train personalized models without revealing raw data.

This paper unreveals the vulnerability of value alignment in aligned open-source LLMs by proposing a series of efficient attack methods (i.e., reverse alignment). Experiments show that simple fine-tuning can significantly compromise the alignment of the LLMs.

This paper for the first time explores how advanced generative models can benefit FL on heterogeneous private data. We show that generative content can not only mitigate data heterogeneity, but also enhance privacy preservation for FL.

This paper proposes to more fundamentally handle data heterogeneity from the perspective of data, which is achieved by extracting consensus data from the global model to complement clients' heterogeneous data.

Based on our empirical and theoretical observations, we propose to aggregate models based on both dataset size and a defined discrepancy value.

We propose a pFedGraph algorithm to promote more collaboration between clients with more similar data distributions.

We propose to align category-wise feature spaces of clients in FL, which achieves pleasant performance with theoretical convergence guarantee.

• National Scholarship for PhD Students, 2024
• National Scholarship for Undergraduates, 2020 (2 out of 150)
• Shanghai Outstanding Graduates, 2022
• Samsung Scholarship, 2023 (only one awardee)
• Mathematical Contest in Modeling, Finalist, 2021 (<1%)
• Shanghai Jiao Tong University Wenjun Wu AI Class, 2022 (16 are selected)
• Shanghai Jiao Tong University Xu Zhang Academician Scholarship, 2022 (3 out of 150)
• Shanghai Jiao Tong University Ceyear Scholarship, 2021
• Shanghai Jiao Tong University Fujian Alumni Association Scholarship, 2019 (the youngest awardee)
• Shanghai Jiao Tong University Class B Scholarship, 2019&2020&2021

Derived from Jon Barron's website.