Rui Ye

Rui Ye / 叶锐

I am a second-year PhD candidate at Shanghai Jiao Tong University (SJTU) in Shanghai, China. Before that, I received my Bachelor degree from SJTU, ranked the first out of 150.

I am currently advised by Prof. Siheng Chen, in the MediaBrain Lab. My research interests are in Responsible AI and Collaborative AI. Specifically, I am interested in tackling data heterogeneity federated learning (FL) and trustworthy large language models (LLMs).

I am actively seeking collaborations and opportunities as a research intern or visiting student (could be one-year long), please feel free to contact me!!!

Email / Google Scholar / Github / LinkedIn / Twitter

🔥 News

[2024.05] One co-authored paper (MATRIX) is accepted by ICML 2024! See you at Vienna!
[2024.02] We release a comprehensive [FL x LLMs] framework OpenFedLLM!!!
[2024.01] One paper (FedCOG) is accepted by ICLR 2024! See you at Vienna!
[2023.11] Checking out from MSRA in November and actively seeking new collaboration opportunities.
[2023.08] One paper (FedFM) is accepted by IEEE Transactions on Signal Processing (TSP)!
[2023.07] Start second internship at Microsoft Research Asia (MSRA), Beijing (on-site).
[2023.04] Two papers (FedDisco & pFedGraph) are accepted by ICML 2023!
[2022.11] Start internship at Microsoft Research Asia (MSRA), Beijing (remote).

📑 Publications

^* denotes equal contribution, ^† denotes corresponding author, see full list in Google Scholar.

2024

	Incentivizing Inclusive Data Contributions in Personalized Federated Learning Enpei Zhang^, Jingyi Chai^, Rui Ye^, Yanfeng Wang, Siheng Chen^† ICLR AGI Workshop and DPFM Workshop*, 2024 OpenReview / bibtex This paper proposes inclusive and incentivized personalized federated learning (iPFL), which incentivizes data holders with diverse purposes to collaboratively train personalized models without revealing raw data.
	OpenFedLLM: Training Large Language Models on Decentralized Private Data via Federated Learning Rui Ye, Wenhao Wang, Jingyi Chai, Dihan Li, Zexi Li, Yinda Xu, Yaxin Du, Yanfeng Wang, Siheng Chen^† ICLR AGI Workshop and DPFM Workshop, 2024 arXiv / bibtex / Code This paper proposes OpenFedLLM for training large language models on Decentralized private data via federated learning, which covers instruction tuning, value alignment, 7 FL algorithms, 8 training datasets, and 30+ evaluation metrics. Based on OpenFedLLM, we conduct a comprehensive empirical study, provide insights, and point out future directions.
	Self-Alignment of Large Language Models via Monopolylogue-based Social Scene Simulation Xianghe Pang^, Shuo Tang^, Rui Ye^, Yuxin Xiong, Bolun Zhang, Yanfeng Wang, Siheng Chen^† International Conference on Machine Learning (ICML), 2024 ICLR AGI Workshop (Oral)*, 2024 arXiv / bibtex / Project This paper proposes to self-align large language models via social scene simulation, which is powered by our proposed simulator called MATRIX. Human evaluations show that our aligned 13/30B LLMs can outperform GPT-4 on value alignment.
	Open-source can be dangerous: On the vulnerability of value alignment in open-source LLMs Jingwei Yi^, Rui Ye^, Qisi Chen, Bin Zhu, Siheng Chen, Defu Lian, Guangzhong Sun, Xing Xie, Fangzhao Wu preprint, 2024 link / bibtex This paper unreveals the vulnerability of value alignment in aligned open-source LLMs by proposing a series of efficient attack methods (i.e., reverse alignment). Experiments show that simple fine-tuning can significantly compromise the alignment of the LLMs.

2023

	Federated Learning Empowered by Generative Content Rui Ye, Xinyu Zhu, Jingyi Chai, Siheng Chen^†, Yanfeng Wang arXiv, 2023 arXiv / bibtex This paper for the first time explores how advanced generative models can benefit FL on heterogeneous private data. We show that generative content can not only mitigate data heterogeneity, but also enhance privacy preservation for FL.
	Fake It Till Make It: Federated Learning with Consensus-Oriented Generation Rui Ye, Yaxin Du, Zhenyang Ni, Siheng Chen^†, Yanfeng Wang International Conference on Learning Representations (ICLR), 2024 arXiv / bibtex / Code This paper proposes to more fundamentally handle data heterogeneity from the perspective of data, which is achieved by extracting consensus data from the global model to complement clients' heterogeneous data.
	FedDisco: Federated Learning with Discrepancy-aware Collaboration Rui Ye, Mingkai Xu, Jianyu Wang, Chenxin Xu, Siheng Chen^†, Yanfeng Wang International Conference on Machine Learning (ICML), 2023 arXiv / bibtex / PMLR / Code Based on our empirical and theoretical observations, we propose to aggregate models based on both dataset size and a defined discrepancy value.
	Personalized Federated Learning with Inferred Collaboration Graphs Rui Ye^, Zhenyang Ni^, Fangzhao Wu, Siheng Chen^†, Yanfeng Wang International Conference on Machine Learning (ICML), 2023 PMLR / bibtex / Code We propose a pFedGraph algorithm to promote more collaboration between clients with more similar data distributions.
	FedFM: Anchor-based Feature Matching for Data Heterogeneity in Federated Learning Rui Ye, Zhenyang Ni, Chenxin Xu, Jianyu Wang, Siheng Chen^†, Yanfeng Wang IEEE Transactions on Signal Processing (TSP), 2023 arXiv / IEEE / bibtex / Code (PyTorch, PaddlePaddle, MindSpore) We propose to align category-wise feature spaces of clients in FL, which achieves pleasant performance with theoretical convergence guarantee.

🎓 Educations

Degree: Bachelor
Period: 2018.09 - 2022.06
Major: Information Engineering (AI Class)
GPA: 3.94/4.3 (ranked 1st out of 150)

🥇 Honors & Awards

National Scholarship, 2020 (2 out of 150)
Shanghai Outstanding Graduates, 2022
Samsung Scholarship, 2023 (only one awardee)
Mathematical Contest in Modeling, Finalist, 2021 (<1%)
Shanghai Jiao Tong University Wenjun Wu AI Class, 2022 (16 are selected)
Shanghai Jiao Tong University Xu Zhang Academician Scholarship, 2022 (3 out of 150)
Shanghai Jiao Tong University Ceyear Scholarship, 2021
Shanghai Jiao Tong University Fujian Alumni Association Scholarship, 2019 (the youngest awardee)
Shanghai Jiao Tong University Class B Scholarship, 2019&2020&2021

👀 Misc

I love playing basketball / listening rap music / casual walk / travelling.

Derived from Jon Barron's website.