👋

I am Rui Li, a 3th year Master’s student at the MOE Key Lab of Computational Linguistics, Peking University, advised by Prof. Zhifang Sui. Prior to that, I received my B.S. in Computer Science and Technology, with a major in Artificial Intelligence, from Shandong University in 2023. I have gained practical experience as an intern at the StepFun AI. I have also worked as an intern at the PLUSLAB Group advised by Prof. Nanyun (Violet) Peng.

Please refer to my CV and Google Scholar for more details.

Research

I am broadly interested in Natural Language Processing. My research focuses on:

1. LLM alignment for harmless, honest, and helpful
2. AI-assisted research

🌈 I have a deep and enthusiastic interest in interdisciplinary studies. My educational background has provided me with a foundational understanding and initial professional competence in fields such as law and art.
🌈 I particularly enjoy the exchange and integration of knowledge across different disciplines and am keen to collaborate with individuals from diverse academic backgrounds in the future. For potential interdisciplinary collaborations, I am very willing to offer technical support and am open to discussing any topics related to my research or broader interests.

News

[2025.09] Got one papers accepted by NeurIPS 2025! 🎉
[2025.08] Got two papers accepted by EMNLP 2025! (1 Main, 1 Findings) 🎉
[2025.06] Got two first-authored papers accepted by ACL 2025! (1 Main, 1 Findings) 🎉

[2024.11] Contributed as a core contributor to a large-scale open-source data project SuperGPQA, which was organized by M-A-P (Multimodal Art Projection, a non-profit open-source AI research community).
[2024.10] My first first-authored paper was accepted by EMNLP 2024 findings, and two other papers I contributed to were also accepted by EMNLP main and EMNLP findings, respectively. 🎉

[2023.09] Started my Master study at the MOE Key Laboratory of Computational Linguistics, Peking University, advised by Prof. Zhifang Sui. 😊
[2023.05] Guided by Professor Qiong Zeng, I completed my graduation thesis which earned me the Outstanding Undergraduate Graduation Thesis Award and Outstanding Thesis Defense Award from Shandong University. This marked the beginning of my academic journey. 🎨

Publications

How Far are LLMs from Being Our Digital Twins? A Benchmark for Persona-Based Behavior Chain Simulation

Rui Li, Heming Xia, Xinfeng Yuan, Qingxiu Dong, Lei Sha, Wenjie Li, Zhifang Sui

ACL 2025 (findings).

(🤫 The total data volume of 1001 for this benchmark was inspired by One Thousand and One Nights, a beloved childhood fairy tale.)

Towards Harmonized Uncertainty Estimation for Large Language Models

Rui Li, Jing Long, Muge Qi, Heming Xia, Lei Sha, Peiyi Wang, Zhifang Sui

ACL 2025. (Oral Presentation)

Be a Multitude to Itself: A Prompt Evolution Framework for Red Teaming

Rui Li, Peiyi Wang, Jingyuan Ma, Di Zhang, Lei Sha, Zhifang Sui

EMNLP 2024 (findings).

SuperGPQA: Scaling LLM Evaluation across 285 Graduate Disciplines

M-A-P (Multimodal Art Projection), Core Contributor

NeurIPS 2025.

Layer-Aware Representation Filtering: Purifying Finetuning Data to Preserve LLM Safety Alignment

Hao Li, Lijun Li, Zhenghao Lu, Xianyi Wei, Rui Li, Jing Shao, Lei Sha

EMNLP 2025.

Beyond Single Frames: Can LMMs Comprehend Implicit Narratives in Comic Strip?

Xiaochen Wang, Heming Xia, Jialin Song, Longyu Guan, Qingxiu Dong, Rui Li, Yixin Yang, Yifan Pu, Weiyao Luo, Yiru Wang, Xiangdi Meng, Wenjie Li, Zhifang Sui

EMNLP 2025 (findings).

A Survey on In-context Learning

Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Baobao Chang, Xu Sun, Lei Li, Zhifang Sui

EMNLP 2024.

ShieldLM: Empowering LLMs as Aligned, Customizable and Explainable Safety Detectors

Zhexin Zhang, Yida Lu, Jingyuan Ma, Di Zhang, Rui Li, Pei Ke, Hao Sun, Lei Sha, Zhifang Sui, Hongning Wang, Minlie Huang
EMNLP 2024 (findings).

Large Language Models Struggle with Unreasonability in Math Problems

Jingyuan Ma, Damai Dai, Zihang Yuan, Rui Li, Weilin Luo, Bin Wang, Qun Liu, Lei Sha, Zhifang Sui
AAAI 2026.

SCoRE: Benchmarking Long-Chain Reasoning in Commonsense Scenarios

Weidong Zhan, Yue Wang, Nan Hu, Liming Xiao, Jingyuan Ma, Yuhang Qin, Zheng Li, Yixin Yang, Sirui Deng, Jinkun Ding, Wenhan Ma, Rui Li, Weilin Luo, Qun Liu, Zhifang Sui
AAAI 2026 Bridge LMReasoning

Under Review & Preprint

A complete list of publications is available in my CV.

LLM-REVal: Can We Trust LLM Reviewers Yet?
Rui Li, Jia-Chen Gu, Po-Nien Kung, Heming Xia, Junfeng Liu, Xiangwen Kong, Zhifang Sui, Nanyun Peng

Merlin’s Whisper: Enabling Efficient Reasoning in LLMs via Black-box Adversarial Prompting
Heming Xia, Cunxiao Du, Rui Li, Chak Tou Leong, Yongqi Li, Wenjie Li

OS-Catalyst: Advancing Computer-Using Agents Efficiency through Adaptive Action

Xinfeng Yuan, Qiushi Sun, Yinghao Chen, Rui Li, Xuetian Chen, Siyu Yuan, Xintao Wang, Zichen Ding, Zonglin Li, Biqing Qi, Deqing Yang

SenseJudge: Human-Centric Preference-Driven Judgment Framework

Rui Li, Junfeng Liu, Xiangwen Kong, Zhifang Sui

HauntAttack: When Attack Follows Reasoning as a Shadow

Jingyuan Ma*, Rui Li*, Zheng Li, Junfeng Liu, Heming Xia, Lei Sha, Zhifang Sui

Step-3 is Large yet Affordable: Model-system Co-design for Cost-effective Decoding
StepFun AI, technical report, Contributor

MACG: A Multi‑Agent Framework for Thematically Structuring and Generation of Related Work
Zhuang Liu, Jian Liu, Chenbin Zhang, Rui Li, Chun Kang, Maolin Wang, Lei Sha

Plug-and-Play Training Framework for Preference Optimization

Jingyuan Ma, Rui Li, Zheng Li, Lei Sha, Zhifang Sui

Text-driven Palette Generation Method

Rui Li, Qiong Zeng

Technical Skills

Languages: C/C++, Python, Java, Shell, MATLAB, HTML/CSS

Developer Tools: VS Code, PyCharm, Git, Linux, Vim

Minor: Law

Internship Experience

Oct. 2025 - Present, ByteDance, Algorithm Intern

Jul. 2024 - Oct. 2025, Stepfun, Algorithm Intern

Jun. 2023 - Sep. 2023, 36Kr, Intern Reporter

Feb. 2023 - Jun. 2023, Shandong University, Introduction to Artificial Intelligence, Teaching Assistant