About me

I am a first-year PhD student at Peking University advised by incredible Prof.Di He. I’m interested in deep learning and its application to NLP and AI4science. I’ve interned at MSRA with Lijun Wu and DP Technology with Guolin Ke.

Publication

Two Stones Hit One Bird: Bilevel Positional Encoding for Better Length Extrapolation, ICML 2024 [paper] [code]
Zhenyu He*, Guhao Feng*, Shengjie Luo*, Kai Yang, Liwei Wang, Jingjing Xu, Zhi Zhang, Hongxia Yang, Di He

Do Efficient Transformers Really Save Computation?, ICML 2024 [paper]
Kai Yang, Jan Ackermann, Zhenyu He, Guhao Feng, Bohang Zhang, Yunzhen Feng, Qiwei Ye, Di He, Liwei Wang

REST: Retrieval-Based Speculative Decoding, NAACL 2024 [paper] [code] [blog] [paperweekly]
Zhenyu He*, Zexuan Zhong*, Tianle Cai*, Jason D. Lee, Di He

Uni-Fold MuSSe: De Novo Protein Complex Prediction with Protein Language Models, ICLR 2023 MLDD workshop, [paper] [code]
Jinhua Zhu*, Zhenyu He*, Ziyao Li, Guolin Ke, Linfeng Zhang

Rumor Detection on Social Media with Event Augmentations, SIGIR 2021(short) [paper] [code]
Zhenyu He*, Ce Li*, Fan Zhou, Yi Yang

Preprint

Examining User-Friendly and Open-Sourced Large GPT Models: A Survey on Language, Multimodal, and Scientific GPT Models, arXiv 2023 [paper]
Kaiyuan Gao*, Sunan He*, Zhenyu He*, Jiacheng Lin*, Qizhi Pei*, Jie Shao*, Wei Zhang*

Two-Stage Neural Contextual Bandits for Personalised News Recommendation, arXiv 2022 [paper]
Mengyan Zhang, Thanh Nguyen-Tang, Fangzhao Wu, Zhenyu He, Xing Xie, Cheng Soon Ong

Education Experience

Peking University

Sep. 2023 - Now

PhD candidate, School of Artificial Intelligence

University of Electronic Science and Technology of China

Sep. 2019 - Jun. 2023

B.E. in Software Engineering

GPA: 91+/100