Jiahao Ying | Singapore Management University

About Me

I am third-year Ph.D. candidate at Singapore Management University. I am currently supervised by Yixin Cao and Qianru Sun. My research focuses on developing robust, fair, and effective methods for the automated evaluation and improvement of Large Language Models (LLMs).

Research Interests

LLMs Evaluation:
- Automated Evaluation Data Generation
- Reliable Evaluator Development
LLMs Improvement:
- Adaptive Learning Strategies

News

[Oct. 2024] One paper is accepted to NeurIPS 2024 Dataset & Benchmark Track.
[Oct. 2024] Two papers are accepted to EMNLP 2024 Findings.
[Jun. 2024] Two papers are accepted to ACL 2024 (One Main conference and one Findings).
[Oct. 2023] One paper is accepted to NeurIPS 2023 Dataset & Benchmark Track.

Publications

EMNLP

Automating Dataset Updates Towards Reliable and Timely Evaluation of Large Language Models

Jiahao Yin, Yixin Cao, Yushi Bai, Qianru Sun, Bo Wang, Wei Tang, Zhaojun Ding, Yizhe Yang, Xuanjing Huang, Shuicheng Yan

Advances in Neural Information Processing Systems 37 NeurIPS, 2024

PDF Code Project Page BibTex Poster

EMNLP

LLMs-as-Instructors: Learning from Errors Toward Automating Model Improvement

Jiahao Yin, Mingbao Lin, Yixin Cao, Wei Tang, Bo Wang, Qianru Sun, Xuanjing Huang, Shuicheng Yan

Empirical Methods in Natural Language Processing EMNLP, 2024.

PDF Code Project Page BibTex Findings

ACL

Intuitive or Dependent? Investigating LLMs’ Behavior Style to Conflicting Prompts

Jiahao Yin, Yixin Cao, Kai Xiong, Yidong He, Long Cui, Yongbin Liu

Association for Computational Linguistics ACL, 2024.

PDF Code Project Page BibTex Main

ACL

A + B: A General Generator-Reader Framework for Optimizing LLMs to Unleash Synergy Potential

Wei Tang, Yixin Cao, Jiahao Ying, Bo Wang, Yuyue Zhao, Yong Liao, Pengyuan Zhou

Association for Computational Linguistics ACL, 2024.

PDF Project Page BibTex Findings

NeurIPS

Benchmarking Foundation Models with Language-Model-as-an-Examiner

Yushi Bai*, Jiahao Ying*, Yixin Cao, Xin Lv, Yuze He, Xiaozhi Wang, Jifan Yu, Kaisheng Zeng, Yijia Xiao, Haozhe Lyu, Jiayin Zhang, Juanzi Li, Lei Hou

Advances in Neural Information Processing Systems 36 NeurIPS, 2023

PDF Code BibTex Poster

Services

Conference Reviewers

Association for Computational Linguistics (ACL) 2023, 2024
Empirical Methods in Natural Language Processing (EMNLP) 2023, 2024
International Conference on Learning Representations (ICLR) 2024
Conference on Language Modeling (COLM) 2024