Qiaoyuan Zheng

I am a Master’s student in Data Science at ETH Zürich and a Research Assistant at the ETH AI Center. My research focuses on LLM evaluation, interpretable capability assessment, and privacy in language agents.

Previously, I studied Computer Science and Mathematics at Sorbonne University. I also worked on time-series forecasting at ESIEE Paris, supervised by Dr. You Jiang.

Qiaoyuan Zheng in front of snowy mountains

Publications

I work on evaluating language models beyond aggregate benchmark scores, including privacy–utility trade-offs, ranking robustness, and interpretable skill profiles.

POLAR-Bench: A Diagnostic Benchmark for Privacy-Utility Trade-offs in LLM Agents

Qiaoyuan Zheng, Yiqu Yang, Qi Gao, Imanol Schlag

NeurIPS 2026 E&D Track.

A diagnostic benchmark with 7,852 samples across 10 domains for evaluating the trade-off between task utility and user-defined privacy constraints in LLM agents.

Are Near-Tied LLM Rankings Robust to Family-DIF-Guided Benchmark Recomposition?

Qiaoyuan Zheng, Yiqu Yang

NeurIPS 2026 TAE Workshop.

We study the robustness of near-tied LLM rankings across five benchmarks using item response theory and differential item functioning, with owner-disjoint evaluation folds and matched-random controls.

What Makes Something Hard(er)? Explaining Question Difficulty in Natural Language

Peng Cui, Qiaoyuan Zheng (co-first author), Rudolf Debelak, Mrinmaya Sachan

Preprint. Under review.

Combining item response theory with LLM-based contrastive hypothesis generation to explain question difficulty, with validation through held-out difficulty prediction and targeted question rewriting across reasoning benchmarks.

SkillEval: Learning Interpretable Ability Profiles of LLMs via Cognitive Diagnosis Models

Berke Arda, Peng Cui, Qiaoyuan Zheng, Rudolf Debelak, Mubashara Akhtar, Mrinmaya Sachan

Under review.

Combining LLM-based skill discovery with neural cognitive diagnosis to construct interpretable ability profiles for 3,811 LLMs across 9,523 items from five benchmarks.

TUPLE-Bench: Evaluating and Mitigating Privacy-Utility Trade-offs in Typed Decision Models

Yiqu Yang (co-first author), Qiaoyuan Zheng (co-first author), Yang Tian, Yu Fan, Elliott Ash

Under review.

A benchmark of 720 synthetic counterfactual pairs across three domains evaluating privacy leakage and downstream utility across five output-release settings. We develop workflow-constrained privacy quantization (WCPQ) to reduce secret recovery while preserving action branches and costs across 18 frozen workflow/cost combinations.

Last Translation Benchmark

Vilém Zouhar, Niyati Bafna, Mukund Choudhary, …, Qiaoyuan Zheng, …

Preprint.

I contributed translation test cases and reviewed English, Chinese, and French benchmark submissions.

Research Experience

ETH AI Center

2026–present

Research Assistant · Zurich, Switzerland

Developing a multilingual benchmark for legal compliance in LLMs, including reproducible scenario-generation, review, and evaluation pipelines. Collaborating with legal experts and investigating applications for LLM post-training.

ESIEE Paris

2023

Research Assistant · Paris, France

Implemented Transformer-based models and PatchTST for time-series forecasting under Dr. You Jiang, adapting training pipelines and comparing model performance and computational efficiency on synthetic datasets. [code]

Education

ETH Zürich

2025–present

MSc in Data Science

Sorbonne University

2022–2025

BSc in Computer Science / Mathematics

Selected Projects

Aperiodic Tiling Generation

2024

Sorbonne University

Implemented Penrose, Wang, and Hat tiling algorithms in Python; optimized generation with duplicate detection and spatial pruning.

Q-Learning and SARSA in Grid Worlds

2023

ESIEE Paris · GitHub

Compared Q-learning and SARSA across 15 custom environments. Built a map editor and environments adopted for master’s-level teaching at UPEC.

A little more about me

I speak Chinese, English, and French. Outside research, I enjoy playing drums and saxophone, cooking, reading, and calligraphy.

You can reach me at zqiaoyuan@ethz.ch.