About Me
Hello! I am a Ph.D. candidate in Computer Science at Texas A&M University (TAMU), advised by Prof. James Caverlee and Prof. Ali Mostafavi. I am currently an Applied Scientist Intern at AWS AI Lab, and I was previously a PhD AI/ML Engineer Intern at LinkedIn.
My research lies at the intersection of multi-agent systems and information retrieval.
I am open to research collaboration. Feel free to reach me at kai_yin@tamu.edu.
๐ฅ News
-
2026.09
๐ผ Joined
AWS AI Lab (New York City) as an Applied Scientist Intern, working on agentic RL for enterprise deep search.
- 2026.09 ๐ Passed the Ph.D. preliminary exam and became a Ph.D. candidate in Computer Science.
-
2026.08
๐๐ DisasterBench is accepted by EMNLP 2026 Findings!
-
2026.08
๐๐ DisasterLex is accepted by EMNLP 2026 Findings!
-
2026.05
๐ผ Joined
LinkedIn (Mountain View) as a PhD AI/ML Engineer Intern, working on post-training small language models for job search.
-
2026.04
๐๐ DMRetriever is accepted by ACL 2026 Main!
-
2026.04
๐๐ DisastQA is accepted by ACL 2026 Findings!
-
2025.11
๐ Released source code, training data, and model checkpoints for DMRetriever (33Mโ7.6B).
-
2025.08
๐๐ DisastIR is accepted by EMNLP 2025 Findings!
-
2025.08
๐ Released source code and dataset for DisastIR.
๐ผ Internship Experience

- Agentic RL for enterprise deep search (work in progress).

FacetLLM-1.5B: post-training a small language model for dynamic facet suggestion in job search.
- Reasoning-enhanced training data generation that follows strict product policy and works without PM-annotated data.
- Multi-task hybrid supervised fine-tuning with a task-weighted loss, so the model internalizes the reasoning chain and keeps performance close to full reasoning at inference.
- Difficulty-aware DPO and multi-reward GRPO with Reinforcement Learning from Verifiable Rewards (RLVR) to further improve model performance.
- Human annotation-driven LLM-as-Judge prompt development that distills expert knowledge into judge prompts for offline evaluation, avoiding expensive human annotation.
๐ Selected Publications and Projects
DMRetriever: Difficulty-aware Progressive Fine-tuning LLM for Improved Textual Retrieval in Data-Scarce Domain
ACL 2026 Main
- Developed DMRetriever, a family of six dense retrieval models (33Mโ7.6B) from encoder (BERT)- and decoder (Qwen)-only backbones, achieving SOTA across 48 retrieval tasks at all scales with exceptional parameter efficiency.
- Proposed difficulty-aware progressive instruction supervised fine-tuning to ensure models at different scales learn effectively.
- Introduced an advanced data refinement method, including domain-specific data synthesis, mutual-agreement-based false positive filtering, and difficulty-aware hard negative mining.
- Proposed multi-teacher knowledge distillation to further improve small-sized model performance and achieve parameter-efficiency for on-device model implementation.
- Introduced a light-weight IR validation set, enabling over 30ร faster model development while maintaining reliable performance rankings.
DisastIR: Automatic Information Retrieval Benchmark Construction for Data-Scarce Domain
EMNLP 2025 Findings
- Developed DisastIR, a comprehensive IR benchmark covering 48 distinct retrieval tasks with over 1.3M automatically labeled query-passage pairs for a data-scarce domain.
- Developed a four-stage automatic query-passage relevance labeling framework which fully replaces human labeling, ensures zero "hole" rate for model evaluation and achieves significant consistency for model performance ranking with human annotations.
- Benchmarked 30 retrieval models (33Mโ7B) under exact and ANN search, guiding model selection.
DisastQA: A Comprehensive Benchmark for Question Answering Evaluation in Disaster Management
ACL 2026 Findings | Equal contribution, author order by alphabetical order
- Developed DisastQA, a QA benchmark covering both multiple choice and open-ended question types with 3,000 QA pairs based on DisastIR.
- Proposed a Human-LLM collaborative pipeline for efficient benchmark development with key point extraction for open-ended questions, ensuring verifiable evaluation.
- Evaluated 18 LLMs considering different upstream retrieval performances under no relevant passage (base), only relevant passage (golden), and mixture (mix) settings.
Disaster-Agent: LLM-based Multi-agent System for Complex Disaster Management Task Solving
This project has led to two papers:
DisasterBench: Benchmarking LLM Planning under Typed Tool Interface Constraints
EMNLP 2026 Findings | Equal contribution, author order by alphabetical order
- Built DisasterBench to test whether LLMs can compose executable multi-step workflows from 26 disaster-response tools, with 233 tasks and 81 typed tool dependencies.
- Proposed First-Point-of-Failure (FPoF), a diagnostic that finds the earliest step where a predicted plan diverges from the ground truth, separating root-cause errors from cascading errors.
- Evaluated 14 LLMs with 5 planning methods, showing that tool mismatch and parameter-binding errors cause most first failures.
From Apparent Search Gains to Selection Failures: Auditing Test-Time Inference in Structured Workflow Planning
arXiv:2611.00579
- Split test-time inference for structured workflow planning into three stages, discovery, structural validation, and selection, and audited each stage on DisasterBench.
- Showed that a nominal tree search behaves like non-adaptive workflow sampling, and that majority voting leaves a discovered correct workflow unselected on 10โ19% of tasks.
CrisisSense-LLM: Instruction Fine-Tuning LLM for Multi-task Social Media Text Processing in Disaster Informatics
Advanced Engineering Informatics (2026)
- Designed fine-tuning prompt for multi-task tuning (text classification and named entity recognition) in multi-turn conversation format to instruction fine-tune Llama3.1-8B in LoRA and full-parameter tuning settings.
- Searched for hyperparameter combinations of LoRA to achieve 96.7% performance of full-parameter tuning, achieving best overall accuracy of 87.2%.
- Fine-tuned with data-parallel mixed-precision using DeepSpeed ZeRO-Stage-3, reducing GPU time by 35%.
๐ Other Publications
-
DisasterLex: An Expert Concept-to-Schema Knowledge Graph for Geospatial Reasoning in Disaster Analytics.
Yiming Xiao, Ankit Basu, Kai Yin, Sahil Rohit Vartak, Christian Swords, Ali Mostafavi (2026). EMNLP 2026 Findings. -
Training-Free Retrieval-Augmented Generation with Reinforced Reasoning for Flood Damage Nowcasting.
Lipai Huang, Kai Yin, Chia-Fu Liu, Ali Mostafavi (2026). (Corresponding author) Computer-Aided Civil and Infrastructure Engineering. -
CrisiSense-RAG: Crisis Sensing Multimodal Retrieval-Augmented Generation for Rapid Disaster Impact Assessment.
Yiming Xiao, Kai Yin, Ali Mostafavi (2026). Computer-Aided Civil and Infrastructure Engineering. -
FloodSQL-Bench: A Retrieval-Augmented Benchmark for Geospatially-Grounded Text-to-SQL.
Hanzhou Liu, Kai Yin, Zhitong Chen, Chenyue Liu, Ali Mostafavi (2026). Proceedings of the AAAI Symposium Series. -
Disaster Management in the Era of Agentic AI Systems: A Vision for Collective Human-Machine Intelligence for Augmented Resilience.
Bo Li, Junwei Ma, Kai Yin, Yiming Xiao, Chia-Wei Hsu, Ali Mostafavi (2025). -
Automated Wildfire Damage Assessment from Multi-View Ground Level Imagery via Vision Language Models.
Miguel Esparza, Archit Gupta, Kai Yin, Yiming Xiao, Ali Mostafavi (2025). -
Deep Learning-driven Community Resilience Rating based on Intertwined Socio-Technical Systems Features.
Kai Yin, Bo Li, Ali Mostafavi (2026). npj Urban Sustainability. -
Unsupervised graph deep learning reveals emergent flood risk profile of urban areas.
Kai Yin, Ali Mostafavi (2023). (Under review, npj Urban Sustainability) -
An integrated resilience assessment model of urban transportation network: A case study of 40 cities in China.
Kai Yin, Wu, J., Wang, W., Lee, D. H., & Wei, Y. Transportation Research Part A.
๐ Education
-
2022.07 โ 2027.05 (exp.)
Texas A&M University, College Station, TX, USA
Ph.D. candidate in Computer Science
๐ Skills
๐ Honors & Awards
- Passed CSE Ph.D. Qualifying Exam with 99th percentile score.
- National Second Prize, 13th National Undergraduate Transportation Science and Technology Competition (First Author).