About Me

Hello! I am a Ph.D. candidate in Computer Science at Texas A&M University (TAMU), advised by Prof. James Caverlee and Prof. Ali Mostafavi. I am currently an Applied Scientist Intern at AWS AI Lab, and I was previously a PhD AI/ML Engineer Intern at LinkedIn.

My research lies at the intersection of multi-agent systems and information retrieval.

I am open to research collaboration. Feel free to reach me at kai_yin@tamu.edu.

๐Ÿ”ฅ News

  • 2026.09 ๐Ÿ’ผ Joined AWS AI Lab (New York City) as an Applied Scientist Intern, working on agentic RL for enterprise deep search.
  • 2026.09 ๐ŸŽ“ Passed the Ph.D. preliminary exam and became a Ph.D. candidate in Computer Science.
  • 2026.08 ๐ŸŽ‰๐ŸŽ‰ DisasterBench is accepted by EMNLP 2026 Findings!  Paper  Code
  • 2026.08 ๐ŸŽ‰๐ŸŽ‰ DisasterLex is accepted by EMNLP 2026 Findings!  Paper
  • 2026.05 ๐Ÿ’ผ Joined LinkedIn (Mountain View) as a PhD AI/ML Engineer Intern, working on post-training small language models for job search.
  • 2026.04 ๐ŸŽ‰๐ŸŽ‰ DMRetriever is accepted by ACL 2026 Main!  arXiv  Code  Data  Model
  • 2026.04 ๐ŸŽ‰๐ŸŽ‰ DisastQA is accepted by ACL 2026 Findings!  arXiv
  • 2025.11 ๐Ÿš€ Released source code, training data, and model checkpoints for DMRetriever (33Mโ€“7.6B).  Code  Data  Model
  • 2025.08 ๐ŸŽ‰๐ŸŽ‰ DisastIR is accepted by EMNLP 2025 Findings!  Paper
  • 2025.08 ๐Ÿš€ Released source code and dataset for DisastIR.  Code  Dataset

๐Ÿ’ผ Internship Experience

Applied Scientist Intern AWS AI Lab, New York City
Sep 2026 โ€“ Dec 2026 (expected)
  • Agentic RL for enterprise deep search (work in progress).
PhD AI/ML Engineer Intern LinkedIn, Mountain View
May 2026 โ€“ Aug 2026

FacetLLM-1.5B: post-training a small language model for dynamic facet suggestion in job search.

  • Reasoning-enhanced training data generation that follows strict product policy and works without PM-annotated data.
  • Multi-task hybrid supervised fine-tuning with a task-weighted loss, so the model internalizes the reasoning chain and keeps performance close to full reasoning at inference.
  • Difficulty-aware DPO and multi-reward GRPO with Reinforcement Learning from Verifiable Rewards (RLVR) to further improve model performance.
  • Human annotation-driven LLM-as-Judge prompt development that distills expert knowledge into judge prompts for offline evaluation, avoiding expensive human annotation.

๐Ÿ“ Selected Publications and Projects

ACL 2026 Main
DMRetriever

DMRetriever: Difficulty-aware Progressive Fine-tuning LLM for Improved Textual Retrieval in Data-Scarce Domain

Kai Yin, Xiangjue Dong, Chengkai Liu, Allen Lin, Lingfeng Shi, Ali Mostafavi, & James Caverlee

ACL 2026 Main

  • Developed DMRetriever, a family of six dense retrieval models (33Mโ€“7.6B) from encoder (BERT)- and decoder (Qwen)-only backbones, achieving SOTA across 48 retrieval tasks at all scales with exceptional parameter efficiency.
  • Proposed difficulty-aware progressive instruction supervised fine-tuning to ensure models at different scales learn effectively.
  • Introduced an advanced data refinement method, including domain-specific data synthesis, mutual-agreement-based false positive filtering, and difficulty-aware hard negative mining.
  • Proposed multi-teacher knowledge distillation to further improve small-sized model performance and achieve parameter-efficiency for on-device model implementation.
  • Introduced a light-weight IR validation set, enabling over 30ร— faster model development while maintaining reliable performance rankings.
EMNLP 2025 Findings
DisastIR

DisastIR: Automatic Information Retrieval Benchmark Construction for Data-Scarce Domain

Kai Yin, Xiangjue Dong, Chengkai Liu, et al., Ali Mostafavi, & James Caverlee

EMNLP 2025 Findings

  • Developed DisastIR, a comprehensive IR benchmark covering 48 distinct retrieval tasks with over 1.3M automatically labeled query-passage pairs for a data-scarce domain.
  • Developed a four-stage automatic query-passage relevance labeling framework which fully replaces human labeling, ensures zero "hole" rate for model evaluation and achieves significant consistency for model performance ranking with human annotations.
  • Benchmarked 30 retrieval models (33Mโ€“7B) under exact and ANN search, guiding model selection.
ACL 2026 Findings
DisastQA

DisastQA: A Comprehensive Benchmark for Question Answering Evaluation in Disaster Management

Zhitong Chen*, Kai Yin*, Xiangjue Dong, Chengkai Liu, Ali Mostafavi, & James Caverlee

ACL 2026 Findings  |  Equal contribution, author order by alphabetical order

  • Developed DisastQA, a QA benchmark covering both multiple choice and open-ended question types with 3,000 QA pairs based on DisastIR.
  • Proposed a Human-LLM collaborative pipeline for efficient benchmark development with key point extraction for open-ended questions, ensuring verifiable evaluation.
  • Evaluated 18 LLMs considering different upstream retrieval performances under no relevant passage (base), only relevant passage (golden), and mixture (mix) settings.
EMNLP 2026 Findings
Disaster-Agent

Disaster-Agent: LLM-based Multi-agent System for Complex Disaster Management Task Solving

This project has led to two papers:

DisasterBench: Benchmarking LLM Planning under Typed Tool Interface Constraints

Zhitong Chen*, Kai Yin*, Weifeng Zhang, Zhiyuan Wang, Xiangjue Dong, Chengkai Liu, Zhewei Liu, Yiming Xiao, Ali Mostafavi, & James Caverlee

EMNLP 2026 Findings  |  Equal contribution, author order by alphabetical order

  • Built DisasterBench to test whether LLMs can compose executable multi-step workflows from 26 disaster-response tools, with 233 tasks and 81 typed tool dependencies.
  • Proposed First-Point-of-Failure (FPoF), a diagnostic that finds the earliest step where a predicted plan diverges from the ground truth, separating root-cause errors from cascading errors.
  • Evaluated 14 LLMs with 5 planning methods, showing that tool mismatch and parameter-binding errors cause most first failures.

From Apparent Search Gains to Selection Failures: Auditing Test-Time Inference in Structured Workflow Planning

Zhitong Chen, Weifeng Zhang, Kai Yin, Zhiyuan Wang, Ali Mostafavi, & James Caverlee

arXiv:2611.00579

  • Split test-time inference for structured workflow planning into three stages, discovery, structural validation, and selection, and audited each stage on DisasterBench.
  • Showed that a nominal tree search behaves like non-adaptive workflow sampling, and that majority voting leaves a discovered correct workflow unselected on 10โ€“19% of tasks.
Adv. Eng. Inform. 2026
CrisisSense-LLM

CrisisSense-LLM: Instruction Fine-Tuning LLM for Multi-task Social Media Text Processing in Disaster Informatics

Kai Yin, Bo Li, Chengkai Liu, Ali Mostafavi, & Xia Hu

Advanced Engineering Informatics (2026)

  • Designed fine-tuning prompt for multi-task tuning (text classification and named entity recognition) in multi-turn conversation format to instruction fine-tune Llama3.1-8B in LoRA and full-parameter tuning settings.
  • Searched for hyperparameter combinations of LoRA to achieve 96.7% performance of full-parameter tuning, achieving best overall accuracy of 87.2%.
  • Fine-tuned with data-parallel mixed-precision using DeepSpeed ZeRO-Stage-3, reducing GPU time by 35%.

๐Ÿ“„ Other Publications

  • DisasterLex: An Expert Concept-to-Schema Knowledge Graph for Geospatial Reasoning in Disaster Analytics.
    Yiming Xiao, Ankit Basu, Kai Yin, Sahil Rohit Vartak, Christian Swords, Ali Mostafavi (2026). EMNLP 2026 Findings.  Paper  arXiv
  • Training-Free Retrieval-Augmented Generation with Reinforced Reasoning for Flood Damage Nowcasting.
    Lipai Huang, Kai Yin, Chia-Fu Liu, Ali Mostafavi (2026). (Corresponding author) Computer-Aided Civil and Infrastructure Engineering.  arXiv
  • CrisiSense-RAG: Crisis Sensing Multimodal Retrieval-Augmented Generation for Rapid Disaster Impact Assessment.
    Yiming Xiao, Kai Yin, Ali Mostafavi (2026). Computer-Aided Civil and Infrastructure Engineering.  arXiv
  • FloodSQL-Bench: A Retrieval-Augmented Benchmark for Geospatially-Grounded Text-to-SQL.
    Hanzhou Liu, Kai Yin, Zhitong Chen, Chenyue Liu, Ali Mostafavi (2026). Proceedings of the AAAI Symposium Series.  arXiv
  • Disaster Management in the Era of Agentic AI Systems: A Vision for Collective Human-Machine Intelligence for Augmented Resilience.
    Bo Li, Junwei Ma, Kai Yin, Yiming Xiao, Chia-Wei Hsu, Ali Mostafavi (2025).  arXiv
  • Automated Wildfire Damage Assessment from Multi-View Ground Level Imagery via Vision Language Models.
    Miguel Esparza, Archit Gupta, Kai Yin, Yiming Xiao, Ali Mostafavi (2025).  arXiv
  • Deep Learning-driven Community Resilience Rating based on Intertwined Socio-Technical Systems Features.
    Kai Yin, Bo Li, Ali Mostafavi (2026). npj Urban Sustainability.  arXiv
  • Unsupervised graph deep learning reveals emergent flood risk profile of urban areas.
    Kai Yin, Ali Mostafavi (2023). (Under review, npj Urban Sustainability)  arXiv
  • An integrated resilience assessment model of urban transportation network: A case study of 40 cities in China.
    Kai Yin, Wu, J., Wang, W., Lee, D. H., & Wei, Y. Transportation Research Part A.  Paper

๐Ÿ“– Education

  • 2022.07 โ€“ 2027.05 (exp.) Texas A&M University, College Station, TX, USA
    Ph.D. candidate in Computer Science

๐Ÿ›  Skills

Python  ยท  PyTorch  ยท  AutoGen  ยท  Hugging Face (Transformers, PEFT)  ยท  DeepSpeed  ยท  Verl

๐Ÿ† Honors & Awards

  • Passed CSE Ph.D. Qualifying Exam with 99th percentile score.
  • National Second Prize, 13th National Undergraduate Transportation Science and Technology Competition (First Author).