Tech Meridian ← LIVE FEED
PROMY MERIDIAN RU

RESEARCH · RESEARCH · #3655

Airbnb introduces CASTLE framework to synthesize queries and relevance labels for natural-language search

Airbnb researchers introduced CASTLE, a framework that uses de-identified booking-session listing pairs and large language models to jointly generate synthetic natural-language search queries and built-in relevance labels, plus a Virtual Judge for cases lacking booking history and a production variant called CASTLE-R. In evaluations CASTLE reduced KL divergence to real-user query distributions (1.01) versus 9.33 for the best baseline and showed 91–93% agreement with human reviewers on sampled relevance assignments.

KEY POINTS

  1. Airbnb researchers introduced CASTLE, a framework that uses de-identified booking-session listing pairs and large language models to jointly generate synthetic natural-language search queries and built-in relevance labels, plus a Virtual Judge for cases lacking booking history and a production variant called CASTLE-R.
  2. In evaluations CASTLE reduced KL divergence to real-user query distributions (1.01) versus 9.33 for the best baseline and showed 91–93% agreement with human reviewers on sampled relevance assignments.
  3. CASTLE offers a practical method to generate realistic synthetic queries and labels before real search traffic exists, addressing cold-start problems for natural-language search systems and improving alignment with real user behavior.

WHY IT MATTERS

CASTLE offers a practical method to generate realistic synthetic queries and labels before real search traffic exists, addressing cold-start problems for natural-language search systems and improving alignment with real user behavior.

SOURCES & TIMELINE

1