Carbon-Aware Routing for Function Calling in Edge-Cloud LLM Systems
An arXiv paper proposes a carbon-aware routing framework that distributes function-calling queries across a three-tier edge-cloud architecture using edge and cloud LLMs. A lightweight k-NN predictor in a unified semantic-lexical embedding space estimates per-query accuracy, delay, and power, and combines these with real-time grid carbon intensity to route queries to the lowest-emission tier; evaluations report matching cloud-level accuracy while cutting operational carbon emissions by about 4× on average.