BM25

2026-10-28

RESEARCH · 1 SOURCE · arXiv cs.AI

When Retrieval Is Not Enough: obsolete retrieved facts can degrade agent accuracy

arXiv:2610.36308v1 analyzes the single-hop FactConsolidation task in MemoryAgentBench and shows that retrieving the newest statement is not sufficient for correct answers. With Qwen2.5-7B-Instruct on a 6K-history setup, BM25 accuracy falls from 45% to 32% when the retrieved evidence budget increases from 64 to 512 tokens despite a gold audit finding the newest statement in each retrieved set; substituting an unrelated statement with an obsolete competing statement drops accuracy from 100% to 44.6% for Qwen2.5 and from 100% to 87.8% for Qwen3-14B (all 50 harmful paired flips were classified as stale), and a 32K-history replication shows BM25-64 outperforming BM25-512 (69% vs 58%).

7.0

2026-10-02

COMPANIES · 1 SOURCE · AWS Machine Learning

Amazon SageMaker AI demonstrates multi-turn reinforcement learning (MTRL) fine-tuning for search agents with Qwen3.6-27B

Amazon’s blog post describes using Amazon SageMaker AI’s multi-turn reinforcement learning (MTRL) capability to fine-tune a Qwen3.6-27B model for an LLM-powered search agent. The write-up details MTRL features—serverless execution, modular agent-environment interfaces, asynchronous rollouts, built-in algorithms (PPO, CISPO, IS), trajectory observability via MLflow, and evaluation jobs—and shows an enterprise search use case combining BM25 and vector search tools.

6.0

2026-10-02

RESEARCH · 1 SOURCE · arXiv cs.AI

Measuring the microtask eligibility gap for off-the-shelf SLMs (arXiv:2610.00025v1)

This paper builds a benchmark of four microtasks around LLM planners (auto-approving commands, writing memory, tool selection, ranking turns) with pre-specified, CI-backed eligibility thresholds and tests off-the-shelf small language models (Qwen3 0.6/1.7/4/8B, greedy FP16, no tuning). Across 16 model×task configurations, none met the practitioner-defined thresholds (0/16). The authors introduce a logprob decision-threshold diagnostic separating capability deficits from decoding-threshold failures, show 4-bit quantization methods (RTN/GPTQ/AWQ) generally worsen performance without enabling eligibility, replicate the gap on Llama-3.x (12/12 ineligible), and show results are robust to anchor choice and prompt paraphrases; they recommend placing SLMs behind a baseline that already meets the CI-backed threshold and only using SLMs where that baseline fails.

7.0

2026-09-15

RESEARCH · 1 SOURCE · arXiv cs.AI

Solar Intelligence — a hybrid retrieval-augmented platform for solar analytics, evidence-grounded QA, and forecasting

This arXiv cs.AI paper introduces Solar Intelligence, a hybrid retrieval-augmented system that unifies structured solar analytics, evidence-grounded scientific question answering, and machine-learning forecasting. The platform integrates daily NASA POWER and Biosphere 2 sensor data with a curated research corpus, uses DuckDB for structured queries, fuses BM25 and ChromaDB dense embeddings via Reciprocal Rank Fusion for retrieval with grounding by llama3.2:3b, produces daily irradiance/temperature/wind forecasts with XGBoost, and is exposed via FastAPI, Streamlit, and an MCP server for use as an app, API, or agent tool.

5.0

2026-05-28

CODING · 1 SOURCE · Mistral AI

Search Toolkit enters public preview — open-source composable framework for AI search pipelines

Search Toolkit is being released in public preview as an open-source, composable framework to build production search pipelines for AI applications. It unifies ingestion, retrieval (BM25, dense embeddings, and hybrid), and built-in evaluation (recall, precision, MRR, NDCG), supports Connectors for live data, and is designed to run on cloud, on-premises, or edge infrastructure.

6.0