Tech Meridian ← ENTITY INDEX
PROMY MERIDIAN RU

COMPANY · ENTITY #13316

7B model

Related event timeline, sources and context from the news index.

EVENT TIMELINE

2

COMPANIES · 1 SOURCE · InfoQ AI, ML & Data Engineering

LinkedIn engineer details production LLM ranking optimizations including 7B→400M distillation

At QCon AI (covered on InfoQ), Sundara Raman Ramachandran (LinkedIn’s LLM inference lead) described how LinkedIn productionizes LLM-based ranking under tight P99 latency SLAs by distilling a 7B model to ~400M parameters, compressing job contexts into single-token embeddings to boost throughput ~10×, and removing system bottlenecks (custom flash-attention masks, Python GIL workarounds, gc.freeze, non-blocking CUDA memory copies).

7.0

RESEARCH · 1 SOURCE · arXiv cs.AI

CourseChat: on‑premises multi-course RAG tutor for business education (arXiv:2610.02510v1)

The authors present CourseChat, an on-premises, multi-course retrieval-augmented generation (RAG) tutor deployed behind a campus web gateway and integrated for use with Moodle; it runs twin edge hosts with FastAPI, a local vector database, and an LLM served via Ollama across six isolated course CRNs. The paper reports two model bake-offs, a source-fidelity comparison, conversation and quiz audits, finds that larger models failed classroom speed constraints while 12B and 7B passed and a mixture-of-experts introduced new errors, and retains an 8B production model pending demonstrated overall improvement; the study does not evaluate learning gains or public-gateway acceptance.

6.0