Tech Meridian ← ENTITY INDEX
PROMY MERIDIAN RU

COMPANY · ENTITY #13532

LiteLLM

Related event timeline, sources and context from the news index.

EVENT TIMELINE

4

MODELS · 1 SOURCE · AWS Machine Learning

Deploy NVIDIA NeMo Switchyard (switchyard-server 0.3.0) with LiteLLM v1.102.0 to route requests to Amazon Bedrock via AWS CDK

This sample shows how to use the AWS CDK to deploy LiteLLM (v1.102.0) with the NVIDIA NeMo Switchyard plugin (switchyard-server 0.3.0) on Amazon ECS/Fargate, routing agent requests to models served through Amazon Bedrock. It demonstrates two Switchyard strategies (stage and random), the request flow through VPC-based Bedrock Converse APIs, and benchmark runners to measure cost per solved coding task; the integration is experimental and intended for non-production evaluation.

5.0

CODING · 1 SOURCE · AWS Machine Learning

Building a self-service LLM gateway with Amazon Bedrock and LiteLLM

A how-to post describes a pattern that uses LiteLLM (running on AWS Fargate) to expose an OpenAI-compatible API that proxies Amazon Bedrock models, enabling per-user API keys, cost controls, and observability via CloudWatch and Langfuse. The walkthrough includes AWS CDK examples, uses Claude Sonnet and Claude Haiku as examples, and outlines storage, caching, and IAM configurations for a production topology.

5.0

CODING · 1 SOURCE · AWS Machine Learning

Amazon publishes 'Pluggable Agentic AI Framework' — a seven-layer reference architecture for Bedrock AgentCore

Amazon published a blog post presenting the Pluggable Agentic AI Framework, a seven-layer composable reference architecture for running multi-agent AI applications at production scale on Amazon Bedrock AgentCore. The framework defines distinct layers (Data & Knowledge, Model & Inference, Orchestration, agent registry, etc.), protocols like MCP and A2A, and example building blocks such as Vector Engine for OpenSearch Serverless, S3 Vectors, DynamoDB, LiteLLM, and Strands Agents.

6.0

CODING · 1 SOURCE · InfoQ AI, ML & Data Engineering

InfoQ: Platform engineering playbook for production LLMs

InfoQ publishes a platform-engineering playbook based on a retail inventory recommendation system that reduced hallucination rate from ~15% to ~1.5% without changing the foundation model by applying platform-level controls (automated retry loops, intent-validation gates, prompt registries, tool authorization, and custom instrumentation). The article links a companion GitHub repo with runnable examples that use a local llama3.1 via Ollama connected through LiteLLM and describes routing through foundation models such as Gemini and GPT under enterprise traffic.

7.0