Tag: semantic-caching

Showing 1-2 of 2

Jun 29, 20267 min readTooling Deep DiveIntermediate120 min build

Deploy AI-Gateway: an open-source semantic-caching reverse proxy to reduce LLM API costs

Guide to running AI-Gateway, an open-source semantic-caching reverse proxy that can cut LLM API/token costs (repo cites ~40–70%). Covers setup checklist, verification, and rollout tips.

ai-gateway semantic-caching reverse-proxy llm-api open-source

+2 more

deployment tutorial

TutorialsFrance

Open

Mar 10, 20268 min readTooling Deep DiveIntermediate480 min build

Audit and lightweight controls to reduce multi-provider LLM API spend

Run an invoice-and-endpoint audit to recover wasted LLM API spend—community examples show ~60% recoverable using model routing, prompt compression, retry dedupe, and semantic caching.

finops api-costs model-routing prompt-compression retry-deduplication

+2 more

semantic-caching observability