Guides → Playground & Guide → Data Freshness / Re-Embedding Cost - The Recurring Data Treadmill
Meet Priya Nair. ML engineer keeping a RAG knowledge base current. "Our corpus changes weekly. Re-embedding and reindexing is not a one-time cost - what does keeping it fresh cost every month?"
🔥 We embedded the corpus once and budgeted for that. Six months later the re-embedding runs are a standing line nobody forecast.
A RAG corpus is never done. Documents change, models update, and stale embeddings quietly degrade answers - so re-embedding and reindexing become recurring, not one-time.
This calculator models the treadmill: re-embedding cadence, reindex cost, pipeline runs, and drift monitoring, rolled into a monthly data-freshness bill.
Seeing the recurring number reframes the decision - sometimes a cheaper embedding model or a smarter refresh cadence saves more than any one-time optimization.
Here are the inputs that move the result the most. Play with the sliders and check it out. The number updates live.
Corpus size x refresh cadence x churn, plus pipeline and monitoring. This is the data-drift fix bill.
💡Monthly = re-embed + vector ops + pipeline per run, plus the amortized full re-embed and monitoring.
Open the full calculator. Pick a model, enter your tokens, see per-call, daily, monthly, and annual cost.
🚀 Open the full calculator →Model the recurring data-drift bill: re-embedding, reindexing, pipeline runs, and drift monitoring for a RAG corpus that has to stay current.
Each input shapes the cost. Click an input on the calculator to set it. The explanations below match the live calculator field by field.
Context: Corpus sizes assume ~260–520 tokens per dense page; tech docs ~5K, legal ~31K, long reports ~80K tokens per document.
| Input | Default | Typical ballparks |
|---|---|---|
corpusTokens
moves the needle
|
50,000,000 | Small KB · ~500 dense pages = 200,000 · Mid KB · ~2,500 pages / ~700 docs = 1,000,000 · Large corpus · ~10K mixed docs = 50,000,000 · Enterprise archive · ~50K docs w/ reports = 500,000,000 |
refreshRunsPerMonth
moves the needle
|
30 | Weekly full refresh = 4 · Nightly refresh = 30 · Twice-daily = 60 · Hourly incremental = 720 |
pctCorpusChangedPerRun
moves the needle
|
2 | Slow-moving KB · ~0.5% = 0.5 · Typical KB · ~1–2%/run = 2 · Fast-moving content · ~5% = 5 · Full re-embed run · 100% = 100 |
fullReembedEveryMonths
|
6 | Monthly rebuild = 1 · Quarterly rebuild = 3 · Semi-annual = 6 · Only on embedding-model change (~yr) = 12 |
pipelineUsdPerRun
|
50 | Light serverless job · $10 * = 10 · Typical ETL run · $50 * = 50 · Heavy multi-source · $250 * = 250 (rough estimates) |
driftMonitoringUsdPerMonth
|
200 | Basic checks · $50/mo = 50 · Managed monitoring · $200/mo = 200 · Full eval suite · $1K/mo = 1,000 |
baseMonthlyTokenUsd
|
0 | Pilot spend · ~$500/mo = 500 · Team · ~$5K/mo = 5,000 · Department · ~$50K/mo = 50,000 · Enterprise · ~$250K/mo = 250,000 |
embeddingModelSlug
|
openai-text-embedding-3-small | — |
vectorOpsUsdPerMillionTokens
|
0.1 | — |
Ballparks are broad industry starting points (sourced ranges; * = rough estimate) — your result gets more accurate as you replace them with measured numbers.
Try them live in the calculator;
API & agent users get the same data from the MCP resource aicost://input-reference/data-freshness-cost.
The gaps we just listed are real, and they are the expensive ones: your actual prompts, your switching cost, your MLOps overhead. An AICost expert spends the hour on your AI and cloud costs, not a generic playbook. You leave with a written report: the way forward, in 30 days of concrete steps.
Not sure yet? The $39 AICost Blueprint credits toward a Session, and the Session fee credits toward any plan. You never pay twice for the same ground. See all pricing →
You have seen the shape of it. Open the calculator with your model, your tokens, your volume.
🚀 Open the full calculator →Author: Subu Vdaygiri, Founder & CEO of CloudIntelligence.ai. 17 years Fortune 100 (Ingram Micro, Siemens). Wharton CTO program · Kellogg CPO program · 10× AWS+Azure certified.
Last-verified date is the most recent successful daily snapshot
(aicost_pricing_snapshots) or, when no snapshot exists yet,
the latest successful crawler run (aicost_crawler_runs).
10 of 10
vendors are currently verified. Aggregator services (TokenCost, AI Pricing Guru, etc.)
are not listed.
Derived from industry conventions, not directly published by the vendor. Typical conventions: cached input = 10% of base (90% off), Batch API = 50% of base (50% off).
| Vendor / Model | Field | Why it’s inferred |
|---|---|---|
| Anthropic — Claude Sonnet 4.6 | cachedInput |
Derived at 10% of input rate — Anthropic publishes 90% cache-hit discount on this tier. |
| Anthropic — Claude Sonnet 4.5 | cachedInput |
Derived at 10% of input rate; same 90% cache-hit convention as Sonnet 4.6. |
| Anthropic — Claude Sonnet 4.5 | batchInput |
Derived at 50% of standard input — Anthropic documents uniform 50% Batch discount. |
| Anthropic — Claude Sonnet 4.5 | batchOutput |
Derived at 50% of standard output — Anthropic documents uniform 50% Batch discount. |
| Anthropic — Claude Haiku 4.5 | cachedInput |
Derived at 10% of input rate — Anthropic 90% cache-hit discount convention. |
| OpenAI — GPT-5.4 Mini | cachedInput |
Derived at 10% of input — OpenAI documents automatic 90% discount on cache hits across GPT-5.x tier. |
| OpenAI — GPT-5.4 Nano | cachedInput |
Derived at 10% of input — OpenAI 90% cache-hit convention. |
| OpenAI — GPT-5.4 Nano | batchInput |
Derived at 50% of input — OpenAI Batch API uniform 50% discount. |
| OpenAI — GPT-5.4 Nano | batchOutput |
Derived at 50% of output — OpenAI Batch API uniform 50% discount. |
| OpenAI — GPT-5.4 Pro | cachedInput |
Derived at 10% of input — OpenAI 90% cache-hit convention. |
| OpenAI — GPT-5.4 Pro | batchInput |
Derived at 50% of input — OpenAI Batch API uniform 50% discount. |
| OpenAI — GPT-5.4 Pro | batchOutput |
Derived at 50% of output — OpenAI Batch API uniform 50% discount. |
| OpenAI — GPT-5.2 | cachedInput |
Derived at 10% of input; no residency uplift. |
| OpenAI — GPT-5.2 | batchInput |
Derived at 50% of input. |
| OpenAI — GPT-5.2 | batchOutput |
Derived at 50% of output. |
| OpenAI — GPT-5 | cachedInput |
Derived at 10% of input. |
| OpenAI — GPT-5 | batchInput |
Derived at 50% of input. |
| OpenAI — GPT-5 | batchOutput |
Derived at 50% of output. |
| OpenAI — GPT-5.5 Pro | cachedInput |
Derived at 10% of input — OpenAI does not publish a cached rate for *-pro models; using the family convention. |
| OpenAI — GPT-5.5 Pro | batchInput |
Derived at 50% of input. |
| OpenAI — GPT-5.5 Pro | batchOutput |
Derived at 50% of output. |
| OpenAI — GPT-5.2 Pro | cachedInput |
Derived at 10% of input — pro-tier convention. |
| OpenAI — GPT-5.2 Pro | batchInput |
Derived at 50% of input. |
| OpenAI — GPT-5.2 Pro | batchOutput |
Derived at 50% of output. |
| OpenAI — GPT-5.1 | batchInput |
Derived at 50% of input. |
| OpenAI — GPT-5.1 | batchOutput |
Derived at 50% of output. |
| OpenAI — GPT-5 Pro | batchInput |
Derived at 50% of input. |
| OpenAI — GPT-5 Pro | batchOutput |
Derived at 50% of output. |
| OpenAI — GPT-5 Nano | cachedInput |
Derived at 10% of input. |
| OpenAI — GPT-5 Nano | batchInput |
Derived at 50% of input. |
| OpenAI — GPT-5 Nano | batchOutput |
Derived at 50% of output. |
| Google — Gemini 3 Flash | cachedInput |
Derived at 10% of input — Google caching discount convention ~90%. |
| Google — Gemini 3.1 Flash-Lite | cachedInput |
Derived at 10% of input — Google caching convention. |
| Google — Gemini 3.1 Flash-Lite | batchInput |
Derived at 50% of input — Google Batch API uniform 50% discount. |
| Google — Gemini 3.1 Flash-Lite | batchOutput |
Derived at 50% of output — Google Batch API uniform 50% discount. |
| Google — Gemini 2.5 Pro | cachedInput |
Derived at 10% of input. |
| Google — Gemini 2.5 Flash | cachedInput |
Derived at 10% of input. |
| Google — Gemini 2.5 Flash-Lite | cachedInput |
Derived at 10% of input — Google caching convention. |
| Google — Gemini 2.5 Flash-Lite | batchInput |
Derived at 50% of input — Google Batch API uniform 50% discount. |
| Google — Gemini 2.5 Flash-Lite | batchOutput |
Derived at 50% of output — Google Batch API uniform 50% discount. |
| Google — Gemini 2.0 Flash | cachedInput |
Derived at 25% of input per Google 2.0 family caching rates. |
| Google — Gemini 2.0 Flash | batchInput |
Derived at 50% of input — Google Batch API uniform 50% discount. |
| Google — Gemini 2.0 Flash | batchOutput |
Derived at 50% of output — Google Batch API uniform 50% discount. |
| Google — Gemini 2.0 Flash-Lite | cachedInput |
Derived at 10% of input — Google caching convention. |
| Google — Gemini 2.0 Flash-Lite | batchInput |
Derived at 50% of input — Google Batch API uniform 50% discount. |
| Google — Gemini 2.0 Flash-Lite | batchOutput |
Derived at 50% of output — Google Batch API uniform 50% discount. |
| xAI — Grok 4 (legacy) | cachedInput |
Extrapolated at 25% of base. |
Pricing is cross-verified against the
LiteLLM community registry
when available. Daily snapshots are kept in aicost_pricing_snapshots;
every change is logged to aicost_price_changelog with old & new
values for full audit trail. Read the full methodology →