How this actually works
Two mirrors and a windshield
Most AI-cost tools stop at the first two. Seeing the crash and understanding the crash are not the same as avoiding it.
See it
Where the money goes, which team, which project, which feature — and why.
- Bill diagnose — paste an export, get the cost mix and the first cuts
- Ledger attribution by feature, customer, model, provider
- Unifies spend scattered across cloud SKUs
- Cost per task, per customer, per successful task
Cut it
Tokens → RAG → agent loops. The whole chain, not just the model price.
- Multi-model router — with a quality floor
- Prompt cache ROI, batch vs realtime
- Chunking, context windows, vector DB
- Agent loops, retries, reasoning tokens
Plan it — and stop it
The part nobody else does. Model the cost before you ship, and cap it in production.
- TCO complete + agentic TCO/ROI — decide before you build
- Scale projection, forecasters — for workloads that refuse to be linear
- Cost Firewall — budgets, daily caps, spike anomaly, kill switch
- Agentic envelope — a hard cap per agent run
Where it plugs in
Three workflows you already have
The engines are not a destination. They run inside the loops your teams already operate.
Cost gate in CI/CD
Platform + engineering leadsA developer opens a PR that adds an AI feature. Your CI job asks AICost what it will cost at projected volume, and fails the build if it blows the budget. The number lands in the PR, where the decision is still cheap to change.
The second-order win: spend gets attributed by team and repo. You can finally answer “is this team’s AI budget buying us productivity, or just tokens?” without reading anyone’s prompts.
# .github/workflows/ai-cost-gate.yml - name: AI cost gate run: | curl -sS -X POST https://mcp.aicost.ai/v1/run \ -H "Authorization: Bearer ${{ secrets.AICOST_KEY }}" \ -H "Content-Type: application/json" \ -d '{"engine_id":"aicost.cost-calculator", "context":{"project_id":"checkout","unit_economics":{"tasks_per_month":250000}}, "inputs":{"requests_per_month":250000,"input_tokens_per_request":1800, "output_tokens_per_request":400}}' \ | tee cost.json # fail the build over budget — exit non-zero node ci/assert-under-budget.js cost.json --max-usd 4000
Bound an agent loop
AI platform + productTicket triage, research, code fixes — the loop is think → tool → observe, and it burns roughly 50× the tokens of a chat turn. The envelope gives every run a hard ceiling on cost, steps, tool calls, retries and reasoning budget. Exceed it and the run stops and escalates to a human, which is a fallback, not a bill.
Report cost per successful task, not per run. That single distinction is usually the difference between an agent that looks cheap and one that is.
RAG in customer support
Support ops + CXA support bot over your docs is the most common enterprise RAG workload — and the one where the LLM is rarely the biggest line. Embedding the corpus, re-embedding on every doc change, the vector store, the retrieval calls, the context you stuff into each answer: that is the bill.
Model the whole pipeline with rag-pipeline, embedding-cost, vector-db-cost, chunking-optimizer and data-freshness-cost — then get cost per deflected ticket with support-deflection-roi. That is the number your CX lead actually reports.
Integration
You change one thing: your gateway config
No agents to deploy. No cloud credentials to hand over. No data pipeline. If you already run an LLM gateway, you are most of the way there.
- Connect the endpoint 5 min
Add
https://mcp.aicost.ai/mcpas a connector in Claude, ChatGPT, Cursor or Perplexity, with your Bearer key. Ask it a question about your spend. That is the whole install. - Set a firewall budget 15 min
Run
aicost.cost-firewallon one workload withcompile_policy: "litellm". You get back budgets, daily hard caps, RPM/TPM limits, a spike-anomaly rule and a kill switch — already in your gateway’s config format. - Paste it into your gateway 20 min
Merge
policy_patch.compiled.configintolitellm-config.yaml, restart, send a request. Exceed the cap and you get an HTTP 402. That is the runaway, stopped, in your own traffic. - Wire the actuals back 45 min
Feed gateway spend logs, a FOCUS export, or AWS CUR / Azure Cost Management / GCP BigQuery into
ledger_ingest_actuals. Nowledger_reconciletells you modeled vs actual and what the savings really were. - Standardise the tags ongoing
Send
tenant_id,project_id,workload_idon every call. Everything downstream — attribution, reconciliation, the QBR report — depends on this and nothing else does the job retroactively.
What we will not do to save you money
Cheap answers are easy. The reason most cost tooling never leaves the pilot is that engineering does not trust it near production traffic. So the policies are constrained by your rules, not ours:
- No silent model downgrade. The router takes a
quality_floorand amax_latency. A cheaper model that fails the floor is never selected — the saving is simply not claimed. - No routing across a compliance boundary. Jurisdiction, data residency and vendor allow-lists are inputs, not afterthoughts. See jurisdiction-risk-scorer.
- No “trust us” numbers. Every output is dated, sourced and confidence-scored. Inferred values are flagged with an asterisk. Savings percentages state the exact spend they apply to — and we tell you when they cannot be added together.
- No access to your data. The engines model from facts you supply. No cloud credentials, no API keys, no customer data. Ever.
- Nothing is auto-applied. AICost emits a policy. A human reviews it and your gateway runs it. We never touch your traffic.
Gateways: LiteLLM (recommended — it natively wraps Bedrock, Azure OpenAI and Vertex), Cloudflare AI Gateway, Portkey, Kong. Both compile from the same decision, so pick per client and the envelope is identical. Policy types your gateway cannot express natively arrive under x_aicost_policy for your app layer, rather than being silently dropped.
For MSPs and consultancies
Sell AI cost control as your product
You already manage your clients’ cloud. AI is the line item now exploding on their bill, and almost nobody in the channel has a tool that prevents the runaway, explains it, or proves the fix. This is that tool — under your brand, with your markup, live in a day.
POST /admin/orgs → POST /admin/keys → set brand, markup, rate limit. Onboard a client in minutes.The demo that closes: the Cost Firewall. A stolen key that ran $82K in 48 hours would have been blocked at a ~$8/day cap on day one. Every client you have felt that story in their chest — and none of them can currently stop it.
Not just tokens
Your AI runs on a cloud. That bill is usually bigger.
Most teams look at the model bill. The infrastructure underneath it — GPUs, Kubernetes, vector stores, egress, the observability you turned all the way up — is where the rest of the money is.
AWS Bedrock
- Model Units: on-demand vs provisioned breakeven
- Knowledge Bases quietly provision OpenSearch — a 2-OCU floor you never asked for
- Cross-region inference routing
- CUR ingest → attribution across scattered SKUs
Azure OpenAI
- PTUs: billed per hour whether or not you use them
- Spillover policy — PTU → standard on 429
- Log Analytics ingest bills as monitoring, not AI
- Cost Management export → attribution
Google Vertex
- GSU provisioned throughput and its breakeven
- Bills per character, not per token — naive comparison misleads
- An idle L4 endpoint costs the same at zero requests
- BigQuery billing export — catches GPU under Compute Engine
And the infrastructure itself — commitments, Spot, Kubernetes right-sizing, storage tiering, egress paths — lives on our sister site. CostOptimization.ai is a vendor-neutral knowledge base of 37 cloud cost categories, each with a modeled savings range, the specific spend it applies to, the provider action, and the reason you cannot simply add them together.
You get two AI bills. The model bill, and the cloud bill underneath it. Most teams only look at the first one.
Access
Getting a key
MCP access is in beta and keys are issued by hand. That is deliberate — we would rather talk to the first hundred teams than watch a signup form.
- RequestTell us your gateway, your rough monthly AI spend, and the one cost question that is bothering you most. Request form or [email protected].
- We issueYou get an
aicost_sk_…Bearer key scoped to your org. MSPs get an admin key and can then issue their own client keys viaPOST /admin/keyswithout asking us. - You connectPaste the key into your MCP client, or use it against
POST /v1/runfrom CI. OAuth viaauth.aicost.aiis in progress for ChatGPT and Perplexity.
Keys are per-org and revocable. Rate limits are per-IP and per-tenant. Rotate any time — ask, and the old key dies immediately.
Start where it hurts
If you do not know where your AI spend goes, start with visibility. If you know and cannot stop it, start with the firewall. If you have not shipped yet, start with the windshield — it is the cheapest moment you will ever have.
Request an MCP beta key → Get the $39 Blueprint99 decision engines · 84 free calculators · no signup for the calculators · [email protected]