From a designed workflow to governed production
AICost decides, prices and compiles policy. Your app, agent, gateway and cloud run the work. This guide shows how the two connect.
Claude, Codex, Cursor, your own agent or CI pipeline calls the engines.
126 deterministic engines price each step, apply cost levers and return policy.
LiteLLM, Cloudflare AI Gateway, Amazon Bedrock, Azure AI Foundry or Vertex AI enforce the policy and run the models.
Bills and usage logs flow back so modeled and actual cost are compared and kept as evidence.
1. Connect to the engines
The engines run on the AICost MCP server. Use MCP from an AI client or agent, or the REST endpoint from code and CI. Both need an AICost key.
| Surface | Endpoint | Use it for |
|---|---|---|
| MCP (Streamable HTTP) | https://mcp.aicost.ai/mcp | Primary transport for Claude, Codex, Cursor and agents |
| MCP (legacy SSE) | https://mcp.aicost.ai/sse | Older clients |
| REST, one engine | POST https://mcp.aicost.ai/v1/run | Apps and back ends |
| REST, many engines | POST https://mcp.aicost.ai/v1/run-batch | CI gates and portfolios |
Authorization: Bearer <your AICost key>
2. The tools you will call
Three catalog tools cover most work. Workflow tools and first-class policy engines are shortcuts for common jobs.
| Tool | What it does |
|---|---|
| list_engines | Lists every engine with its inputs |
| recommend_engine | Takes a plain question and returns the engines that answer it |
| run_decision_engine | Runs one engine: engine_id, inputs, context |
| Workflow tools | estimate_llm_workload_cost, optimize_existing_workload, design_ai_stack, estimate_multimodal_cost, analyze_workload_portfolio and more |
| Ledger tools | ledger_ingest_actuals, ledger_reconcile, ledger_attribution |
{"name": "recommend_engine", "arguments": {"question": "our agent bill doubled last week, need to cap it"}}{"name": "run_decision_engine", "arguments": {"engine_id": "aicost.cost-firewall", "inputs": {}, "context": {"ledger": true}}}
Empty inputs run an engine on its defaults; every default is listed in the result as an assumption, marked with an asterisk when inferred.
3. What every engine returns
Each call returns the same decision envelope, so an agent can act on it and an auditor can trace it.
- result: the numbers or decision
- assumptions and confidence, with the inputs that drove them
- evidence with dates for every price used
- warnings for inferred inputs
- policy_patch: the policy the result implies
- companion_url: the same scenario in the web calculator
- engine_version, so a past answer stays reproducible; pricing can be pinned to a date with context.pricing_as_of
4. Run a designed workflow
Design or pick a workflow, then export its engine chain. The chain is an ordered list of run_decision_engine calls, one per step, plus the cost levers that apply.
Start from one of the enterprise workflows or describe your own in the designer.
Every step is priced by its engine; levers and governance controls are applied.
Download the chain JSON from the Cost and governance tab.
Give the chain to Claude or your agent, or call /v1/run-batch from code.
Add the CI cost gate so a change that breaks the budget fails the build.
POST https://mcp.aicost.ai/v1/run-batch
{"runs": [
{"engine_id": "aicost.rag-pipeline", "inputs": {}},
{"engine_id": "aicost.hitl-review-cost", "inputs": {}},
{"engine_id": "aicost.cost-firewall", "inputs": {}}
]}- name: AICost budget check
uses: aicost-ai/aicost-mcp/cicd@main
with:
config: .aicost.json
env:
AICOST_API_KEY: ${{ secrets.AICOST_API_KEY }}
5. Enforce policy in your LLM gateway
AICost compiles one gateway-neutral policy and exports it for the gateway you run. Your gateway enforces it; AICost stays out of the request path.
| Gateway or cloud | What you get | Status |
|---|---|---|
| LiteLLM | Proxy config: model list, routing and fallbacks, cache, retries, budgets. Rules LiteLLM has no setting for are kept under x_aicost_policy for your app layer. | Built |
| Cloudflare AI Gateway | Gateway settings: caching, retries, rate limiting. | Built |
| Amazon Bedrock | Model access template (approved models), cross-region inference policy. | Built |
| Azure AI Foundry | Provisioned throughput spillover to standard deployments. | Built |
| Vertex AI | Context caching policy for large repeated prompts. | Built |
| Portkey, Kong | Use the policy JSON; native exporters are planned. | Planned |
| AICost enforcing proxy | Optional proxy in front of LiteLLM that enforces caps itself: emergency stop, rate limit, daily hard cap, per-run caps, soft-cap downgrade. | Opt-in |
Budget rules in an export are starting values derived from the priced workflow. Review them before you apply them.
6. Connect AWS, Azure and GCP actuals
Modeled cost is only half the answer. Feed actual spend back and AICost compares the two, workload by workload.
| Source | How | Status |
|---|---|---|
| AWS | Live read-only connection: a CloudFormation role with an External ID lets AICost read Cost Explorer, including a Bedrock breakdown by usage type. CUR 2.0 and Bedrock invocation logs can also be loaded. | Live |
| Azure | Cost Management daily export (actual or amortized) or FOCUS 1.x export. | Export |
| GCP | BigQuery billing export rows, including Vertex AI and GPU compute. | Export |
| Any cloud | FOCUS 1.x billing files. | Export |
| LLM gateways | LiteLLM spend logs for per-team, per-workload cost. | Export |
Live API connections for Azure and GCP are planned. Exports work today.
7. Keep it governed
The same engines check the workflow on a schedule and keep the record.
- Governance sweep re-checks prices, models, vendors and spend against your gates
- An evidence journal keeps a hash-chained record of every finding, decision and policy change
- Ledger reconcile shows drift between modeled and actual cost, and the savings realized
- Each control has an AICost governance control number you can cite in audits
8. Worked example: customer support with RAG
A support team answers 60,000 questions a month from product documentation. The designer prices six steps: front door, retrieval, model answer, screening, human review and the cost firewall.
- MCP chain: rag-pipeline, cost-calculator, guardrail-security-cost, hitl-review-cost, cost-firewall
- Levers applied by the engines: prompt caching, model routing for routine questions, batching for non-urgent work
- Policy exported to LiteLLM: routing rules, cache, monthly budget, daily warning and hard stop
- Actuals: LiteLLM spend logs and the cloud bill flow back; drift over 10% is flagged
- Governance: grounded answers only, hallucination checks, PII redaction, audit log
Try it on your own workflow
Get a free MCP key, or price a workflow in the designer first.