Assessment · Governance · Test and control

10Answer quality and grounding

Are the answers right, and grounded in our own data?

Answers checked for made-up facts and kept to the data you give the model.

Stakeholder interviewsAutomatedUses your existing assessments 2 controls 3 engines

How we collect

  • Stakeholder interviewsCustomer and consultant set acceptance thresholds before the final runRuns today
  • AutomatedGolden set checked for edge, adversarial and refusal casesRuns today
  • AutomatedYour system runs the evaluation pack; AICost scores the resultsNeeds your export or setup
  • Uses your existing assessmentsResults from your eval tools (LangSmith, Braintrust, Arize, Promptfoo)Needs your export or setup

Status labels show what runs today, what needs an export or setup from you, and what is planned.

What you get

  • Pass or fail per consequence class, never one universal percentage
  • Which steps should be AI, which deterministic, which human
  • Release regressions

Questions we ask

  • What answer would be harmful if wrong?

We can start from

Eval results and test suites you already run

Data sources

SourceWhat we readStatus
AI gateway telemetryLiteLLM, Cloudflare AI Gateway or AICost CostWall logs: cost per team, workload, agent and outcome.Partly automated
QuestionnaireThe short interview for what data cannot show: purpose, owners, decisions AI informs, regulations in scope.Runs today

Built on governance engine modules

P12 PlannedQuality, Hallucination & Evaluation

How wrong is acceptable, per consequence class, and how do you know?

Security & Assurance

AICost governance controls in this module

AICost-GovControl601Runs today
Testing before release
What it is

An agreed test set and pass mark for each AI use, run before release and after every model or prompt change.

Why it is needed

Without a gate, quality drops show up as customer complaints.

Why enterprises need it

Product and risk teams need evidence that the system met the bar when it shipped.

What you get:Releases blocked below the pass markChanges retested automaticallyEvidence for audits
Run by
AI Quality and Acceptance Gate engineQuality Testing engine
Data
AI gateway telemetry, Questionnaire
Evidence produced
Acceptance record: system, test set, pass mark, result, date, approver.
Maps to
NIST AI RMF: Measure; ISO/IEC 42001: verification and validation; EU AI Act: accuracy and robustness
AICost-GovControl602Partly automated
Wrong answers and AI-generated content
What it is

Controls for inaccurate AI output: answer checking, sourcing, labeling, and who signs off before AI content is used.

Why it is needed

AI states errors with confidence. Wrong figures in a contract or a customer reply have real cost.

Why enterprises need it

Teams need clear sign-off rules for AI content in contracts, reports and customer messages.

What you get:Fewer wrong answers reaching customersClear sign-off for high-stakes contentLabels where rules require them
Run by
AI Quality and Acceptance Gate engineAnswer Checking engineSafety Screening engine
Data
AI gateway telemetry, Questionnaire
Evidence produced
Output control table: use, checking method, sign-off role, label required, error rate.
Maps to
EU AI Act: Article 50 transparency; NIST AI 600-1 generative AI profile

Frameworks

NIST AI RMF and NIST AI 600-1Govern, Map, Measure and Manage activities, including the generative AI risks
EU AI ActCategory and role per system, prohibited uses, high-risk duties, transparency, AI literacy, incident reporting

Stakeholders

Head of AIQABusiness owners
← 9. Vendor and model due diligence All modules 11. Security and red teaming →
📖 Data sources & methodology 172 text models · 9 embeddings · 44 vision · 62 audio · 8 vector DBs across 10 vendor pages · last verified 2026-10-18

Methodology

  • All prices are USD per 1 million tokens, current as of 2026-10-18.
  • Vendor-published values have no mark. Inferred/extrapolated values are marked with * and listed below.
  • Batch API discounts are 50% off standard rates across providers that offer Batch mode.
  • Prompt caching discounts vary by provider (typically 80-90% off cached input tokens).
  • Regional data-residency surcharges (Anthropic 1.1x, OpenAI 1.1x, Google regional tiers) are NOT included in base rates.
  • Long-context pricing tiers apply when input exceeds model threshold.
  • Embedding prices are input-only (no output tokens generated).

Primary sources

Last-verified date is the most recent successful daily snapshot (aicost_pricing_snapshots) or, when no snapshot exists yet, the latest successful crawler run (aicost_crawler_runs). 10 of 10 vendors are currently verified. Aggregator services (TokenCost, AI Pricing Guru, etc.) are not listed.

Anthropic
2026-10-18
https://www.anthropic.com/pricing
Daily snapshot since Sep 2023 · 704 days captured
Anthropic Docs
2026-10-18
https://platform.claude.com/docs/en/about-claude/pricing
Daily snapshot since Sep 2023 · 704 days captured
OpenAI
2026-10-18
https://openai.com/api/pricing/
Daily snapshot since Sep 2023 · 705 days captured
Google AI
2026-10-18
https://ai.google.dev/gemini-api/docs/pricing
Daily snapshot since Dec 2023 · 680 days captured
Google Vertex
2026-10-18
https://cloud.google.com/vertex-ai/generative-ai/pricing
Daily snapshot since Dec 2023 · 680 days captured
DeepSeek
2026-10-18
https://api-docs.deepseek.com/quick_start/pricing
Daily snapshot since May 2024 · 619 days captured
xAI
2026-10-18
https://x.ai/api
Daily snapshot since Nov 2024 · 537 days captured
Mistral
2026-10-18
https://mistral.ai/pricing
Daily snapshot since Dec 2023 · 678 days captured
Cohere
2026-10-18
https://cohere.com/pricing
Daily snapshot since Sep 2023 · 704 days captured
Voyage AI
2026-10-18
https://docs.voyageai.com/docs/pricing

Inferred values (marked with * in calculator tables)

Derived from industry conventions, not directly published by the vendor. Typical conventions: cached input = 10% of base (90% off), Batch API = 50% of base (50% off).

Vendor / Model Field Why it’s inferred
Anthropic — Claude Sonnet 4.6 cachedInput Derived at 10% of input rate — Anthropic publishes 90% cache-hit discount on this tier.
Anthropic — Claude Sonnet 4.5 cachedInput Derived at 10% of input rate; same 90% cache-hit convention as Sonnet 4.6.
Anthropic — Claude Sonnet 4.5 batchInput Derived at 50% of standard input — Anthropic documents uniform 50% Batch discount.
Anthropic — Claude Sonnet 4.5 batchOutput Derived at 50% of standard output — Anthropic documents uniform 50% Batch discount.
Anthropic — Claude Haiku 4.5 cachedInput Derived at 10% of input rate — Anthropic 90% cache-hit discount convention.
OpenAI — GPT-5.4 Mini cachedInput Derived at 10% of input — OpenAI documents automatic 90% discount on cache hits across GPT-5.x tier.
OpenAI — GPT-5.4 Nano cachedInput Derived at 10% of input — OpenAI 90% cache-hit convention.
OpenAI — GPT-5.4 Nano batchInput Derived at 50% of input — OpenAI Batch API uniform 50% discount.
OpenAI — GPT-5.4 Nano batchOutput Derived at 50% of output — OpenAI Batch API uniform 50% discount.
OpenAI — GPT-5.4 Pro cachedInput Derived at 10% of input — OpenAI 90% cache-hit convention.
OpenAI — GPT-5.4 Pro batchInput Derived at 50% of input — OpenAI Batch API uniform 50% discount.
OpenAI — GPT-5.4 Pro batchOutput Derived at 50% of output — OpenAI Batch API uniform 50% discount.
OpenAI — GPT-5.2 cachedInput Derived at 10% of input; no residency uplift.
OpenAI — GPT-5.2 batchInput Derived at 50% of input.
OpenAI — GPT-5.2 batchOutput Derived at 50% of output.
OpenAI — GPT-5 cachedInput Derived at 10% of input.
OpenAI — GPT-5 batchInput Derived at 50% of input.
OpenAI — GPT-5 batchOutput Derived at 50% of output.
OpenAI — GPT-5.5 Pro cachedInput Derived at 10% of input — OpenAI does not publish a cached rate for *-pro models; using the family convention.
OpenAI — GPT-5.5 Pro batchInput Derived at 50% of input.
OpenAI — GPT-5.5 Pro batchOutput Derived at 50% of output.
OpenAI — GPT-5.2 Pro cachedInput Derived at 10% of input — pro-tier convention.
OpenAI — GPT-5.2 Pro batchInput Derived at 50% of input.
OpenAI — GPT-5.2 Pro batchOutput Derived at 50% of output.
OpenAI — GPT-5.1 batchInput Derived at 50% of input.
OpenAI — GPT-5.1 batchOutput Derived at 50% of output.
OpenAI — GPT-5 Pro batchInput Derived at 50% of input.
OpenAI — GPT-5 Pro batchOutput Derived at 50% of output.
OpenAI — GPT-5 Nano cachedInput Derived at 10% of input.
OpenAI — GPT-5 Nano batchInput Derived at 50% of input.
OpenAI — GPT-5 Nano batchOutput Derived at 50% of output.
Google — Gemini 3 Flash cachedInput Derived at 10% of input — Google caching discount convention ~90%.
Google — Gemini 3.1 Flash-Lite cachedInput Derived at 10% of input — Google caching convention.
Google — Gemini 3.1 Flash-Lite batchInput Derived at 50% of input — Google Batch API uniform 50% discount.
Google — Gemini 3.1 Flash-Lite batchOutput Derived at 50% of output — Google Batch API uniform 50% discount.
Google — Gemini 2.5 Pro cachedInput Derived at 10% of input.
Google — Gemini 2.5 Flash cachedInput Derived at 10% of input.
Google — Gemini 2.5 Flash-Lite cachedInput Derived at 10% of input — Google caching convention.
Google — Gemini 2.5 Flash-Lite batchInput Derived at 50% of input — Google Batch API uniform 50% discount.
Google — Gemini 2.5 Flash-Lite batchOutput Derived at 50% of output — Google Batch API uniform 50% discount.
xAI — Grok 4 (legacy) cachedInput Extrapolated at 25% of base.

Pricing is cross-verified against the LiteLLM community registry when available. Daily snapshots are kept in aicost_pricing_snapshots; every change is logged to aicost_price_changelog with old & new values for full audit trail. Read the full methodology →