Production readiness review for LLM features
Product teams and CTOs bringing an AI feature — chat, RAG search, extraction or an agent — into an existing product.
Updated: 2026-10
The review checks an LLM feature before launch for what demos do not show: prompt injection attacks, quality measured against a fixed evaluation set, data flows and contracts with model providers, cost per use, monitoring and AI Act classification. The result is a prioritised action plan.
An LLM feature almost always works in the demo. In production, real users, real data and real costs meet a probabilistic system. Typical problems are answers without source grounding, tools with overly broad permissions, customer data at providers without a suitable contract and costs nobody measures per tenant.
Many teams only notice after launch, when a customer complains about wrong information or the model provider’s invoice comes as a surprise. A review before launch is considerably cheaper than correcting afterwards.
What you get
Threat model
Analysis of attack surfaces, in particular direct and indirect prompt injection, tool permissions, tenant boundaries and data leakage through outputs, with concrete architectural measures.
Evaluation set and baseline
A versioned set of realistic and hard cases with scoring criteria and a first measurement against which future changes are checked.
Data flow and contracts
An overview of which data goes to which provider, where it is processed and whether training is excluded, with notes on processing agreements and disclosure abroad.
Cost and abuse concept
Measurement of cost per request, quotas per tenant, rate limits and a kill switch for the cost case.
Monitoring and fallbacks
Metrics, logging to the permitted extent and defined behaviour on errors and provider outages.
AI Act classification
Role as provider or deployer, risk class and implementation of transparency obligations, such as the chatbot notice.
How it runs
- 1
Introduction
Walk through use case, architecture, data flows and error costs with the team.
- 2
Analysis
Review code, prompts, tools and configuration; create the threat model and evaluation set.
- 3
Measurement
Run the evaluation set and interpret results with the team.
- 4
Action plan
Prioritised plan with an effort estimate per measure and optional support with implementation.
Price framing
Fixed price by scope. The range depends on the number and complexity of AI features and is set in a short initial conversation.
Indicative, not a binding quote — the frame is confirmed during scoping.
What drives the price? → Cost guide
Parent service: SaaS Product Development
FAQ
Which models and providers does the review cover?
All common providers and self-hosted models. The principles — architecture against prompt injection, evaluation, cost control — are provider-independent.
Is this a penetration test?
No. The review covers architecture and targeted prompt injection tests. A comprehensive application penetration test is a separate service from specialised providers and can be a useful complement.
What if the feature is already live?
Then the review is just as useful; the action plan prioritises what to tackle immediately and what fits the normal development rhythm.
Do we keep the evaluation set?
Yes. The set and the evaluation scripts are yours and can be integrated into your pipeline.
Production readiness review for LLM features
Product teams and CTOs bringing an AI feature — chat, RAG search, extraction or an agent — into an existing product.