In a demo, a prompt that looks good on three hand-picked examples is enough. In a product it has to work reliably across thousands of different inputs, be maintained by people who did not write it, and survive model changes. Prompt engineering for products therefore has little to do with clever wording and a lot to do with structure, versioning and tests.
Prompts are code
A prompt in a product deserves the same care as any other program logic. It lives in version control, not in a database row someone can edit in an admin panel. Changes go through review and are checked against an evaluation set. Every model request logs which prompt version it used. Only then can you trace why a feature's behaviour changed.
Structure of a system prompt
A fixed structure works well: first the role and the task in a few clear sentences, then the context the model needs, then the rules, then the expected output format, finally examples. Rules should be phrased positively — what to do — and explained where the behaviour is not self-evident. Long lists of prohibitions tend to make models uncertain and evasive.
Stable parts go first, variable parts last. That eases maintenance and, with many providers, allows the stable part to be cached, lowering cost and latency.
Separate inputs from instructions
Content from users, documents or emails should be clearly marked as data in the prompt, for example in clearly named sections. The model must know that text inside that section is to be processed and contains no instructions to follow. This separation reduces the risk of prompt injection but does not replace structural safeguards: what a model must not do, it should not technically be able to do.
Enforce outputs instead of asking for them
When code processes the result further, it should be structured — and not merely because the prompt politely asks. Many providers support structured outputs against a given schema or tool calls with defined parameters. Your own code additionally validates every output against the schema and handles deviations in a controlled way: retry, fall back to a simpler path, or show an understandable error.
Examples, used carefully
Examples in a prompt are effective because they show format and tone instead of describing them. They have a side effect though: models tend to copy content and phrasing from the examples. A few deliberately different examples covering the range of expected cases work best, ideally with obviously fictional content.
One prompt per task
A single, ever-growing prompt meant to cover every case becomes more fragile with each addition. Several specialised prompts for clearly separated tasks, connected by code, are better: one step classifies the request, a second answers it with the matching prompt. That makes each part more understandable and testable, and allows a smaller model for simple sub-tasks.
Surviving model changes
Prompts tuned tightly to one model's quirks often work worse with the next. Clear, structured prompts with explicit requirements are more robust than wording based on chance observations. Before every switch the evaluation set runs; adjustments are made where it reveals weaknesses.
Language and region
For German-language products the prompt should state what Swiss, German or Austrian users expect: «ss» spelling for Switzerland, form of address, date and number formats, technical terms. Such rules should be maintained centrally and included in all prompts rather than written into each one separately.
In short
- Version, review and test prompts against a data set.
- Fixed structure, stable parts first, data clearly marked.
- Enforce structured outputs and validate them in code.
- Prefer several specialised prompts over one catch-all.
- Maintain language and format rules centrally.
Innopulse treats prompts as code in its own products, and builds them the same way for clients.