Anyone building with language models calculates in tokens. Not in words, not in characters. The context window, maximum answer length, the API provider’s bill and much of the response time depend on this unit. A basic understanding is therefore a prerequisite for any cost planning.
How text becomes tokens
A tokenizer splits text into frequent character sequences. Short, frequent words are often a single token; rare or long words are split into several parts. Numbers, code and special characters behave differently from prose. Every model has its own tokenizer; the same input can yield different token counts with two providers.
German costs more
Many tokenizers are optimised for English-language data. German compounds and inflections are split more often, so the same content in German needs more tokens in many cases. For DACH products this means: cost estimates based on English examples are too low, and the context window fills faster.
Input and output
Providers bill input and output tokens separately, output tokens usually at a higher rate. Output tokens also drive latency more, because they are generated one after another. A long system prompt costs on every call; a long answer costs and takes time. Many providers offer caching for recurring prompt parts, which can substantially reduce input cost and latency.
Managing token consumption
Proven levers: keep system prompts short and stable so caching applies. In RAG, insert only the most relevant sections instead of filling the context; good chunking helps. Limit output length and request structured outputs. Use smaller models for simple tasks and escalate only when needed. And measure consumption per feature and tenant, otherwise nobody knows which feature drives the bill.
Tokens in the business model
For SaaS products with AI features, tokens are variable costs per use. Unlimited AI usage in a flat plan can eat the margin if individual users consume a lot. Many products therefore work with quotas or credits per plan. How to plan this is described in our article on AI cost management.