Idempotency means an operation performed several times leaves the same state as performing it once. Setting an address to a particular value is idempotent — performed ten times the result is the same. Adding an amount to an account is not.
Why retries are unavoidable
In a distributed system a request can arrive and its response be lost. The sender then does not know whether the operation was performed. It has two options: not retry and risk nothing having happened, or retry and risk it happening twice. Without idempotency both are dangerous. With idempotency the retry is risk-free and the problem disappears.
The idempotency key
For operations that are not naturally idempotent, the usual route is a key supplied by the caller. The receiving system remembers which keys it has already processed. If the same request arrives again with the same key, it is not executed again; the stored result is returned. What matters is that the key comes from the caller and stays identical on retry — a newly generated key renders the mechanism useless.
Where it matters in practice
Three areas dominate. Payments, where a double charge causes immediate harm. Webhook processing, where repeat delivery is the norm. And data imports, where an aborted and restarted run must not leave a half or duplicated state. In all three, idempotency is not an improvement but the condition for reliability.
How it differs from deduplication
Idempotency and removing duplicates afterwards are occasionally confused. Idempotency prevents a duplicate arising at all. Deduplication cleans up afterwards. The second route is inferior, because between creation and clean-up a state exists in which the duplicate took effect — the second email is sent before anyone removes it.
Where it gets difficult
Difficult are operations with side effects outside your own system: a message sent, a call to a third party. Those cannot be undone, so the check must happen before execution rather than after. Also difficult is how long to retain keys: too short, and a late retry executes again; too long, and storage grows without bound.
Practical consequence
For every interface that changes state, the question of what happens on a second call belongs answered. If the answer is not the same as on the first, an idempotency key is needed — before the first duplicate appears in production.