The bill arrived twice. You should still pay it once.
Duplicate messages are normal, even from a bank. Whoever moves the money has to catch them, and sometimes that's a vendor you can't see into.
In the early days of a fintech I worked at, everything was being built for the first time, and I was integrating boleto, a Brazilian payment slip, as a new payment method. Until then it had only carried small volumes. Then a large auction we supported started settling its payments, and reports came in that the auctioneer had been credited twice for some of the slips. It went on for a couple of hours before it stopped.
Something upstream said the same thing twice, and the system that moved the money believed it both times. Senders repeat themselves all the time. Whoever applies the effect has to notice.
The mailman and the bill
Imagine a mailman delivers the same bill twice. Maybe he wasn't sure the first one went through, maybe the sorting office printed two. You still pay it once. You don't need to know why it came twice. You only need to recognise it's the same bill, because it has the same number on it.
That's idempotency: doing something twice has the same effect as doing it once. The duplicate message is the second delivery. The payment is the effect that must happen once. And the bill's number is what makes it possible, an ID that names the operation, not the envelope it came in.
With boleto the analogy barely needs translating. A boleto is a bill.
The balance wasn't ours to fix
Our setup had three parties. The bank collects the payment and lists it in a file. A third-party provider reads those files, keeps each account's balance and settles the money. We, the fintech, received a webhook (an automatic message from the provider) every time a slip changed status.
When we heard about it, I went to the provider's webhook logs. The pattern was there: the same slip, marked as paid, more than once. What I couldn't do was stop it. Those webhooks were status updates. The balance lived with the provider, so the second credit had already happened by the time we heard about it. There was nothing on our side to deduplicate.
So the work was proving it. We spent the night on a call with the provider, walking through the logs slip by slip. The next day they confirmed the cause. The bank's return files, the standard files Brazilian banks use to tell a company which slips were paid, had listed some of the same payments twice. The provider settled every entry it read, and nothing checked whether that slip had already been settled.
I never saw their code, so I can't tell you how the check was missing, only that it was. In total, six figures in dollars were credited twice before it stopped. We were able to roll all of it back.
Even the bank sends twice
It's tempting to stop at the bank. The file had duplicates, so the file was wrong. Maybe it was, but systems that move money are built expecting this. Retries are how they survive: if a sender isn't sure a message arrived, sending it again is safer than staying quiet. A network can drop the acknowledgement and deliver the message. A job can crash after sending and before recording that it sent. Each of those is reasonable on its own, and each produces a duplicate.
Payment providers put this in writing. Their webhook docs commonly say events are delivered "at least once", so the same event can arrive more than once, and tell you to store each event's ID to skip repeats. That's the guarantee you can realistically get from a sender. Exactly once is something the receiver builds, by remembering what it has already done.
The fix is small
In the provider's place, the check is short. Give every settlement a key that names the payment itself, here the bank plus the slip's own number (the "nosso número", the slip's unique ID at the bank), and let the database refuse a second one:
create table settlements (
bank_code text not null,
nosso_numero text not null,
amount_cents bigint not null,
unique (bank_code, nosso_numero)
);
insert into settlements (bank_code, nosso_numero, amount_cents)
values ($1, $2, $3)
on conflict (bank_code, nosso_numero) do nothing;If the insert adds a row, credit the balance in the same transaction. If it adds nothing, the slip was already settled, and the duplicate stops there. For readers who don't write SQL: the database keeps a list of slips it has paid, and it won't write the same one down twice.
The key has to identify the payment, not the message. A file line number or a delivery timestamp changes every time the file is re-sent, so a key built on those lets the duplicate through. And the uniqueness has to live in the database, not in an "if I haven't seen it" check in code, because two copies arriving at the same moment will both pass that check before either is saved.
APIs do the same thing with an Idempotency-Key header: the client sends a unique value with the request, and the server uses it to recognise a retry. Stripe's API works this way, and an IETF draft standardises the header.
It isn't only money
Payments are where I ran into it, but the same bill arrives twice anywhere a message can be retried and the effect is hard to undo:
A checkout times out on a slow connection, the app retries, and the warehouse packs two boxes.
A queue consumer crashes after doing the work but before acknowledging the message, so the next consumer does it again. Most message queues promise "at least once" delivery for this reason.
A deploy overlaps the old version with the new one, the scheduled job runs in both, and customers get the same invoice email twice.
A booking confirmation is retried and the same seat is reserved twice.
A command to a physical device, like opening a gate or starting a machine, is resent after a timeout that hid a success.
The fix has the same shape in each: name the operation with a key, and let whatever applies the effect remember it. What changes is the cost of getting it wrong, from an extra email to a second shipment.
You can't add a check to a system you don't own
Which brings me back to the auction. The fix is a few lines, and we couldn't write them. The check has to live wherever the money moves, and if that's someone else's system, you're trusting a line of code you'll never read.
You don't get out of that by choosing a better vendor. Any provider is a system you can't see into.
Handle it yourself when you can
If the effect happens in your system, put the check there, even when the sender says it won't repeat itself. Their promise is one more thing that can break.
When it happens somewhere you don't control, ask and watch. Before integrating a provider, I now ask how they handle a notice that arrives twice. A good answer names the key. A vague answer is information too. And after the auction we added reconciliation on our side, comparing what the provider reports against what we expected to happen, so a duplicate shows up as a mismatch. It doesn't prevent anything, but it means the next time, we raise the alert ourselves instead of hearing it from someone else.
The question I ask in every integration now is: what happens if this arrives twice, and who checks? It's much cheaper to hear the answer in a meeting than during an auction. Where in your system would a duplicate do damage today, and do you know who's supposed to catch it?
References
FEBRABAN, CNAB 240 layout for collections (return files), as published by Santander.
J. Jena, S. Dalal and D. Tishkin, "The Idempotency-Key HTTP Header Field", IETF Internet-Draft, revision 08, October 2026.
Stripe, "Idempotent requests", API reference.
Comments
Loading comments