A customer is charged twice, data is processed twice in an ETL pipeline or hundreds of other scenarios when you have to atomically analyse your code to prevent race conditions.
Most teams use Redis Setnx or have homegrown lease tables and idempotency tables, try to make every endpoint idempotent or check and update in the same transaction or adopt very heavy commitment infra what if you could do it with just postgres ?
I had to do the same, an idempotency table with states with the frontend sending a randomised session id
Made my endpoint that receives payment webhooks always check and update
This was annoying, to say the least to do across inventory, orders and payments
My entire infrastructure depends on postgres select for update and wherever i forgot to put it, say across a direct order or cart order, a race condition lives
So i made Sentinel, zero infra just postgres, it makes any endpoint you wrap it around functionally idempotent, it's caches the result and replays it, handles the entire life cycle from the work being claimed till completion with any errors being surfaced with reconciliation tooling
Uses fencing tokens to throw out stale work, leases and heartbeat for the lifetime of the work and states to track progress.
from sentinel import Sentinel import psycopg
sentinel = Sentinel(get_conn=get_conn)
def process_payment_webhook(charge_id: str, amount: int): result = sentinel.once( key=f"stripe-webhook:{charge_id}", fn=charge_customer, kwargs={"charge_id": charge_id, "amount": amount}, ttl_ms=5000, hard_ttl_ms=30000 )
if result.execution_alive:
# Another worker is actively processing this charge
return {"status": "processing"}
elif result.uncertain:
# Execution failed midway, side effects may have partially applied
# Use reconciliation tooling to inspect and resolve
return {"status": "uncertain", "reconcile": result.reconcile}
else:
# Newly completed or replayed from cache
# Safe to return regardless of how many times this webhook fired
return {"status": "ok", "response": result.response}
Temporal and Airflow assume your tasks are idempotent. Sentinel is what makes them actually idempotent.
"GitHub: github.com/Sreejay-Reddy/Sentinel | pip install sentinel-coordination"