When your backend goes down for maintenance or encounters a transient database lock during webhook processing, webhook providers don't just give up. They retry. And if your service isn't built to handle retries properly, you risk double-charging customers, provisioning duplicate licenses, or getting taken down by retry storms.
Understanding the retry schedules of major webhook providers — and designing your consumer endpoints to be idempotent — is the difference between an unnoticeable network blip and a severe production incident.
How Major Webhook Providers Retry
Each platform uses a slightly different backoff strategy:
- Stripe: Retries up to 3 days (roughly 16 attempts) in live mode. Stripe uses exponential backoff starting around 1 hour after the first failure and lengthening to several hours. Stripe separates live mode from test mode (in test mode, it only retries 3 times over a few hours).
- Shopify: Retries 19 times over a 48-hour period with increasing backoff intervals. If an endpoint fails continuously for 48 hours, Shopify disables the webhook subscription entirely and alerts the store owner.
-
GitHub: Retries automatically for up to 24 hours. GitHub includes the header
X-GitHub-Deliverywhich remains identical across all retries for the same event. - Svix / Modern Platforms: Modern webhook delivery services implement Exponential Backoff with Full Jitter (e.g. initial attempt + random jitter between 5s, 15s, 1m, 5m, 30m, 2h, 6h, 12h, 24h).
The Anatomy of a Retry Storm
Imagine your webhook endpoint experiences a 10-minute database slowdown. During those 10 minutes, 1,000 webhook events fail with HTTP 504 Gateway Timeout:
The Cascade Effect:
- 1,000 webhook events fail at 12:00.
- Without jitter, all 1,000 providers retry simultaneously at 12:05.
- Your already-recovering database gets hit with 1,000 retries plus incoming new events.
- The database crashes again, pushing retries into an even larger synchronized wave.
Designing Idempotent Webhook Consumers
Because retries can occur even when your server actually processed the request (e.g., if the HTTP 200 response got dropped on the wire before reaching the provider), your handlers must be idempotent:
1. Store the Provider's Unique Event ID
Every major provider assigns a unique event ID to each event payload:
- Stripe:
event.id(e.g.evt_1Nz4...) - GitHub:
X-GitHub-Deliveryheader (UUID) - Shopify:
X-Shopify-Webhook-Idheader
Before processing an event, check your database if this event ID has already been recorded:
// In your webhook consumer handler
func HandleWebhook(w http.ResponseWriter, r *http.Request) {
eventID := r.Header.Get("X-GitHub-Delivery")
// 1. Check if already processed
processed, err := db.HasProcessedEvent(r.Context(), eventID)
if err != nil {
http.Error(w, "internal error", http.StatusInternalServerError)
return
}
if processed {
// Acknowledge immediately with 200 OK without re-running actions
w.WriteHeader(http.StatusOK)
return
}
// 2. Process the event inside an atomic transaction
if err := db.ProcessOrderAndRecordEvent(r.Context(), eventID, payload); err != nil {
http.Error(w, "processing failed", http.StatusInternalServerError)
return
}
w.WriteHeader(http.StatusOK)
}
2. Acknowledge Fast (Under 5 Seconds)
Most webhook providers timeout after 5 to 15 seconds. If your webhook handler initiates long-running tasks (like PDF generation or sending transactional emails), do not block the HTTP response.
Validate the signature, persist the event to an internal job queue (or Redis/Postgres queue), and reply 200 OK immediately.
Testing Retry Behavior Safely
The biggest blind spot in webhook development is assuming that retries only happen when your code crashes.
With SnapHook, you can test how your upstream provider behaves when your endpoint is unavailable:
# Configure a synthetic 500 error on your capture session
curl -X PUT https://snaphook.site/api/{slug}/response \
-H 'Content-Type: application/json' \
-d '{"status":500,"body":"simulated upstream outage"}'
Trigger an event in Stripe or GitHub, and observe the retry cadence in the live inspector:
- Does each retry keep the same delivery ID?
- Does the timestamp header update or stay constant?
- Does your consumer's retry logic handle the subsequent delivery gracefully once you restore the response to
200 OK?
Rule of Thumb:
Never deploy a webhook integration to production without intentionally testing a failure state first. A webhook integration that works on the happy path is only half tested.