Holding 99.9% sync uptime across three review platforms

The problem

Everything downstream — the AI response agents, the customer dashboards, the reporting — assumes reviews are present and current. Three upstream platforms provide them, and no two behave the same way.

One pushes webhooks that occasionally arrive twice and occasionally not at all. One offers polling with a rate limit strict enough that a naive full sync exhausts the quota before finishing. One changes response shapes without notice. All three have outages that don't coincide with each other.

The old integration treated a missed event as an event that didn't happen. That assumption is what made "is this dashboard current?" an unanswerable question.

Approach

The design rule was that no single delivery mechanism is trusted to be complete.

Webhooks for latency, polling for truth. Where a platform pushes events, we take them for freshness. But a reconciliation sweep runs on a schedule regardless, comparing what we hold against what the platform reports. A webhook that never arrived is found by the sweep rather than never noticed.

Idempotency on the platform's own review id. Duplicate webhooks are a non-event. This removed the entire class of double-response bugs, which matter disproportionately here — the visible failure is a customer receiving two different replies to one review.

Per-platform backoff, shared retry semantics. Each integration declares its own rate limits and retry policy, but they all fail into the same queue with the same visibility:

# Each platform gets its own budget; the retry contract is shared.
GOOGLE = PlatformConfig(
    rate_limit=RateLimit(per_minute=60),
    retry=Retry(attempts=5, backoff="exponential", jitter=True),
    # Yelp and Facebook differ in limits, not in how failure is handled.
)
 
async def sync_location(platform: PlatformConfig, location_id: str) -> SyncResult:
    async with platform.limiter:
        try:
            page = await platform.fetch_reviews(location_id)
        except RateLimited as exc:
            # Not an error — the platform told us when to come back.
            raise Retryable(after=exc.retry_after) from exc
 
        # onConflictDoNothing on the platform's review id: replays are free.
        return await store.upsert_reviews(page.reviews)

Making it observable

Every sync carries a trace from the upstream call through parsing to the final write. "Why is this location's count wrong?" resolves to a span with a timestamp instead of an afternoon of investigation. Uptime became a measured number precisely because failures stopped being invisible.

Outcome

99.9% data-sync uptime across all three platforms. The operational change that mattered more than the number: staleness became a property the system reports about itself, so the AI agents and dashboards downstream could stop assuming their inputs were complete and start knowing.