Stateless Auth: Managing JWT Blacklisting at Scale

Stateless Auth: JWT Revocation

Mental Model

Connecting isolated components into a resilient, scalable, and observable distributed web.

JWTs are often marketed as "stateless authentication", but the first real security requirement (logout, account disable, token theft response) immediately introduces state.

If you do not design revocation explicitly, you either accept long exposure windows or degrade user experience with aggressive token expiry.

Why revocation matters

graph LR
    Producer[Producer Service] -->|Publish Event| Kafka[Kafka / Event Bus]
    Kafka -->|Consume| Consumer1[Consumer Group A]
    Kafka -->|Consume| Consumer2[Consumer Group B]
    Consumer1 --> DB1[(Primary DB)]
    Consumer2 --> Cache[(Redis)]

Common revocation scenarios:

• user clicks logout on shared/public device
• account is disabled by admin
• password reset after suspected compromise
• refresh token family is invalidated due to theft detection

Without revocation, a valid unexpired JWT continues to grant access.

The core model: token identity + deny list

Issue access tokens with a unique jti (JWT ID). On each authenticated request:

1. validate signature, issuer, audience, and expiry
2. check if jti is revoked in low-latency store (usually Redis)
3. reject if revoked

This preserves scalable verification while enabling immediate invalidation.

Data structures for revocation in Redis

A practical key model:

• key: revoked:jti:<id>
• value: reason or metadata
• TTL: token remaining lifetime (exp - now)

Auto-expiry keeps memory bounded.

For account-wide invalidate:

• key: user:revoked_after:<user_id> timestamp
• reject token if iat < revoked_after

This avoids writing millions of per-token entries in compromise events.

Choosing token lifetime strategy

Security and performance trade off here:

• short-lived access token (5-15 min) reduces blast radius
• refresh token rotation handles session continuity
• revocation checks still needed for immediate logout/high-risk events

Long-lived access tokens without revocation checks are operationally risky.

Gateway integration pattern

Place revocation check in API gateway or shared auth middleware so every service does not reinvent logic.

Flow:

1. parse and verify JWT
2. check local cache for recently seen non-revoked/revoked JTIs
3. fallback to Redis lookup
4. attach auth context to upstream request

Use tiny in-process caches to reduce Redis read pressure on hot tokens.

Preventing Redis from becoming bottleneck

At high RPS, naive per-request Redis lookups can be expensive.

Mitigations:

• cache positive/negative revocation checks briefly (seconds)
• use Redis cluster with key hashing by jti
• pipeline lookups where middleware handles batched traffic
• monitor miss ratio and p99 lookup latency

Fail-open vs fail-closed behavior must be explicit and risk-based.

Fail-open vs fail-closed policy

If Redis is unavailable:

• fail-closed is safer (deny requests), but risks broad outage
• fail-open keeps availability, but allows potentially revoked tokens

A common approach:

• fail-closed for admin/financial scopes
• bounded fail-open for low-risk read-only scopes with alerting

Make this a policy decision, not accidental behavior.

Event-driven revocation propagation

In microservices, revocation events should be published:

• TOKEN_REVOKED
• USER_REVOKED_AFTER_UPDATED

Consumers can warm local caches proactively. This reduces consistency lag during incidents and supports edge deployments.

Handling logout correctly

Logout should invalidate:

• current access token (jti)
• optionally current refresh token
• optionally all session family refresh tokens (for "logout all devices")

Do not rely on client-side token deletion only.

Security hardening essentials

• sign with strong asymmetric keys where possible
• enforce aud, iss, and clock skew checks
• rotate signing keys with kid
• store minimal claims (avoid sensitive data in JWT payload)
• detect replay with jti + device/session context when needed

JWT is transport format, not complete security architecture.

Observability and operations

Track:

• revocation check latency (p50, p95, p99)
• revoked-token access attempts
• Redis availability and timeout rates
• auth decision split (valid/revoked/invalid/expired)
• false rejection incidents

Security controls without observability are hard to trust.

Common implementation mistakes

• no jti claim, making selective revocation hard
• storing revocations forever (unbounded memory growth)
• not syncing clock assumptions (iat/exp) across services
• revoking only refresh tokens but not active access tokens
• missing admin emergency "revoke all for tenant/user" path

Reference architecture

• Identity service issues signed JWTs with jti
• API gateway verifies and checks revocation
• Redis stores revocation markers with TTL
• Event bus propagates revocation updates
• Auth dashboard allows targeted and bulk revocation operations

This architecture keeps request path fast while giving security teams immediate control.

Final takeaway

Stateless auth is a scalability optimization, not a security exemption. At scale, robust JWT systems combine short-lived tokens, centralized revocation checks, and operational observability to keep both performance and incident response strong.

Engineering Standard: The "Staff" Perspective

In high-throughput distributed systems, the code we write is often the easiest part. The difficulty lies in how that code interacts with other components in the stack.

1. Data Integrity and The "P" in CAP

Whenever you are dealing with state (Databases, Caches, or In-memory stores), you must account for Network Partitions. In a standard Java microservice, we often choose Availability (AP) by using Eventual Consistency patterns. However, for financial ledgers, we must enforce Strong Consistency (CP), which usually involves distributed locks (Redis Redlock or Zookeeper) or a strictly linearizable sequence.

2. The Observability Pillar

Writing logic without observability is like flying a plane without a dashboard. Every production service must implement:

• Tracing (OpenTelemetry): Track a single request across 50 microservices.
• Metrics (Prometheus): Monitor Heap usage, Thread saturation, and P99 latencies.
• Structured Logging (ELK/Splunk): Never log raw strings; use JSON so you can query logs like a database.

3. Production Incident Prevention

To survive a 3:00 AM incident, we use:

• Circuit Breakers: Stop the bleeding if a downstream service is down.
• Bulkheads: Isolate thread pools so one failing endpoint doesn't crash the entire app.
• Retries with Exponential Backoff: Avoid the "Thundering Herd" problem when a service comes back online.

Critical Interview Nuance

When an interviewer asks you about this topic, don't just explain the code. Explain the Trade-offs. A Staff Engineer is someone who knows that every architectural decision is a choice between two "bad" outcomes. You are picking the one that aligns with the business goal.

Performance Checklist for High-Load Systems:

1. Minimize Object Creation: Use primitive arrays and reusable buffers.
2. Batching: Group 1,000 small writes into 1 large batch to save I/O cycles.
3. Async Processing: If the user doesn't need the result immediately, move it to a Message Queue (Kafka/SQS).

Technical Trade-offs: Messaging Systems

Key Takeaways

• user clicks logout on shared/public device
• account is disabled by admin
• password reset after suspected compromise

Read Next

• Chaos Engineering for Data Infrastructure: Testing Distributed Resilience
• System Design: Building a Metrics Platform Like Prometheus
• Production Incident Playbooks: Debugging Latency, Errors, and Traffic Spikes

Verbal Interview Script

Interviewer: "How would you ensure high availability and fault tolerance for this specific architecture?"

Candidate: "To achieve 'Five Nines' (99.999%) availability, we must eliminate all Single Points of Failure (SPOF). I would deploy the API Gateway and stateless microservices across multiple Availability Zones (AZs) behind an active-active load balancer. For the data layer, I would use asynchronous replication to a read-replica in a different region for disaster recovery. Furthermore, it's not enough to just deploy redundantly; we must protect the system from cascading failures. I would implement strict timeouts, retry mechanisms with exponential backoff and jitter, and Circuit Breakers (using a library like Resilience4j) on all synchronous network calls between microservices."