System Architecture & Design

distlimit was engineered from the ground up to address performance bottlenecks, thundering herd risks, and global mutex contention common in traditional rate-limiting libraries.

---

High-Level Architecture Diagram

flowchart TD
    Req[HTTP or gRPC Incoming Request] --> Shield[Trusted Proxies Check]
    Shield --> Limiter[distlimit Limiter]

    subgraph CoreEngine [Core Execution Engine]
        direction LR
        Limiter --> Alg[Pluggable Algorithms]
        Limiter --> Driver[Storage Drivers]
    end

    CoreEngine --> Observer[metrics Observer Engine]

---

Core Component Breakdown

1. 64-Sharded Concurrent Memory Map

Standard Go maps protected by a single sync.RWMutex experience severe lock contention under heavy multi-threaded workloads.

distlimit solves this by partitioning key storage across 64 independent shards:

type Driver struct {
    shards [64]*shard
}

• Shard Indexing: Key placement is calculated via fnv32a hashing: shardIndex = hash(key) % 64.
• Lock Granularity: Each shard maintains its own sync.RWMutex. Concurrent requests targeting different keys hit distinct shards in parallel, eliminating global lock bottlenecks.
• GC Pause Elimination: Stored entry structures are optimized for in-memory reusability, keeping allocation at 0 B/op.

---

2. Redis Cluster Hash Tags {} & Atomic Lua Scripts

Distributed rate limiting requires atomic state evaluation across Redis nodes to prevent race conditions.

Cluster Hash Tags {}

When operating inside a Redis Cluster, operations affecting multiple keys must target the same hash slot to avoid CROSSSLOT errors. distlimit wraps all Redis key names in hash tags: distlimit:{key}.

Redis calculates the hash slot using only the characters inside {}. This guarantees all operations for a given rate-limiting key execute on the same node cluster slot.

Atomic Lua Script Execution

All algorithm state transitions (token refills, timestamp purging, counter increments) are written in optimized Lua scripts. Redis executes these Lua scripts atomically in a single thread, guaranteeing zero race conditions between distributed application nodes.

---

3. Dual-Tier Hybrid Driver & Half-Open Circuit Breaker

The Hybrid Driver layers a primary Redis driver over a secondary In-Memory fallback driver:

flowchart TD
    Req[Incoming Request] --> Hybrid[Hybrid Driver]

    Hybrid -->|Primary OK| Redis[Redis Driver]
    Hybrid -->|Primary Fail| CB[Circuit Breaker - Cool-Off State]

    Redis -->|Error| CB
    CB --> Memory[Failover Memory Driver]

State Transition Lifecycle

1. Normal Flow (Closed): All rate checks execute against Redis Primary.
2. Failover (Open): If Redis returns a network error or connection timeout, the Circuit Breaker opens instantly. Requests fail over to the local 64-sharded In-Memory driver without blocking client requests.
3. Cool-Off Duration: The driver remains in fallback mode for a configurable duration (default: 5 seconds), suppressing failing network calls to Redis.
4. Half-Open Probing: Once the cool-off timer expires, the Hybrid driver enters a Half-Open state. A single goroutine probes Redis using an atomic CompareAndSwap(false, true) CAS lock.
   • If probing succeeds, the Circuit Breaker closes and normal Redis operations resume.
   • If probing fails, the cool-off timer resets. This eliminates Thundering Herd spikes against recovering Redis instances.

---

4. Lock-Free Dynamic Policy Engine

distlimit uses atomic.Pointer[Policy] to store active rate limiting rules:

type Limiter struct {
    policy atomic.Pointer[Policy]
}

When limiter.UpdatePolicy(newLimit, newWindow) is invoked at runtime:

1. A new Policy struct is allocated.
2. policy.Store() performs an atomic pointer swap.

Goroutines executing AllowKey() load the pointer via policy.Load(). This provides zero-downtime, zero-lock-contention limit reloads with $O(1)$ nanosecond execution latency.