Skip to main content

​
Overview

The adaptive middleware dynamically adjusts rate limits based on system metrics like latency, error rate, and resource utilization. Use it when you need:
  • Auto-scaling rate limits
  • Load-based throttling
  • Self-tuning protection

​
Installation

import "github.com/go-mizu/mizu/middlewares/adaptive"

​
Quick Start

app := mizu.New()

// Adaptive limiting based on latency
app.Use(adaptive.New(adaptive.Options{
    MinRate:         10,
    MaxRate:         1000,
    TargetLatency:   100 * time.Millisecond,
}))

​
Configuration

​
Options

OptionTypeDefaultDescription
MinRateint10Minimum requests/sec
MaxRateint1000Maximum requests/sec
TargetLatencytime.Duration100msTarget response time
TargetErrorRatefloat640.01Target error rate (1%)
Windowtime.Duration10sMeasurement window
AdjustIntervaltime.Duration1sRate adjustment interval

​
Examples

​
Latency-Based

app.Use(adaptive.New(adaptive.Options{
    MinRate:       50,
    MaxRate:       500,
    TargetLatency: 200 * time.Millisecond,
}))

​
Error Rate-Based

app.Use(adaptive.New(adaptive.Options{
    MinRate:         10,
    MaxRate:         1000,
    TargetErrorRate: 0.05, // 5% error rate
}))

​
Combined Metrics

app.Use(adaptive.New(adaptive.Options{
    MinRate:         50,
    MaxRate:         500,
    TargetLatency:   100 * time.Millisecond,
    TargetErrorRate: 0.01,
    Window:          30 * time.Second,
    AdjustInterval:  5 * time.Second,
}))

​
How It Works

  1. Measure latency and error rate in sliding window
  2. If metrics exceed targets, decrease rate
  3. If metrics are good, gradually increase rate
  4. Rate stays within min/max bounds

​
API Reference

​
Functions

// New creates adaptive rate limiter
func New(opts Options) mizu.Middleware

// GetCurrentRate returns current rate limit
func GetCurrentRate(c *mizu.Ctx) int

​
Technical Details

​
Architecture

The adaptive middleware uses a token bucket algorithm with dynamic rate adjustment:
  • Token Bucket: Implements a refillable token bucket where tokens are added at the current rate per second
  • Atomic Operations: Uses sync/atomic for lock-free access to metrics (request count, error count, latency)
  • Background Adjustment: Runs a goroutine that periodically adjusts the rate based on collected metrics

​
Rate Adjustment Algorithm

The middleware adjusts rates based on three conditions:
  1. High Error Rate: If error rate > threshold, reduce rate by 20%
  2. High Latency: If average latency > target, reduce rate by 10%
  3. Good Performance: If latency < target/2 AND error rate < threshold/2, increase rate by 10%
All adjustments are clamped to the configured MinRate and MaxRate bounds.

​
Metrics Collection

The middleware tracks:
  • Request Count: Total requests in current adjustment interval
  • Error Count: Requests that returned errors
  • Total Latency: Cumulative response time in nanoseconds
  • Current Rate: Dynamically adjusted rate limit
Metrics are atomically swapped and reset on each adjustment interval to prevent data races.

​
Token Refill Mechanism

Tokens are refilled based on elapsed time: 
refill = (elapsed_nanoseconds * current_rate) / 1_second_in_nanoseconds
Tokens are capped at the current rate to prevent unbounded accumulation.

​
Best Practices

  • Start with conservative limits
  • Set reasonable target latency
  • Monitor rate adjustments
  • Use with circuit breaker for full protection

​
Testing

The adaptive middleware includes comprehensive test coverage for all features and edge cases:
Test CaseDescriptionExpected Behavior
TestNewBasic middleware creationMiddleware processes request successfully
TestLimiter_RateLimitedRate limiting enforcementFirst request succeeds, immediate second request is blocked with 429
TestLimiter_CurrentRateCurrent rate retrievalReturns the configured initial rate
TestLimiter_StatsStatistics retrievalReturns current rate, min/max bounds, and request/error counts
TestSimpleSimple preset configurationMiddleware works with default simple settings
TestHighThroughputHigh throughput presetHandles 100 concurrent requests with some successes
TestConservativeConservative presetMiddleware works with conservative settings
TestRetryAfterHeaderRetry-After header on rate limitReturns “Retry-After: 1” header when rate limited
TestCustomErrorHandlerCustom error handlerUses custom JSON error response instead of default
TestNewLimiter_DefaultsDefault value handlingNegative values are replaced with sensible defaults
TestLimiter_AdjustAdjustment with no requestsRate remains unchanged when no requests are made
TestLimiter_AdjustHighErrorRateHigh error rate adjustmentRate decreases when error rate exceeds threshold
TestLimiter_AdjustHighLatencyHigh latency adjustmentRate decreases when latency exceeds target
TestLimiter_AdjustLowLatencyAndErrorsGood performance adjustmentRate increases when latency is low and errors are minimal
TestLimiter_AdjustClampToMinMinimum rate clampingRate cannot decrease below configured minimum
TestLimiter_AdjustClampToMaxMaximum rate clampingRate cannot increase above configured maximum
TestLimiter_AllowTokenRefillToken refill mechanismTokens are refilled over time after exhaustion
TestLimiter_AllowTokenCapToken accumulation limitTokens are capped at current rate, not unlimited
TestLimiter_ErrorCountingError metric trackingRequests are counted correctly in statistics