Adaptive

​

Overview

• Auto-scaling rate limits
• Load-based throttling
• Self-tuning protection

​

Installation

import "github.com/go-mizu/mizu/middlewares/adaptive"

​

Quick Start

app := mizu.New()

// Adaptive limiting based on latency
app.Use(adaptive.New(adaptive.Options{
    MinRate:         10,
    MaxRate:         1000,
    TargetLatency:   100 * time.Millisecond,
}))

​

Configuration

​

Options

​

Examples

​

Latency-Based

app.Use(adaptive.New(adaptive.Options{
    MinRate:       50,
    MaxRate:       500,
    TargetLatency: 200 * time.Millisecond,
}))

​

Error Rate-Based

app.Use(adaptive.New(adaptive.Options{
    MinRate:         10,
    MaxRate:         1000,
    TargetErrorRate: 0.05, // 5% error rate
}))

​

Combined Metrics

app.Use(adaptive.New(adaptive.Options{
    MinRate:         50,
    MaxRate:         500,
    TargetLatency:   100 * time.Millisecond,
    TargetErrorRate: 0.01,
    Window:          30 * time.Second,
    AdjustInterval:  5 * time.Second,
}))

​

How It Works

1. Measure latency and error rate in sliding window
2. If metrics exceed targets, decrease rate
3. If metrics are good, gradually increase rate
4. Rate stays within min/max bounds

​

API Reference

​

Functions

// New creates adaptive rate limiter
func New(opts Options) mizu.Middleware

// GetCurrentRate returns current rate limit
func GetCurrentRate(c *mizu.Ctx) int

​

Technical Details

​

Architecture

• Token Bucket: Implements a refillable token bucket where tokens are added at the current rate per second
• Atomic Operations: Uses sync/atomic for lock-free access to metrics (request count, error count, latency)
• Background Adjustment: Runs a goroutine that periodically adjusts the rate based on collected metrics

​

Rate Adjustment Algorithm

1. High Error Rate: If error rate > threshold, reduce rate by 20%
2. High Latency: If average latency > target, reduce rate by 10%
3. Good Performance: If latency < target/2 AND error rate < threshold/2, increase rate by 10%

​

Metrics Collection

• Request Count: Total requests in current adjustment interval
• Error Count: Requests that returned errors
• Total Latency: Cumulative response time in nanoseconds
• Current Rate: Dynamically adjusted rate limit

​

Token Refill Mechanism

refill = (elapsed_nanoseconds * current_rate) / 1_second_in_nanoseconds

​

Best Practices

• Start with conservative limits
• Set reasonable target latency
• Monitor rate adjustments
• Use with circuit breaker for full protection

​

Testing

​

Related Middlewares

• ratelimit - Fixed rate limiting
• circuitbreaker - Circuit breaker
• bulkhead - Bulkhead pattern