Throttle

​

Overview

• Limit concurrent request processing
• Prevent resource exhaustion
• Control server load
• Queue requests during high traffic

​

Installation

import "github.com/go-mizu/mizu/middlewares/throttle"

​

Quick Start

app := mizu.New()

// Allow max 100 concurrent requests
app.Use(throttle.New(100))

​

Configuration

​

Options

​

Examples

​

Simple Throttle

// Allow max 100 concurrent requests
app.Use(throttle.New(100))

​

With Backlog

// 50 concurrent with 200 waiting in queue
app.Use(throttle.WithOptions(throttle.Options{
    Limit:   50,
    Backlog: 200,
}))

​

With Timeout

// Wait max 10 seconds for a slot
app.Use(throttle.WithOptions(throttle.Options{
    Limit:   50,
    Timeout: 10 * time.Second,
}))

​

No Backlog (Immediate Rejection)

// Reject immediately when all slots busy
app.Use(throttle.WithOptions(throttle.Options{
    Limit:      100,
    Backlog:    0,
    BacklogSet: true, // Required to use Backlog: 0
}))

​

With OnThrottle Callback

app.Use(throttle.WithOptions(throttle.Options{
    Limit: 100,
    OnThrottle: func(c *mizu.Ctx) {
        log.Printf("Request throttled: %s", c.Request().URL.Path)
    },
}))

​

API Reference

​

Functions

// New creates throttle with concurrent request limit
func New(limit int) mizu.Middleware

// WithOptions creates throttle with configuration
func WithOptions(opts Options) mizu.Middleware

// Concurrency is an alias for New
func Concurrency(limit int) mizu.Middleware

​

Technical Details

​

Implementation

1. Semaphore: A buffered channel (chan struct{}) acts as a semaphore with capacity equal to the Limit. Each slot represents permission to process one request concurrently.
2. Request Flow:
   • When a request arrives, the middleware attempts to acquire a slot from the semaphore (non-blocking)
   • If a slot is available immediately, the request proceeds
   • If no slot is available, the request checks the backlog capacity
3. Backlog Queue:
   • A counter tracks the number of requests waiting for a slot
   • If backlog capacity is reached, new requests are rejected immediately with 503 Service Unavailable
   • Requests in the backlog wait with a timeout for a slot to become available
4. Timeout Handling:
   • Waiting requests use time.NewTimer with the configured Timeout
   • Three exit conditions: slot acquired, timeout reached, or request context cancelled
   • On timeout or cancellation, the backlog counter is decremented
5. Thread Safety: A sync.Mutex protects the backlog counter to prevent race conditions.

​

Default Values

• Limit: 100 concurrent requests
• Backlog: 1000 waiting requests
• Timeout: 30 seconds

​

Error Responses

• 503 Service Unavailable with “service busy” - backlog capacity exceeded
• 503 Service Unavailable with “request timeout” - timeout waiting for slot

​

Throttle vs Rate Limit

​

Best Practices

• Use for upstream service protection
• Set reasonable wait timeouts
• Monitor queue depths
• Combine with rate limiting for full control
• Set BacklogSet: true when using Backlog: 0 to disable queueing
• Use OnThrottle callback for monitoring and metrics collection

​

Testing

​

Related Middlewares

• ratelimit - Rate limiting
• bulkhead - Concurrency isolation
• adaptive - Adaptive rate limiting