Hedge

​

Overview

• Reduce p99 latency
• Handle slow backend responses
• Improve user experience

​

Installation

import "github.com/go-mizu/mizu/middlewares/hedge"

​

Quick Start

app := mizu.New()

// Send hedge request after 100ms
app.Use(hedge.New(100 * time.Millisecond))

​

Configuration

​

Options

​

Examples

​

Basic Hedging

// Hedge after 200ms
app.Use(hedge.New(200 * time.Millisecond))

​

Multiple Hedges

app.Use(hedge.WithOptions(hedge.Options{
    Delay:     100 * time.Millisecond,
    MaxHedges: 2, // Up to 2 parallel requests
}))

​

Skip Non-Idempotent

app.Use(hedge.WithOptions(hedge.Options{
    Delay: 100 * time.Millisecond,
    Skip: func(c *mizu.Ctx) bool {
        // Don't hedge mutations
        return c.Request().Method != http.MethodGet
    },
}))

​

How It Works

1. Request arrives
2. Start processing
3. After delay, if no response, send hedge request
4. Return first successful response
5. Cancel other requests

​

API Reference

​

Functions

// New creates hedge middleware
func New(delay time.Duration) mizu.Middleware

// WithOptions creates with configuration
func WithOptions(opts Options) mizu.Middleware

​

When to Use

• High-latency backends
• P99 latency optimization
• Read-heavy workloads

​

When NOT to Use

• Non-idempotent operations (POST, PUT, DELETE)
• Resource-intensive operations
• When backend can’t handle extra load

​

Technical Details

​

Implementation

1. Request Buffering: The middleware reads and buffers the request body to enable multiple identical requests
2. Response Recording: Each request (original and hedges) writes to a responseRecorder that captures headers, status code, and body
3. Atomic Winner Selection: Uses atomic operations (CompareAndSwapInt32) to ensure only the first completing request wins
4. Context Management: Each request receives hedge metadata through context values (HedgeInfo)
5. Graceful Cancellation: When a winner is selected, remaining requests are cancelled via context

​

Key Components

• Hedger: Main struct managing options and statistics
• Options: Configuration including delay, max hedges, timeout, and callbacks
• HedgeInfo: Context data containing hedge number, total hedges, winner, and duration
• Stats: Tracks total requests, hedged requests, hedges triggered, and win rates
• responseRecorder: Custom http.ResponseWriter that buffers responses

​

Statistics Tracking

• Total requests processed
• Requests eligible for hedging
• Number of hedges actually triggered
• Wins by original vs hedged requests

​

Callbacks

• OnHedge: Called when a hedge request is triggered
• OnComplete: Called when the winning response is selected

​

Default Values

​

Best Practices

• Only use for idempotent operations
• Set delay based on p50 latency
• Monitor hedge rate
• Ensure backend can handle increased load

​

Testing

​

Related Middlewares

• timeout - Request timeout
• retry - Automatic retries
• circuitbreaker - Circuit breaker