Hedge - 💧 Mizu

Overview

The hedge middleware sends parallel backup requests after a delay, returning the first successful response. This reduces tail latency by hedging against slow responses. Use it when you need:

Reduce p99 latency
Handle slow backend responses
Improve user experience

Installation

import "github.com/go-mizu/mizu/middlewares/hedge"

Quick Start

app := mizu.New()

// Send hedge request after 100ms
app.Use(hedge.New(100 * time.Millisecond))

Configuration

Options

Option	Type	Default	Description
`Delay`	`time.Duration`	Required	Time before hedge
`MaxHedges`	`int`	`1`	Max parallel hedges
`Skip`	`func(*mizu.Ctx) bool`	`nil`	Skip hedging

Examples

Basic Hedging

// Hedge after 200ms
app.Use(hedge.New(200 * time.Millisecond))

Multiple Hedges

app.Use(hedge.WithOptions(hedge.Options{
    Delay:     100 * time.Millisecond,
    MaxHedges: 2, // Up to 2 parallel requests
}))

Skip Non-Idempotent

app.Use(hedge.WithOptions(hedge.Options{
    Delay: 100 * time.Millisecond,
    Skip: func(c *mizu.Ctx) bool {
        // Don't hedge mutations
        return c.Request().Method != http.MethodGet
    },
}))

How It Works

Request arrives
Start processing
After delay, if no response, send hedge request
Return first successful response
Cancel other requests

API Reference

Functions

// New creates hedge middleware
func New(delay time.Duration) mizu.Middleware

// WithOptions creates with configuration
func WithOptions(opts Options) mizu.Middleware

When to Use

High-latency backends
P99 latency optimization
Read-heavy workloads

When NOT to Use

Non-idempotent operations (POST, PUT, DELETE)
Resource-intensive operations
When backend can’t handle extra load

Technical Details

Implementation

The hedge middleware uses a sophisticated concurrent request pattern:

Request Buffering: The middleware reads and buffers the request body to enable multiple identical requests
Response Recording: Each request (original and hedges) writes to a responseRecorder that captures headers, status code, and body
Atomic Winner Selection: Uses atomic operations (CompareAndSwapInt32) to ensure only the first completing request wins
Context Management: Each request receives hedge metadata through context values (HedgeInfo)
Graceful Cancellation: When a winner is selected, remaining requests are cancelled via context

Key Components

Hedger: Main struct managing options and statistics
Options: Configuration including delay, max hedges, timeout, and callbacks
HedgeInfo: Context data containing hedge number, total hedges, winner, and duration
Stats: Tracks total requests, hedged requests, hedges triggered, and win rates
responseRecorder: Custom http.ResponseWriter that buffers responses

Statistics Tracking

The middleware tracks comprehensive statistics:

Total requests processed
Requests eligible for hedging
Number of hedges actually triggered
Wins by original vs hedged requests

Callbacks

Two callback hooks are available:

OnHedge: Called when a hedge request is triggered
OnComplete: Called when the winning response is selected

Default Values

Setting	Default Value	Purpose
Delay	100ms	Time before triggering hedge
MaxHedges	1	Maximum concurrent hedges
Timeout	30s	Overall request timeout

Best Practices

Only use for idempotent operations
Set delay based on p50 latency
Monitor hedge rate
Ensure backend can handle increased load

Testing

The hedge middleware includes comprehensive test coverage:

Test Case	Description	Expected Behavior
TestNew	Basic middleware creation	Middleware works with default options
TestFastResponse	Fast response before hedge delay	Original request completes, no hedge triggered
TestHedgeTriggered	Slow response triggers hedge	Hedge request is started after delay
TestShouldHedge	Conditional hedging based on path	Only specified paths are hedged
TestOnComplete	OnComplete callback execution	Callback receives hedge number and duration
TestStats	Statistics tracking	Tracks total requests and hedged requests
TestGetHedgeInfo	Retrieve hedge metadata from context	Returns HedgeInfo with hedge number
TestIsHedge	Check if request is a hedge	Original request returns false
TestConditional	Conditional middleware for specific methods	Only GET requests are hedged
TestForSlowRequests	Hedge slow requests with threshold	Hedges triggered after custom delay
TestMaxHedges	Multiple concurrent hedges	Up to MaxHedges requests run in parallel
TestTimeout	Request timeout handling	Requests timeout after configured duration

timeout - Request timeout
retry - Automatic retries
circuitbreaker - Circuit breaker

​Overview

​Installation

​Quick Start

​Configuration

​Options

​Examples

​Basic Hedging

​Multiple Hedges

​Skip Non-Idempotent

​How It Works

​API Reference

​Functions

​When to Use

​When NOT to Use

​Technical Details

​Implementation

​Key Components

​Statistics Tracking

​Callbacks

​Default Values

​Best Practices

​Testing

​Related Middlewares