Documentation Index
Fetch the complete documentation index at: https://docs.go-mizu.dev/llms.txt
Use this file to discover all available pages before exploring further.
Overview
The hedge middleware sends parallel backup requests after a delay, returning the first successful response. This reduces tail latency by hedging against slow responses.
Use it when you need:
- Reduce p99 latency
- Handle slow backend responses
- Improve user experience
Installation
import "github.com/go-mizu/mizu/middlewares/hedge"
Quick Start
app := mizu.New()
// Send hedge request after 100ms
app.Use(hedge.New(100 * time.Millisecond))
Configuration
Options
| Option | Type | Default | Description |
|---|
Delay | time.Duration | Required | Time before hedge |
MaxHedges | int | 1 | Max parallel hedges |
Skip | func(*mizu.Ctx) bool | nil | Skip hedging |
Examples
Basic Hedging
// Hedge after 200ms
app.Use(hedge.New(200 * time.Millisecond))
Multiple Hedges
app.Use(hedge.WithOptions(hedge.Options{
Delay: 100 * time.Millisecond,
MaxHedges: 2, // Up to 2 parallel requests
}))
Skip Non-Idempotent
app.Use(hedge.WithOptions(hedge.Options{
Delay: 100 * time.Millisecond,
Skip: func(c *mizu.Ctx) bool {
// Don't hedge mutations
return c.Request().Method != http.MethodGet
},
}))
How It Works
- Request arrives
- Start processing
- After delay, if no response, send hedge request
- Return first successful response
- Cancel other requests
API Reference
Functions
// New creates hedge middleware
func New(delay time.Duration) mizu.Middleware
// WithOptions creates with configuration
func WithOptions(opts Options) mizu.Middleware
When to Use
- High-latency backends
- P99 latency optimization
- Read-heavy workloads
When NOT to Use
- Non-idempotent operations (POST, PUT, DELETE)
- Resource-intensive operations
- When backend can’t handle extra load
Technical Details
Implementation
The hedge middleware uses a sophisticated concurrent request pattern:
- Request Buffering: The middleware reads and buffers the request body to enable multiple identical requests
- Response Recording: Each request (original and hedges) writes to a
responseRecorder that captures headers, status code, and body
- Atomic Winner Selection: Uses atomic operations (
CompareAndSwapInt32) to ensure only the first completing request wins
- Context Management: Each request receives hedge metadata through context values (
HedgeInfo)
- Graceful Cancellation: When a winner is selected, remaining requests are cancelled via context
Key Components
- Hedger: Main struct managing options and statistics
- Options: Configuration including delay, max hedges, timeout, and callbacks
- HedgeInfo: Context data containing hedge number, total hedges, winner, and duration
- Stats: Tracks total requests, hedged requests, hedges triggered, and win rates
- responseRecorder: Custom
http.ResponseWriter that buffers responses
Statistics Tracking
The middleware tracks comprehensive statistics:
- Total requests processed
- Requests eligible for hedging
- Number of hedges actually triggered
- Wins by original vs hedged requests
Callbacks
Two callback hooks are available:
OnHedge: Called when a hedge request is triggeredOnComplete: Called when the winning response is selected
Default Values
| Setting | Default Value | Purpose |
|---|
| Delay | 100ms | Time before triggering hedge |
| MaxHedges | 1 | Maximum concurrent hedges |
| Timeout | 30s | Overall request timeout |
Best Practices
- Only use for idempotent operations
- Set delay based on p50 latency
- Monitor hedge rate
- Ensure backend can handle increased load
Testing
The hedge middleware includes comprehensive test coverage:
| Test Case | Description | Expected Behavior |
|---|
| TestNew | Basic middleware creation | Middleware works with default options |
| TestFastResponse | Fast response before hedge delay | Original request completes, no hedge triggered |
| TestHedgeTriggered | Slow response triggers hedge | Hedge request is started after delay |
| TestShouldHedge | Conditional hedging based on path | Only specified paths are hedged |
| TestOnComplete | OnComplete callback execution | Callback receives hedge number and duration |
| TestStats | Statistics tracking | Tracks total requests and hedged requests |
| TestGetHedgeInfo | Retrieve hedge metadata from context | Returns HedgeInfo with hedge number |
| TestIsHedge | Check if request is a hedge | Original request returns false |
| TestConditional | Conditional middleware for specific methods | Only GET requests are hedged |
| TestForSlowRequests | Hedge slow requests with threshold | Hedges triggered after custom delay |
| TestMaxHedges | Multiple concurrent hedges | Up to MaxHedges requests run in parallel |
| TestTimeout | Request timeout handling | Requests timeout after configured duration |