Circuit Breaker - 💧 Mizu

Overview

The circuitbreaker middleware implements the circuit breaker pattern to prevent cascading failures. When errors exceed a threshold, it “opens” the circuit, immediately failing requests without calling the handler, giving the system time to recover. Use it when you need:

Protection against cascading failures
Graceful degradation
System resilience
External service call protection

Installation

import "github.com/go-mizu/mizu/middlewares/circuitbreaker"

Quick Start

app := mizu.New()

// Default settings: 5 failures opens circuit for 30s
app.Use(circuitbreaker.New())

Configuration

Options

Option	Type	Default	Description
`Threshold`	`int`	`5`	Failures before opening
`Timeout`	`time.Duration`	`30s`	Time before half-open
`MaxRequests`	`int`	`1`	Requests allowed in half-open
`OnStateChange`	`func(from, to State)`	-	State change callback
`IsFailure`	`func(error) bool`	Any error	Failure detection
`ErrorHandler`	`func(*mizu.Ctx) error`	-	Open circuit handler

States

const (
    StateClosed   // Normal operation, requests pass through
    StateOpen     // Circuit tripped, requests fail immediately
    StateHalfOpen // Testing recovery, limited requests allowed
)

Examples

Basic Circuit Breaker

// Opens after 5 failures, resets after 30s
app.Use(circuitbreaker.New())

Custom Threshold

app.Use(circuitbreaker.WithOptions(circuitbreaker.Options{
    Threshold: 10,           // Open after 10 failures
    Timeout:   time.Minute,  // Stay open for 1 minute
}))

Custom Failure Detection

app.Use(circuitbreaker.WithOptions(circuitbreaker.Options{
    Threshold: 5,
    IsFailure: func(err error) bool {
        // Only count specific errors as failures
        if err == nil {
            return false
        }
        // Don't count client errors
        var httpErr *HTTPError
        if errors.As(err, &httpErr) && httpErr.Code < 500 {
            return false
        }
        return true
    },
}))

State Change Monitoring

app.Use(circuitbreaker.WithOptions(circuitbreaker.Options{
    Threshold: 5,
    OnStateChange: func(from, to circuitbreaker.State) {
        log.Printf("Circuit breaker: %s -> %s", from, to)

        if to == circuitbreaker.StateOpen {
            // Alert operations team
            alertOps("Circuit breaker opened!")
        }
    },
}))

Custom Error Response

app.Use(circuitbreaker.WithOptions(circuitbreaker.Options{
    Threshold: 5,
    ErrorHandler: func(c *mizu.Ctx) error {
        return c.JSON(503, map[string]string{
            "error":   "Service temporarily unavailable",
            "message": "Please try again in a few moments",
        })
    },
}))

Per-Route Circuit Breakers

// Separate circuit breakers for different services
externalAPI := circuitbreaker.WithOptions(circuitbreaker.Options{
    Threshold: 3,
    Timeout:   time.Minute,
})

database := circuitbreaker.WithOptions(circuitbreaker.Options{
    Threshold: 5,
    Timeout:   30 * time.Second,
})

app.Get("/api/external", externalHandler, externalAPI)
app.Get("/api/data", dataHandler, database)

Recovery Testing

app.Use(circuitbreaker.WithOptions(circuitbreaker.Options{
    Threshold:   5,
    Timeout:     30 * time.Second,
    MaxRequests: 3, // Allow 3 test requests in half-open state
}))

With Metrics

var (
    circuitOpens   = prometheus.NewCounter(...)
    circuitCloses  = prometheus.NewCounter(...)
)

app.Use(circuitbreaker.WithOptions(circuitbreaker.Options{
    Threshold: 5,
    OnStateChange: func(from, to circuitbreaker.State) {
        switch to {
        case circuitbreaker.StateOpen:
            circuitOpens.Inc()
        case circuitbreaker.StateClosed:
            circuitCloses.Inc()
        }
    },
}))

Fallback Response

func withFallback(fallback func(*mizu.Ctx) error) mizu.Middleware {
    cb := circuitbreaker.WithOptions(circuitbreaker.Options{
        Threshold: 5,
        ErrorHandler: func(c *mizu.Ctx) error {
            return fallback(c)
        },
    })
    return cb
}

// Use cached data when circuit opens
app.Get("/api/data", dataHandler, withFallback(func(c *mizu.Ctx) error {
    cached := cache.Get("api_data")
    return c.JSON(200, map[string]any{
        "data":   cached,
        "cached": true,
    })
}))

State Machine

                    ┌─────────────┐
         success    │             │  failure >= threshold
    ┌──────────────│   Closed    │──────────────┐
    │               │             │              │
    │               └─────────────┘              ▼
    │                     ▲               ┌─────────────┐
    │                     │               │             │
    │              success│               │    Open     │
    │              (all)  │               │             │
    │                     │               └──────┬──────┘
    │               ┌─────┴─────┐                │
    │               │           │                │ timeout
    │               │ Half-Open │◄───────────────┘
    │               │           │
    │               └─────┬─────┘
    │                     │
    │                     │ failure
    └─────────────────────┼─────────────────────►
                          │
                          ▼
                         Open

API Reference

Functions

// New creates circuit breaker with defaults
func New() mizu.Middleware

// WithOptions creates circuit breaker with custom options
func WithOptions(opts Options) mizu.Middleware

State Type

type State int

func (s State) String() string // Returns "closed", "open", or "half-open"

Technical Details

State Machine Implementation

The circuit breaker implements a thread-safe state machine with three states:

Closed State: Normal operation where all requests are allowed through. Failures are counted, and when the failure count reaches the Threshold, the circuit transitions to Open.
Open State: Protective state where all requests are immediately rejected without calling the handler. After the Timeout period, the circuit automatically transitions to Half-Open.
Half-Open State: Recovery testing state where a limited number of requests (controlled by MaxRequests) are allowed through. If all test requests succeed, the circuit closes. If any request fails, the circuit immediately reopens.

Threshold Mechanism

The failure threshold works differently in each state:

In Closed State: Consecutive failures are counted. Once the count reaches Threshold, the circuit opens. Successful requests reset the failure counter to zero.
In Open State: No requests are processed, so failures are not counted. The circuit waits for the Timeout period before transitioning to Half-Open.
In Half-Open State: Successes are counted. The circuit requires MaxRequests consecutive successes to close. A single failure immediately reopens the circuit.

Thread Safety

The circuit breaker uses a mutex (sync.Mutex) to protect concurrent access to:

Current state
Failure and success counters
Last failure timestamp

All state transitions and counter updates are atomic operations protected by the mutex.

Failure Detection

The IsFailure function determines what constitutes a failure:

Default behavior: Any non-nil error is considered a failure
Custom behavior: Can be configured to filter specific errors (e.g., ignore 4xx client errors, only count 5xx server errors)

Best Practices

Set thresholds based on normal error rates
Use meaningful timeout values
Monitor state changes for alerting
Consider different breakers for different dependencies
Implement fallback responses when possible

Testing

The circuit breaker middleware includes comprehensive test coverage:

Test Case	Description	Expected Behavior
`TestNew`	Default circuit breaker with 5 failure threshold	After 5 consecutive failures, circuit opens and blocks subsequent requests with 503 status
`TestWithOptions_Threshold`	Custom threshold of 3 failures	Circuit opens after exactly 3 failures instead of default 5
`TestWithOptions_Timeout`	Custom timeout of 100ms	After timeout period, circuit transitions from Open to Half-Open and allows test requests
`TestWithOptions_OnStateChange`	State transition callback	Callback is invoked with correct from/to states on each transition (e.g., closed->open, open->half-open)
`TestWithOptions_ErrorHandler`	Custom error handler for open circuit	Custom error handler is invoked with proper JSON response when circuit is open
`TestWithOptions_IsFailure`	Custom failure detection logic	Only errors matching custom criteria count toward threshold; other errors are ignored
`TestState_String`	State string representation	Each state (Closed, Open, Half-Open, Unknown) returns correct string representation

ratelimit - Rate limiting
timeout - Request timeout
recover - Panic recovery

​Overview

​Installation

​Quick Start

​Configuration

​Options

​States

​Examples

​Basic Circuit Breaker

​Custom Threshold

​Custom Failure Detection

​State Change Monitoring

​Custom Error Response

​Per-Route Circuit Breakers

​Recovery Testing

​With Metrics

​Fallback Response

​State Machine

​API Reference

​Functions

​State Type

​Technical Details

​State Machine Implementation

​Threshold Mechanism

​Thread Safety

​Failure Detection

​Best Practices

​Testing

​Related Middlewares