Skip to main content

​
Overview

The chaos middleware injects failures and latency into your application for chaos engineering and resilience testing. Use it to verify how your system handles errors and delays.

​
Installation

import "github.com/go-mizu/mizu/middlewares/chaos"

​
Quick Start

app := mizu.New()

// Inject 10% errors with random latency
app.Use(chaos.WithOptions(chaos.Options{
    Enabled:    true,
    ErrorRate:  10,
    LatencyMin: 100 * time.Millisecond,
    LatencyMax: 500 * time.Millisecond,
}))

​
Configuration

OptionTypeDefaultDescription
EnabledboolfalseEnable chaos injection
ErrorRateint0Percentage of requests to fail (0-100)
ErrorCodeint500HTTP status code for errors
LatencyMintime.Duration0Minimum latency to inject
LatencyMaxtime.Duration0Maximum latency to inject
Selectorfunc(*mizu.Ctx) bool-Filter which requests to affect

​
Examples

​
Error Injection

// Fail 20% of requests with 500 error
app.Use(chaos.Error(20, 500))

​
Latency Injection

// Add 100-500ms latency to all requests
app.Use(chaos.Latency(100*time.Millisecond, 500*time.Millisecond))

​
Combined Chaos

app.Use(chaos.WithOptions(chaos.Options{
    Enabled:    true,
    ErrorRate:  5,          // 5% errors
    ErrorCode:  503,        // Service Unavailable
    LatencyMin: 50 * time.Millisecond,
    LatencyMax: 200 * time.Millisecond,
}))

​
Selective Chaos (Path-based)

app.Use(chaos.WithOptions(chaos.Options{
    Enabled:   true,
    ErrorRate: 50,
    Selector:  chaos.PathSelector("/api/orders", "/api/payments"),
}))
// Only affects /api/orders and /api/payments

​
Selective Chaos (Method-based)

app.Use(chaos.WithOptions(chaos.Options{
    Enabled:   true,
    ErrorRate: 10,
    Selector:  chaos.MethodSelector("POST", "PUT", "DELETE"),
}))
// Only affects write operations

​
Header-triggered Chaos

app.Use(chaos.WithOptions(chaos.Options{
    Enabled:   true,
    ErrorRate: 100,
    Selector:  chaos.HeaderSelector("X-Chaos-Test"),
}))
// Only affects requests with X-Chaos-Test header

​
Dynamic Control

controller := chaos.NewController()

app.Use(controller.Middleware())

// Control chaos via admin endpoints
app.Post("/admin/chaos/enable", func(c *mizu.Ctx) error {
    controller.Enable()
    return c.Text(200, "Chaos enabled")
})

app.Post("/admin/chaos/disable", func(c *mizu.Ctx) error {
    controller.Disable()
    return c.Text(200, "Chaos disabled")
})

app.Post("/admin/chaos/config", func(c *mizu.Ctx) error {
    rate, _ := strconv.Atoi(c.FormValue("error_rate"))
    controller.SetErrorRate(rate)

    latencyMs, _ := strconv.Atoi(c.FormValue("latency_ms"))
    controller.SetLatency(
        time.Duration(latencyMs/2)*time.Millisecond,
        time.Duration(latencyMs)*time.Millisecond,
    )

    return c.Text(200, "Configured")
})

app.Get("/admin/chaos/status", func(c *mizu.Ctx) error {
    return c.JSON(200, map[string]bool{
        "enabled": controller.IsEnabled(),
    })
})

​
Environment-Based

// Only enable in testing environment
if os.Getenv("CHAOS_ENABLED") == "true" {
    rate, _ := strconv.Atoi(os.Getenv("CHAOS_ERROR_RATE"))
    app.Use(chaos.WithOptions(chaos.Options{
        Enabled:   true,
        ErrorRate: rate,
    }))
}

​
Custom Selector

app.Use(chaos.WithOptions(chaos.Options{
    Enabled:   true,
    ErrorRate: 30,
    Selector: func(c *mizu.Ctx) bool {
        // Only affect non-admin users
        userRole := c.Get("user_role")
        return userRole != "admin"
    },
}))

​
Per-Route Chaos

// Global middleware (disabled)
app.Use(chaos.New())

// Enable chaos for specific route
app.Get("/test/chaos",
    chaos.Error(50, 500),
    func(c *mizu.Ctx) error {
        return c.Text(200, "Success!")
    },
)

​
Simulate Downstream Failures

app.Get("/api/external", func(c *mizu.Ctx) error {
    // Middleware adds latency/errors before reaching here
    result, err := callExternalService()
    if err != nil {
        return c.JSON(500, map[string]string{"error": err.Error()})
    }
    return c.JSON(200, result)
})

​
Selectors

SelectorDescription
PathSelector(paths...)Match specific URL paths
MethodSelector(methods...)Match HTTP methods
HeaderSelector(header)Match requests with header

​
Controller Methods

func NewController() *Controller
func (c *Controller) Enable()
func (c *Controller) Disable()
func (c *Controller) IsEnabled() bool
func (c *Controller) SetErrorRate(rate int)
func (c *Controller) SetErrorCode(code int)
func (c *Controller) SetLatency(min, max time.Duration)
func (c *Controller) SetSelector(selector func(*mizu.Ctx) bool)
func (c *Controller) Middleware() mizu.Middleware

​
API Reference

func New() mizu.Middleware
func WithOptions(opts Options) mizu.Middleware
func Error(rate int, code int) mizu.Middleware
func Latency(min, max time.Duration) mizu.Middleware

// Selectors
func PathSelector(paths ...string) func(*mizu.Ctx) bool
func MethodSelector(methods ...string) func(*mizu.Ctx) bool
func HeaderSelector(header string) func(*mizu.Ctx) bool

​
Safety

Always protect chaos endpoints: 
// Restrict chaos control to admin
admin := app.Group("/admin")
admin.Use(basicauth.New(basicauth.Options{
    Users: map[string]string{"admin": "secret"},
}))

// Add chaos control routes to admin group
admin.Post("/chaos/enable", enableChaosHandler)

​
Technical Details

​
Architecture

The chaos middleware implements failure injection through a middleware chain pattern, supporting both static and dynamic configuration:
  • Static Configuration: WithOptions(), Error(), and Latency() create middleware with fixed settings
  • Dynamic Configuration: Controller provides thread-safe runtime configuration changes

​
Implementation Details

Random Number Generation:
  • Uses math/rand instead of crypto/rand for performance (intentionally weak RNG)
  • Error injection: Generates random number 0-99 and compares against error rate percentage
  • Latency injection: Calculates random duration between min and max using rand.Int63n()
Latency Calculation: 
latency = LatencyMin + rand.Int63n(LatencyMax - LatencyMin)
Request Flow:
  1. Check if chaos is enabled
  2. Apply selector filter (if configured)
  3. Inject latency (if configured)
  4. Inject error based on probability (if configured)
  5. Pass to next handler (if no error injected)
Selector Functions:
  • PathSelector: Creates a map of paths for O(1) lookup
  • MethodSelector: Creates a map of HTTP methods for O(1) lookup
  • HeaderSelector: Checks header presence using standard library
Controller Thread Safety:
  • Options are read atomically during request processing
  • Enable/Disable uses boolean flag
  • Configuration methods update fields directly (suitable for controlled admin access)

​
Performance Considerations

  • Selector maps provide constant-time path/method matching
  • No mutex locking on hot path for static configuration
  • time.Sleep() blocks the handler goroutine (intentional for testing)
  • Random number generation is non-cryptographic for speed

​
Best Practices

  • Never enable in production without safeguards
  • Use selectors to limit scope
  • Start with low error rates
  • Monitor during chaos testing
  • Use header-triggered chaos for CI/CD tests
  • Implement circuit breakers in your clients

​
Testing

The chaos middleware includes comprehensive test coverage for all features:
Test CaseDescriptionExpected Behavior
TestNewDefault middleware creationReturns 200 OK (chaos disabled by default)
TestErrorError injection with 100% rateReturns 503 Service Unavailable
TestLatencyLatency injection (50-100ms)Request takes >= 50ms to complete
TestWithOptions_DisabledDisabled chaos with 100% error rateReturns 200 OK (no error injection)
TestWithOptions_SelectorCustom selector for specific pathInjects error on /chaos, allows /normal
TestControllerController enable/disableCorrectly toggles enabled state
TestController_MiddlewareController middleware integrationInjects errors only when enabled
TestController_SetErrorCodeCustom error code via controllerReturns 502 Bad Gateway
TestPathSelectorPath-based selectorMatches /api/chaos, ignores /api/normal
TestHeaderSelectorHeader presence selectorSelects requests with X-Chaos header
TestMethodSelectorHTTP method selectorMatches POST, ignores GET