Why This Matters - The Digital Front Door Revolution
Think of a sprawling shopping mall with hundreds of stores, each with its own entrance, security guards, and information desk. Customers would be confused, frustrated, and likely leave. This is what happens when microservices expose their APIs directly without a unified entry point.
API gateways are the professional front desk of your digital infrastructure. They provide a single, elegant entry point that handles authentication, routing, rate limiting, and transformation - allowing your services to focus on what they do best: business logic.
Real-world impact: Netflix processes over 2 billion requests per day through their API gateway, handling authentication, routing to hundreds of microservices, and rate limiting for millions of users. Stripe's gateway processes payment requests from around the world, applying security rules and routing to appropriate services while maintaining sub-millisecond response times.
Without API gateways, these companies would need to replicate authentication, rate limiting, and routing logic across hundreds of services - a security nightmare and operational disaster.
Learning Objectives
By the end of this article, you will be able to:
- Design effective API gateway architectures for microservices environments
- Implement production-ready rate limiting algorithms
- Build robust authentication and authorization systems
- Create intelligent routing and request transformation mechanisms
- Apply resilience patterns for high availability
- Monitor and optimize gateway performance with comprehensive observability
- Scale gateway infrastructure to handle enterprise-level traffic
Core Concepts - Understanding Gateway Architecture
The Gateway Dilemma: Centralization vs Decentralization
Before diving into implementations, let's understand the fundamental trade-offs every API gateway must balance:
Centralization: Single point of control for consistency, security, and observability, but potential single point of failure
Decentralization: Individual service autonomy and resilience, but inconsistent security policies and operational complexity
💡 Critical Insight: The right gateway strategy balances these concerns by providing centralized control while maintaining service autonomy through well-defined contracts and protocols.
Gateway Responsibilities: More Than Just Routing
A production API gateway handles multiple cross-cutting concerns:
1/*
2API Gateway Responsibilities Visualized:
3
4Client Request Gateway Journey
5┌─────────────┐ ┌─────────────────────────────────────────────────────────────────────────────────────────────────┐
6│ Client │ → │ API GATEWAY │
7│ │ │ ┌─────────────┬──────────────┬──────────────┬──────────────┬─────────────┐ │
8│ │ │ │ Auth & │ Rate │ Route │ Transform │ Monitor │ │
9│ │ │ │ Authorization│ Limiting │ │ │ & Log │ │
10│ │ │ └─────────────┴──────────────┴──────────────┴──────────────┴─────────────┘ │
11└─────────────┘ │ │
12 │ ┌─────────────────────────────────────────────────────────────────────────────────────────┐
13 → │ BACKEND SERVICES │
14 │ ┌─────────────┬──────────────┬──────────────┬──────────────┬─────────────┐ │
15 │ │ User │ Order │ Payment │ Product │ Inventory │ │
16 │ │ Service │ Service │ Service │ Service │ Service │ │
17 │ └─────────────┴──────────────┴──────────────┴──────────────┴─────────────┘ │
18 │ │
19 └─────────────────────────────────────────────────────────────────────────────────────────┘
20
21Key Responsibilities:
221. Authentication & Authorization: Who is allowed to access what?
232. Rate Limiting: How many requests can clients make?
243. Request Routing: Which service should handle this request?
254. Request/Response Transformation: Convert formats, add headers, etc.
265. Circuit Breaking: Prevent cascade failures when services are down
276. Monitoring & Logging: Track all requests for debugging and analytics
28*/
Practical Examples - Building Production-Ready API Gateways
Gateway Foundation: Architecture and Core Components
Let's build a production-ready API gateway from scratch, focusing on reliability, performance, and maintainability.
Core Gateway Structure
1package gateway
2
3import (
4 "context"
5 "fmt"
6 "net/http"
7 "sync"
8 "time"
9
10 "github.com/go-chi/chi/v5"
11 "github.com/go-chi/chi/v5/middleware"
12)
13
14type APIGateway struct {
15 // Core components
16 router *chi.Mux
17 backends map[string]*Backend
18 middleware []Middleware
19 config *Config
20
21 // Cross-cutting concerns
22 authenticator Authenticator
23 rateLimiter RateLimiter
24 circuitBreaker CircuitBreaker
25 monitor Monitor
26
27 // Runtime state
28 stats *GatewayStats
29 healthChecker *HealthChecker
30}
31
32type Config struct {
33 // Gateway settings
34 Port string `yaml:"port"`
35 ReadTimeout time.Duration `yaml:"read_timeout"`
36 WriteTimeout time.Duration `yaml:"write_timeout"`
37 ShutdownTimeout time.Duration `yaml:"shutdown_timeout"`
38
39 // Security settings
40 JWTSecret string `yaml:"jwt_secret"`
41 RateLimitRPS int `yaml:"rate_limit_rps"`
42 AllowedOrigins []string `yaml:"allowed_origins"`
43
44 // Backend configuration
45 Backends []BackendConfig `yaml:"backends"`
46
47 // Circuit breaker settings
48 CircuitBreakerTimeout time.Duration `yaml:"circuit_breaker_timeout"`
49 CircuitBreakerThreshold int `yaml:"circuit_breaker_threshold"`
50
51 // Monitoring
52 EnableMetrics bool `yaml:"enable_metrics"`
53 MetricsEndpoint string `yaml:"metrics_endpoint"`
54}
55
56type Backend struct {
57 Name string `json:"name"`
58 URL string `json:"url"`
59 Client *http.Client `json:"-"`
60 HealthURL string `json:"health_url"`
61 Weight int `json:"weight"`
62 IsHealthy bool `json:"is_healthy"`
63 LastChecked time.Time `json:"last_checked"`
64}
65
66type Middleware func(http.Handler) http.Handler
67
68func NewAPIGateway(config *Config) {
69 // Validate configuration
70 if err := validateConfig(config); err != nil {
71 return nil, fmt.Errorf("invalid gateway config: %w", err)
72 }
73
74 // Create router with middleware
75 r := chi.NewRouter()
76
77 // Apply global middleware
78 r.Use(middleware.RequestID)
79 r.Use(middleware.RealIP)
80 r.Use(middleware.Recoverer)
81 r.Use(middleware.Timeout(config.ReadTimeout))
82 r.Use(corsMiddleware(config.AllowedOrigins))
83 r.Use(loggingMiddleware())
84
85 gateway := &APIGateway{
86 router: r,
87 backends: make(map[string]*Backend),
88 config: config,
89 stats: NewGatewayStats(),
90 }
91
92 // Initialize components
93 if err := gateway.initializeComponents(); err != nil {
94 return nil, fmt.Errorf("failed to initialize gateway components: %w", err)
95 }
96
97 // Setup routes
98 gateway.setupRoutes()
99
100 // Start background processes
101 gateway.startBackgroundProcesses()
102
103 return gateway, nil
104}
105
106func initializeComponents() error {
107 // Initialize authenticator
108 auth, err := NewJWTAuthenticator(g.config.JWTSecret)
109 if err != nil {
110 return fmt.Errorf("failed to initialize authenticator: %w", err)
111 }
112 g.authenticator = auth
113
114 // Initialize rate limiter
115 g.rateLimiter = NewTokenBucketRateLimiter(g.config.RateLimitRPS)
116
117 // Initialize circuit breaker
118 g.circuitBreaker = NewCircuitBreaker(
119 g.config.CircuitBreakerThreshold,
120 g.config.CircuitBreakerTimeout,
121 )
122
123 // Initialize monitor
124 g.monitor = NewPrometheusMonitor()
125
126 // Setup backends
127 for _, backendConfig := range g.config.Backends {
128 backend := &Backend{
129 Name: backendConfig.Name,
130 URL: backendConfig.URL,
131 Weight: backendConfig.Weight,
132 Client: &http.Client{
133 Timeout: g.config.WriteTimeout,
134 Transport: &http.Transport{
135 MaxIdleConns: 100,
136 MaxIdleConnsPerHost: 10,
137 IdleConnTimeout: 90 * time.Second,
138 TLSHandshakeTimeout: 10 * time.Second,
139 },
140 },
141 HealthURL: backendConfig.HealthURL,
142 IsHealthy: false,
143 }
144 g.backends[backendConfig.Name] = backend
145 }
146
147 // Initialize health checker
148 g.healthChecker = NewHealthChecker(g.backends, 30*time.Second)
149
150 return nil
151}
152
153func setupRoutes() {
154 // API routes
155 g.router.Route("/api/v1", func(r chi.Router) {
156 // Apply authentication middleware
157 r.Use(g.authMiddleware)
158
159 // Apply rate limiting
160 r.Use(g.rateLimitMiddleware)
161
162 // Apply circuit breaker
163 r.Use(g.circuitBreakerMiddleware)
164
165 // Backend routes
166 for _, backend := range g.backends {
167 r.Mount("/"+backend.Name, g.proxyHandler(backend))
168 }
169 })
170
171 // Health check endpoint
172 g.router.Get("/health", g.healthHandler)
173
174 // Metrics endpoint
175 if g.config.EnableMetrics {
176 g.router.Get(g.config.MetricsEndpoint, g.metricsHandler)
177 }
178
179 // Admin endpoints
180 g.router.Route("/admin", func(r chi.Router) {
181 r.Use(g.adminAuthMiddleware)
182 r.Get("/backends", g.listBackendsHandler)
183 r.Post("/backends/{name}/toggle", g.toggleBackendHandler)
184 r.Get("/stats", g.statsHandler)
185 })
186}
187
188func Start(ctx context.Context) error {
189 // Start health checker
190 go g.healthChecker.Start(ctx)
191
192 // Start HTTP server
193 server := &http.Server{
194 Addr: g.config.Port,
195 Handler: g.router,
196 ReadTimeout: g.config.ReadTimeout,
197 WriteTimeout: g.config.WriteTimeout,
198 IdleTimeout: 60 * time.Second,
199 }
200
201 log.Printf("Starting API Gateway on %s", g.config.Port)
202
203 // Graceful shutdown
204 go func() {
205 <-ctx.Done()
206 log.Println("Shutting down API Gateway...")
207
208 shutdownCtx, cancel := context.WithTimeout(context.Background(), g.config.ShutdownTimeout)
209 defer cancel()
210
211 if err := server.Shutdown(shutdownCtx); err != nil {
212 log.Printf("Gateway shutdown error: %v", err)
213 }
214 }()
215
216 return server.ListenAndServe()
217}
Advanced Rate Limiting: Token Bucket Implementation
1package ratelimit
2
3import (
4 "context"
5 "fmt"
6 "net/http"
7 "sync"
8 "time"
9
10 "github.com/redis/go-redis/v9"
11)
12
13// TokenBucketRateLimiter implements the token bucket algorithm
14type TokenBucketRateLimiter struct {
15 capacity int64 // Maximum tokens in bucket
16 refillRate int64 // Tokens per second refill rate
17 buckets map[string]*TokenBucket
18 mu sync.RWMutex
19 redis *redis.Client // For distributed rate limiting
20 distributed bool // Whether to use Redis
21}
22
23type TokenBucket struct {
24 tokens int64
25 lastRefill time.Time
26 mu sync.Mutex
27}
28
29func NewTokenBucketRateLimiter(requestsPerSecond int) *TokenBucketRateLimiter {
30 return &TokenBucketRateLimiter{
31 capacity: int64(requestsPerSecond * 2), // Allow burst capacity
32 refillRate: int64(requestsPerSecond),
33 buckets: make(map[string]*TokenBucket),
34 }
35}
36
37func NewDistributedTokenBucketRateLimiter(redis *redis.Client, requestsPerSecond int) *TokenBucketRateLimiter {
38 return &TokenBucketRateLimiter{
39 capacity: int64(requestsPerSecond * 2),
40 refillRate: int64(requestsPerSecond),
41 buckets: make(map[string]*TokenBucket),
42 redis: redis,
43 distributed: true,
44 }
45}
46
47func Allow(key string) bool {
48 if tbl.distributed {
49 return tbl.allowDistributed(key)
50 }
51 return tbl.allowLocal(key)
52}
53
54func allowLocal(key string) bool {
55 tbl.mu.RLock()
56 bucket, exists := tbl.buckets[key]
57 tbl.mu.RUnlock()
58
59 if !exists {
60 tbl.mu.Lock()
61 // Double-check after acquiring write lock
62 if bucket, exists = tbl.buckets[key]; !exists {
63 bucket = &TokenBucket{
64 tokens: tbl.capacity,
65 lastRefill: time.Now(),
66 }
67 tbl.buckets[key] = bucket
68 }
69 tbl.mu.Unlock()
70 }
71
72 bucket.mu.Lock()
73 defer bucket.mu.Unlock()
74
75 // Refill tokens based on elapsed time
76 now := time.Now()
77 elapsed := now.Sub(bucket.lastRefill).Seconds()
78 tokensToAdd := int64(elapsed * float64(tbl.refillRate))
79
80 if tokensToAdd > 0 {
81 bucket.tokens += tokensToAdd
82 if bucket.tokens > tbl.capacity {
83 bucket.tokens = tbl.capacity
84 }
85 bucket.lastRefill = now
86 }
87
88 // Check if tokens are available
89 if bucket.tokens > 0 {
90 bucket.tokens--
91 return true
92 }
93
94 return false
95}
96
97func allowDistributed(key string) bool {
98 script := redis.NewScript(`
99 local key = KEYS[1]
100 local capacity = tonumber(ARGV[1])
101 local refill_rate = tonumber(ARGV[2])
102 local now = tonumber(ARGV[3])
103
104 local bucket = redis.call('HMGET', key, 'tokens', 'last_refill')
105 local tokens = tonumber(bucket[1]) or capacity
106 local last_refill = tonumber(bucket[2]) or now
107
108 local elapsed = now - last_refill
109 local tokens_to_add = math.floor(elapsed * refill_rate)
110
111 tokens = math.min(capacity, tokens + tokens_to_add)
112
113 if tokens >= 1 then
114 tokens = tokens - 1
115 redis.call('HMSET', key, 'tokens', tokens, 'last_refill', now)
116 redis.call('EXPIRE', key, 60) -- Auto-expire after 60 seconds
117 return 1
118 else
119 redis.call('HMSET', key, 'tokens', tokens, 'last_refill', now)
120 redis.call('EXPIRE', key, 60)
121 return 0
122 end
123 `)
124
125 ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
126 defer cancel()
127
128 now := time.Now().Unix()
129 result, err := script.Run(ctx, tbl.redis,
130 []string{fmt.Sprintf("ratelimit:%s", key)},
131 tbl.capacity, tbl.refillRate, now).Result()
132
133 if err != nil {
134 // Fallback to local rate limiting on Redis failure
135 return tbl.allowLocal(key)
136 }
137
138 return result.(int64) == 1
139}
140
141// RateLimitMiddleware applies rate limiting based on client IP
142func Middleware(next http.Handler) http.Handler {
143 return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
144 // Extract client IP
145 clientIP := getClientIP(r)
146
147 // Generate unique key for this client
148 key := fmt.Sprintf("ip:%s", clientIP)
149
150 // Check rate limit
151 if !tbl.Allow(key) {
152 w.Header().Set("X-RateLimit-Limit", fmt.Sprintf("%d", tbl.capacity))
153 w.Header().Set("X-RateLimit-Remaining", "0")
154 w.Header().Set("X-RateLimit-Reset", fmt.Sprintf("%d",
155 time.Now().Add(time.Second).Unix()))
156
157 w.WriteHeader(http.StatusTooManyRequests)
158 w.Write([]byte(`{"error": "Rate limit exceeded"}`))
159 return
160 }
161
162 next.ServeHTTP(w, r)
163 })
164}
165
166func getClientIP(r *http.Request) string {
167 // Check X-Forwarded-For header first
168 forwarded := r.Header.Get("X-Forwarded-For")
169 if forwarded != "" {
170 // Take the first IP in the list
171 ips := strings.Split(forwarded, ",")
172 return strings.TrimSpace(ips[0])
173 }
174
175 // Check X-Real-IP header
176 realIP := r.Header.Get("X-Real-IP")
177 if realIP != "" {
178 return realIP
179 }
180
181 // Fall back to RemoteAddr
182 return strings.Split(r.RemoteAddr, ":")[0]
183}
JWT Authentication: Secure and Efficient
1package auth
2
3import (
4 "context"
5 "net/http"
6 "strings"
7 "time"
8
9 "github.com/golang-jwt/jwt/v5"
10)
11
12type JWTAuthenticator struct {
13 secretKey []byte
14 issuer string
15 tokenExpiry time.Duration
16}
17
18type Claims struct {
19 UserID string `json:"user_id"`
20 Email string `json:"email"`
21 Roles []string `json:"roles"`
22 jwt.RegisteredClaims
23}
24
25type UserInfo struct {
26 UserID string
27 Email string
28 Roles []string
29}
30
31func NewJWTAuthenticator(secretKey, issuer string) {
32 if len(secretKey) < 32 {
33 return nil, fmt.Errorf("JWT secret key must be at least 32 characters")
34 }
35
36 return &JWTAuthenticator{
37 secretKey: []byte(secretKey),
38 issuer: issuer,
39 tokenExpiry: 24 * time.Hour, // Default expiry
40 }, nil
41}
42
43func GenerateToken(userID, email string, roles []string) {
44 now := time.Now()
45
46 claims := Claims{
47 UserID: userID,
48 Email: email,
49 Roles: roles,
50 RegisteredClaims: jwt.RegisteredClaims{
51 ExpiresAt: jwt.NewNumericDate(now.Add(j.tokenExpiry)),
52 IssuedAt: jwt.NewNumericDate(now),
53 NotBefore: jwt.NewNumericDate(now),
54 Issuer: j.issuer,
55 },
56 }
57
58 token := jwt.NewWithClaims(jwt.SigningMethodHS256, claims)
59 return token.SignedString(j.secretKey)
60}
61
62func ValidateToken(tokenString string) {
63 token, err := jwt.ParseWithClaims(tokenString, &Claims{}, func(token *jwt.Token) {
64 if _, ok := token.Method.(*jwt.SigningMethodHMAC); !ok {
65 return nil, fmt.Errorf("unexpected signing method: %v", token.Header["alg"])
66 }
67 return j.secretKey, nil
68 })
69
70 if err != nil {
71 return nil, fmt.Errorf("failed to parse JWT token: %w", err)
72 }
73
74 if claims, ok := token.Claims.(*Claims); ok && token.Valid {
75 return claims, nil
76 }
77
78 return nil, fmt.Errorf("invalid JWT token")
79}
80
81func Middleware(next http.Handler) http.Handler {
82 return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
83 // Extract token from Authorization header
84 authHeader := r.Header.Get("Authorization")
85 if authHeader == "" {
86 http.Error(w, "Missing Authorization header", http.StatusUnauthorized)
87 return
88 }
89
90 // Check Bearer token format
91 tokenParts := strings.Split(authHeader, " ")
92 if len(tokenParts) != 2 || tokenParts[0] != "Bearer" {
93 http.Error(w, "Invalid Authorization header format", http.StatusUnauthorized)
94 return
95 }
96
97 tokenString := tokenParts[1]
98
99 // Validate token
100 claims, err := j.ValidateToken(tokenString)
101 if err != nil {
102 http.Error(w, "Invalid or expired token", http.StatusUnauthorized)
103 return
104 }
105
106 // Add user info to context
107 userInfo := &UserInfo{
108 UserID: claims.UserID,
109 Email: claims.Email,
110 Roles: claims.Roles,
111 }
112
113 ctx := context.WithValue(r.Context(), "user", userInfo)
114 next.ServeHTTP(w, r.WithContext(ctx))
115 })
116}
117
118// RequireRole middleware for role-based authorization
119func RequireRole(requiredRole string) func(http.Handler) http.Handler {
120 return func(next http.Handler) http.Handler {
121 return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
122 userInfo, ok := r.Context().Value("user").(*UserInfo)
123 if !ok {
124 http.Error(w, "User not authenticated", http.StatusUnauthorized)
125 return
126 }
127
128 // Check if user has required role
129 hasRole := false
130 for _, role := range userInfo.Roles {
131 if role == requiredRole {
132 hasRole = true
133 break
134 }
135 }
136
137 if !hasRole {
138 http.Error(w, "Insufficient permissions", http.StatusForbidden)
139 return
140 }
141
142 next.ServeHTTP(w, r)
143 })
144 }
145}
146
147// GetUserFromContext extracts user info from request context
148func GetUserFromContext(r *http.Request) {
149 user, ok := r.Context().Value("user").(*UserInfo)
150 return user, ok
151}
Circuit Breaker: Preventing Cascade Failures
1package circuitbreaker
2
3import (
4 "context"
5 "fmt"
6 "net/http"
7 "sync"
8 "time"
9)
10
11type State int
12
13const (
14 StateClosed State = iota
15 StateOpen
16 StateHalfOpen
17)
18
19type CircuitBreaker struct {
20 name string
21 maxFailures int
22 resetTimeout time.Duration
23 state State
24 failures int
25 lastFailure time.Time
26 mu sync.RWMutex
27 onRequestStart func(name string)
28 onRequestEnd func(name string, success bool, duration time.Duration)
29}
30
31func NewCircuitBreaker(maxFailures int, resetTimeout time.Duration) *CircuitBreaker {
32 return &CircuitBreaker{
33 maxFailures: maxFailures,
34 resetTimeout: resetTimeout,
35 state: StateClosed,
36 }
37}
38
39func Execute(fn func() error) error {
40 if !cb.canExecute() {
41 return fmt.Errorf("circuit breaker '%s' is open", cb.name)
42 }
43
44 start := time.Now()
45 if cb.onRequestStart != nil {
46 cb.onRequestStart(cb.name)
47 }
48
49 defer func() {
50 duration := time.Since(start)
51 if cb.onRequestEnd != nil {
52 cb.onRequestEnd(cb.name, true, duration)
53 }
54 }()
55
56 err := fn()
57 cb.recordResult(err)
58 return err
59}
60
61func canExecute() bool {
62 cb.mu.RLock()
63 defer cb.mu.RUnlock()
64
65 switch cb.state {
66 case StateClosed:
67 return true
68 case StateOpen:
69 // Check if we should transition to half-open
70 if time.Since(cb.lastFailure) > cb.resetTimeout {
71 return true
72 }
73 return false
74 case StateHalfOpen:
75 return true
76 }
77
78 return false
79}
80
81func recordResult(err error) {
82 cb.mu.Lock()
83 defer cb.mu.Unlock()
84
85 if err != nil {
86 cb.failures++
87 cb.lastFailure = time.Now()
88
89 if cb.failures >= cb.maxFailures {
90 cb.state = StateOpen
91 }
92 } else {
93 // Success - reset failure count
94 if cb.state == StateHalfOpen {
95 cb.state = StateClosed
96 }
97 cb.failures = 0
98 }
99}
100
101func GetState() State {
102 cb.mu.RLock()
103 defer cb.mu.RUnlock()
104 return cb.state
105}
106
107func Middleware(next http.Handler) http.Handler {
108 return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
109 err := cb.Execute(func() error {
110 // Execute the handler
111 w2 := &responseWriter{
112 ResponseWriter: w,
113 statusCode: http.StatusOK,
114 }
115
116 next.ServeHTTP(w2, r)
117
118 // Consider 5xx as failures
119 if w2.statusCode >= 500 {
120 return fmt.Errorf("server error: %d", w2.statusCode)
121 }
122
123 return nil
124 })
125
126 if err != nil {
127 if cb.GetState() == StateOpen {
128 http.Error(w, "Service temporarily unavailable", http.StatusServiceUnavailable)
129 } else {
130 http.Error(w, err.Error(), http.StatusInternalServerError)
131 }
132 }
133 })
134}
135
136// Custom response writer to capture status code
137type responseWriter struct {
138 http.ResponseWriter
139 statusCode int
140}
141
142func WriteHeader(code int) {
143 rw.statusCode = code
144 rw.ResponseWriter.WriteHeader(code)
145}
Common Patterns and Pitfalls - Learning from Production Experience
Pattern 1: Load Balancing with Health Checks
1type LoadBalancer struct {
2 backends []*Backend
3 currentIndex int
4 strategy string // "round_robin", "weighted_round_robin", "least_connections"
5 mu sync.RWMutex
6}
7
8func NewLoadBalancer(backends []*Backend, strategy string) *LoadBalancer {
9 return &LoadBalancer{
10 backends: backends,
11 strategy: strategy,
12 }
13}
14
15func NextBackend() *Backend {
16 lb.mu.RLock()
17 defer lb.mu.RUnlock()
18
19 // Filter healthy backends
20 healthyBackends := make([]*Backend, 0)
21 for _, backend := range lb.backends {
22 if backend.IsHealthy {
23 healthyBackends = append(healthyBackends, backend)
24 }
25 }
26
27 if len(healthyBackends) == 0 {
28 return nil // No healthy backends available
29 }
30
31 switch lb.strategy {
32 case "round_robin":
33 return lb.roundRobin(healthyBackends)
34 case "weighted_round_robin":
35 return lb.weightedRoundRobin(healthyBackends)
36 case "least_connections":
37 return lb.leastConnections(healthyBackends)
38 default:
39 return lb.roundRobin(healthyBackends)
40 }
41}
42
43func roundRobin(backends []*Backend) *Backend {
44 backend := backends[lb.currentIndex%len(backends)]
45 lb.currentIndex++
46 return backend
47}
48
49func weightedRoundRobin(backends []*Backend) *Backend {
50 // Implement weighted round-robin logic
51 totalWeight := 0
52 for _, backend := range backends {
53 totalWeight += backend.Weight
54 }
55
56 if totalWeight == 0 {
57 return backends[0] // Fallback to first backend
58 }
59
60 // Simple weighted selection
61 target := lb.currentIndex % totalWeight
62 currentWeight := 0
63
64 for _, backend := range backends {
65 currentWeight += backend.Weight
66 if target < currentWeight {
67 lb.currentIndex++
68 return backend
69 }
70 }
71
72 lb.currentIndex++
73 return backends[0]
74}
Pattern 2: Request Transformation
1package transform
2
3type RequestTransformer interface {
4 Transform(*http.Request)
5}
6
7type HeaderTransformer struct {
8 AddHeaders map[string]string
9 RemoveHeaders []string
10}
11
12func Transform(r *http.Request) {
13 // Create a copy of the request to avoid mutating original
14 req := r.Clone(r.Context())
15
16 // Remove headers
17 for _, header := range ht.RemoveHeaders {
18 req.Header.Del(header)
19 }
20
21 // Add headers
22 for key, value := range ht.AddHeaders {
23 req.Header.Set(key, value)
24 }
25
26 return req, nil
27}
28
29type JSONTransformer struct {
30 AddFields map[string]interface{}
31 RemoveFields []string
32}
33
34func Transform(r *http.Request) {
35 if r.Method != http.MethodPost && r.Method != http.MethodPut {
36 return r, nil // Only transform POST/PUT requests
37 }
38
39 // Read and parse JSON body
40 var data map[string]interface{}
41 if err := json.NewDecoder(r.Body).Decode(&data); err != nil {
42 return r, err
43 }
44
45 // Remove fields
46 for _, field := range jt.RemoveFields {
47 delete(data, field)
48 }
49
50 // Add fields
51 for key, value := range jt.AddFields {
52 data[key] = value
53 }
54
55 // Encode modified JSON
56 modifiedBody, err := json.Marshal(data)
57 if err != nil {
58 return r, err
59 }
60
61 // Create new request with modified body
62 req := r.Clone(r.Context())
63 req.Body = io.NopCloser(strings.NewReader(string(modifiedBody)))
64 req.ContentLength = int64(len(modifiedBody))
65
66 return req, nil
67}
Common Pitfalls and Solutions
Pitfall 1: Not Handling Backend Failures Gracefully
1// BAD: No error handling for backend failures
2func proxyRequest(backend *Backend, w http.ResponseWriter, r *http.Request) {
3 resp, err := backend.Client.Do(r) // Can fail!
4 if err != nil {
5 http.Error(w, "Backend error", http.StatusInternalServerError)
6 return
7 }
8 // ... handle response
9}
10
11// GOOD: Proper error handling with retries and fallbacks
12func proxyRequestWithFallback(w http.ResponseWriter, r *http.Request) error {
13 // Try multiple backends if one fails
14 for attempt := 0; attempt < 3; attempt++ {
15 backend := g.loadBalancer.NextBackend()
16 if backend == nil {
17 return fmt.Errorf("no healthy backends available")
18 }
19
20 resp, err := g.makeRequest(backend, r)
21 if err == nil {
22 return g.copyResponse(w, resp)
23 }
24
25 // Backend failed, mark as unhealthy
26 g.markBackendUnhealthy(backend)
27
28 if attempt < 2 {
29 // Exponential backoff before retry
30 time.Sleep(time.Duration(1<<uint(attempt)) * time.Second)
31 continue
32 }
33 }
34
35 // All backends failed
36 return fmt.Errorf("all backend attempts failed")
37}
Pitfall 2: Not Implementing Proper Timeout Handling
1// GOOD: Implement context-aware timeout handling
2func makeRequestWithContext(backend *Backend, r *http.Request) {
3 // Create context with timeout
4 ctx, cancel := context.WithTimeout(r.Context(), 10*time.Second)
5 defer cancel()
6
7 // Clone request with new context
8 req := r.WithContext(ctx)
9
10 return backend.Client.Do(req)
11}
Integration and Mastery - Building Complete Gateway Solutions
Comprehensive Monitoring
1package monitoring
2
3import (
4 "context"
5 "net/http"
6 "time"
7
8 "github.com/prometheus/client_golang/prometheus"
9 "github.com/prometheus/client_golang/prometheus/promauto"
10)
11
12type GatewayMonitor struct {
13 // Prometheus metrics
14 requestDuration *prometheus.HistogramVec
15 requestsTotal *prometheus.CounterVec
16 activeConnections *prometheus.GaugeVec
17 backendHealth *prometheus.GaugeVec
18}
19
20func NewGatewayMonitor() *GatewayMonitor {
21 return &GatewayMonitor{
22 requestDuration: promauto.NewHistogramVec(
23 prometheus.HistogramOpts{
24 Name: "gateway_request_duration_seconds",
25 Help: "Request duration in seconds",
26 Buckets: prometheus.DefBuckets,
27 },
28 []string{"method", "path", "backend", "status"},
29 ),
30 requestsTotal: promauto.NewCounterVec(
31 prometheus.CounterOpts{
32 Name: "gateway_requests_total",
33 Help: "Total number of requests",
34 },
35 []string{"method", "path", "backend", "status"},
36 ),
37 activeConnections: promauto.NewGaugeVec(
38 prometheus.GaugeOpts{
39 Name: "gateway_active_connections",
40 Help: "Number of active connections",
41 },
42 []string{"backend"},
43 ),
44 backendHealth: promauto.NewGaugeVec(
45 prometheus.GaugeOpts{
46 Name: "gateway_backend_health",
47 Help: "Backend health status",
48 },
49 []string{"backend"},
50 ),
51 }
52}
53
54func RecordRequest(method, path, backend, status string, duration time.Duration) {
55 gm.requestDuration.WithLabelValues(method, path, backend, status).Observe(duration.Seconds())
56 gm.requestsTotal.WithLabelValues(method, path, backend, status).Inc()
57}
58
59func RecordBackendHealth(backend string, healthy bool) {
60 value := float64(0)
61 if healthy {
62 value = 1
63 }
64 gm.backendHealth.WithLabelValues(backend).Set(value)
65}
66
67func Middleware(next http.Handler) http.Handler {
68 return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
69 start := time.Now()
70
71 // Create response writer to capture status
72 rw := &responseWriter{ResponseWriter: w, statusCode: http.StatusOK}
73
74 next.ServeHTTP(rw, r)
75
76 duration := time.Since(start)
77 status := fmt.Sprintf("%d", rw.statusCode)
78 path := r.URL.Path
79 method := r.Method
80
81 // Record metrics
82 gm.RecordRequest(method, path, "unknown", status, duration)
83 })
84}
Configuration Management
1type ConfigurationManager struct {
2 config *Config
3 reloadChan chan *Config
4 mu sync.RWMutex
5}
6
7func NewConfigurationManager(configFile string) {
8 config, err := loadConfig(configFile)
9 if err != nil {
10 return nil, err
11 }
12
13 cm := &ConfigurationManager{
14 config: config,
15 reloadChan: make(chan *Config, 1),
16 }
17
18 // Start configuration watcher
19 go cm.watchConfigFile(configFile)
20
21 return cm, nil
22}
23
24func GetConfig() *Config {
25 cm.mu.RLock()
26 defer cm.mu.RUnlock()
27 return cm.config
28}
29
30func Reload() error {
31 cm.mu.Lock()
32 defer cm.mu.Unlock()
33
34 newConfig, err := loadConfig(cm.configFile)
35 if err != nil {
36 return err
37 }
38
39 cm.config = newConfig
40 return nil
41}
Advanced Rate Limiting Strategies
Multi-Tier Rate Limiting
Production API gateways need sophisticated rate limiting that considers multiple factors: IP addresses, API keys, user tiers, and endpoints.
1package ratelimit
2
3import (
4 "context"
5 "fmt"
6 "time"
7)
8
9// MultiTierRateLimiter implements tiered rate limiting
10type MultiTierRateLimiter struct {
11 limiters map[string]*TokenBucketRateLimiter
12 tiers map[string]RateLimitTier
13}
14
15type RateLimitTier struct {
16 Name string
17 RequestsPerSec int
18 BurstCapacity int
19 QuotaPerDay int64
20}
21
22// Define standard tiers
23var (
24 FreeTier = RateLimitTier{
25 Name: "free",
26 RequestsPerSec: 10,
27 BurstCapacity: 20,
28 QuotaPerDay: 10000,
29 }
30
31 ProTier = RateLimitTier{
32 Name: "pro",
33 RequestsPerSec: 100,
34 BurstCapacity: 200,
35 QuotaPerDay: 1000000,
36 }
37
38 EnterpriseTier = RateLimitTier{
39 Name: "enterprise",
40 RequestsPerSec: 1000,
41 BurstCapacity: 2000,
42 QuotaPerDay: -1, // Unlimited
43 }
44)
45
46func NewMultiTierRateLimiter() *MultiTierRateLimiter {
47 return &MultiTierRateLimiter{
48 limiters: make(map[string]*TokenBucketRateLimiter),
49 tiers: map[string]RateLimitTier{
50 "free": FreeTier,
51 "pro": ProTier,
52 "enterprise": EnterpriseTier,
53 },
54 }
55}
56
57// AllowRequest checks if request is allowed for given tier
58func (m *MultiTierRateLimiter) AllowRequest(clientID, tier string) (bool, *RateLimitInfo) {
59 tierConfig, exists := m.tiers[tier]
60 if !exists {
61 tierConfig = FreeTier // Default to free tier
62 }
63
64 // Get or create limiter for this client
65 key := fmt.Sprintf("%s:%s", tier, clientID)
66 limiter, exists := m.limiters[key]
67 if !exists {
68 limiter = NewTokenBucketRateLimiter(tierConfig.RequestsPerSec)
69 m.limiters[key] = limiter
70 }
71
72 // Check rate limit
73 allowed := limiter.Allow(clientID)
74
75 // Check daily quota if applicable
76 if tierConfig.QuotaPerDay > 0 {
77 dailyUsage := m.getDailyUsage(clientID)
78 if dailyUsage >= tierConfig.QuotaPerDay {
79 allowed = false
80 }
81 }
82
83 info := &RateLimitInfo{
84 Tier: tier,
85 RequestsPerSec: tierConfig.RequestsPerSec,
86 Remaining: m.getRemainingQuota(clientID, tierConfig),
87 ResetAt: m.getResetTime(),
88 Allowed: allowed,
89 }
90
91 if allowed {
92 m.incrementUsage(clientID)
93 }
94
95 return allowed, info
96}
97
98type RateLimitInfo struct {
99 Tier string
100 RequestsPerSec int
101 Remaining int64
102 ResetAt time.Time
103 Allowed bool
104}
105
106// Implement rate limit middleware with tier support
107func (m *MultiTierRateLimiter) Middleware(next http.Handler) http.Handler {
108 return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
109 // Extract client ID and tier from context/headers
110 clientID := getClientID(r)
111 tier := getClientTier(r)
112
113 allowed, info := m.AllowRequest(clientID, tier)
114
115 // Set rate limit headers
116 w.Header().Set("X-RateLimit-Limit", fmt.Sprintf("%d", info.RequestsPerSec))
117 w.Header().Set("X-RateLimit-Remaining", fmt.Sprintf("%d", info.Remaining))
118 w.Header().Set("X-RateLimit-Reset", fmt.Sprintf("%d", info.ResetAt.Unix()))
119 w.Header().Set("X-RateLimit-Tier", info.Tier)
120
121 if !allowed {
122 w.WriteHeader(http.StatusTooManyRequests)
123 json.NewEncoder(w).Encode(map[string]interface{}{
124 "error": "Rate limit exceeded",
125 "tier": info.Tier,
126 "reset_at": info.ResetAt,
127 "upgrade_url": "/pricing",
128 })
129 return
130 }
131
132 next.ServeHTTP(w, r)
133 })
134}
135
136func (m *MultiTierRateLimiter) getDailyUsage(clientID string) int64 {
137 // Implementation would query Redis or similar storage
138 return 0
139}
140
141func (m *MultiTierRateLimiter) getRemainingQuota(clientID string, tier RateLimitTier) int64 {
142 if tier.QuotaPerDay < 0 {
143 return -1 // Unlimited
144 }
145 return tier.QuotaPerDay - m.getDailyUsage(clientID)
146}
147
148func (m *MultiTierRateLimiter) getResetTime() time.Time {
149 now := time.Now()
150 tomorrow := now.Add(24 * time.Hour)
151 return time.Date(tomorrow.Year(), tomorrow.Month(), tomorrow.Day(), 0, 0, 0, 0, now.Location())
152}
153
154func (m *MultiTierRateLimiter) incrementUsage(clientID string) {
155 // Implementation would increment counter in Redis
156}
Adaptive Rate Limiting
Implement intelligent rate limiting that adapts to backend health and load:
1// run
2package main
3
4import (
5 "context"
6 "fmt"
7 "sync"
8 "sync/atomic"
9 "time"
10)
11
12// AdaptiveRateLimiter adjusts limits based on backend health
13type AdaptiveRateLimiter struct {
14 baseLimit int64
15 currentLimit int64
16 minLimit int64
17 maxLimit int64
18 healthChecker HealthChecker
19 adjustmentRate float64
20 mu sync.RWMutex
21}
22
23type HealthChecker interface {
24 GetHealthScore() float64 // Returns 0.0 to 1.0
25}
26
27func NewAdaptiveRateLimiter(baseLimit int64, health HealthChecker) *AdaptiveRateLimiter {
28 return &AdaptiveRateLimiter{
29 baseLimit: baseLimit,
30 currentLimit: baseLimit,
31 minLimit: baseLimit / 10,
32 maxLimit: baseLimit * 2,
33 healthChecker: health,
34 adjustmentRate: 0.1,
35 }
36}
37
38// Start begins monitoring and adjusting rate limits
39func (a *AdaptiveRateLimiter) Start(ctx context.Context) {
40 ticker := time.NewTicker(10 * time.Second)
41 defer ticker.Stop()
42
43 for {
44 select {
45 case <-ctx.Done():
46 return
47 case <-ticker.C:
48 a.adjustLimits()
49 }
50 }
51}
52
53func (a *AdaptiveRateLimiter) adjustLimits() {
54 healthScore := a.healthChecker.GetHealthScore()
55
56 a.mu.Lock()
57 defer a.mu.Unlock()
58
59 // Calculate new limit based on health score
60 targetLimit := int64(float64(a.baseLimit) * healthScore)
61
62 // Gradually adjust current limit
63 if targetLimit > a.currentLimit {
64 // Increase slowly
65 a.currentLimit = int64(float64(a.currentLimit) * (1 + a.adjustmentRate))
66 } else {
67 // Decrease quickly
68 a.currentLimit = int64(float64(a.currentLimit) * (1 - a.adjustmentRate*2))
69 }
70
71 // Enforce bounds
72 if a.currentLimit < a.minLimit {
73 a.currentLimit = a.minLimit
74 }
75 if a.currentLimit > a.maxLimit {
76 a.currentLimit = a.maxLimit
77 }
78
79 fmt.Printf("Rate limit adjusted: %d (health: %.2f)\n", a.currentLimit, healthScore)
80}
81
82func (a *AdaptiveRateLimiter) GetCurrentLimit() int64 {
83 a.mu.RLock()
84 defer a.mu.RUnlock()
85 return a.currentLimit
86}
87
88// Simple health checker implementation
89type SimpleHealthChecker struct {
90 errorRate atomic.Value // float64
91 latency atomic.Value // time.Duration
92}
93
94func (h *SimpleHealthChecker) RecordRequest(duration time.Duration, err error) {
95 // Update latency
96 h.latency.Store(duration)
97
98 // Update error rate
99 currentRate, _ := h.errorRate.Load().(float64)
100 if err != nil {
101 h.errorRate.Store(currentRate*0.9 + 0.1) // Increase error rate
102 } else {
103 h.errorRate.Store(currentRate * 0.95) // Decrease error rate
104 }
105}
106
107func (h *SimpleHealthChecker) GetHealthScore() float64 {
108 errorRate, _ := h.errorRate.Load().(float64)
109 latency, _ := h.latency.Load().(time.Duration)
110
111 // Calculate score based on error rate and latency
112 errorScore := 1.0 - errorRate
113
114 latencyScore := 1.0
115 if latency > 0 {
116 targetLatency := 100 * time.Millisecond
117 if latency > targetLatency {
118 latencyScore = float64(targetLatency) / float64(latency)
119 }
120 }
121
122 return (errorScore * 0.7) + (latencyScore * 0.3)
123}
124
125func main() {
126 healthChecker := &SimpleHealthChecker{}
127 limiter := NewAdaptiveRateLimiter(1000, healthChecker)
128
129 ctx, cancel := context.WithTimeout(context.Background(), 1*time.Minute)
130 defer cancel()
131
132 go limiter.Start(ctx)
133
134 // Simulate varying health conditions
135 for i := 0; i < 6; i++ {
136 // Simulate requests
137 if i%2 == 0 {
138 // Healthy requests
139 healthChecker.RecordRequest(50*time.Millisecond, nil)
140 } else {
141 // Unhealthy requests
142 healthChecker.RecordRequest(500*time.Millisecond, fmt.Errorf("backend error"))
143 }
144
145 time.Sleep(12 * time.Second)
146 fmt.Printf("Current limit: %d\n", limiter.GetCurrentLimit())
147 }
148}
Advanced Authentication and Authorization
API Key Management System
Production gateways need sophisticated API key management with scopes, rate limits, and rotation:
1package auth
2
3import (
4 "context"
5 "crypto/rand"
6 "crypto/sha256"
7 "encoding/base64"
8 "encoding/hex"
9 "fmt"
10 "time"
11)
12
13// APIKeyManager manages API keys with scopes and permissions
14type APIKeyManager struct {
15 store APIKeyStore
16}
17
18type APIKey struct {
19 ID string
20 Key string // Hashed key stored in database
21 PlainKey string // Only available during creation
22 ClientID string
23 Name string
24 Scopes []string
25 RateLimit int
26 Tier string
27 CreatedAt time.Time
28 ExpiresAt *time.Time
29 LastUsedAt *time.Time
30 IsActive bool
31 Metadata map[string]string
32}
33
34type APIKeyStore interface {
35 Create(ctx context.Context, key *APIKey) error
36 GetByHash(ctx context.Context, hash string) (*APIKey, error)
37 Update(ctx context.Context, key *APIKey) error
38 Revoke(ctx context.Context, keyID string) error
39 List(ctx context.Context, clientID string) ([]*APIKey, error)
40}
41
42func NewAPIKeyManager(store APIKeyStore) *APIKeyManager {
43 return &APIKeyManager{store: store}
44}
45
46// GenerateAPIKey creates a new API key
47func (m *APIKeyManager) GenerateAPIKey(ctx context.Context, req APIKeyRequest) (*APIKey, error) {
48 // Generate secure random key
49 plainKey, err := generateSecureKey()
50 if err != nil {
51 return nil, fmt.Errorf("failed to generate key: %w", err)
52 }
53
54 // Hash the key for storage
55 hashedKey := hashKey(plainKey)
56
57 key := &APIKey{
58 ID: generateID(),
59 Key: hashedKey,
60 PlainKey: plainKey, // Only returned once
61 ClientID: req.ClientID,
62 Name: req.Name,
63 Scopes: req.Scopes,
64 RateLimit: req.RateLimit,
65 Tier: req.Tier,
66 CreatedAt: time.Now(),
67 IsActive: true,
68 Metadata: req.Metadata,
69 }
70
71 if req.ExpiresInDays > 0 {
72 expiresAt := time.Now().Add(time.Duration(req.ExpiresInDays) * 24 * time.Hour)
73 key.ExpiresAt = &expiresAt
74 }
75
76 if err := m.store.Create(ctx, key); err != nil {
77 return nil, fmt.Errorf("failed to store key: %w", err)
78 }
79
80 return key, nil
81}
82
83type APIKeyRequest struct {
84 ClientID string
85 Name string
86 Scopes []string
87 RateLimit int
88 Tier string
89 ExpiresInDays int
90 Metadata map[string]string
91}
92
93// ValidateAPIKey validates an API key and returns its metadata
94func (m *APIKeyManager) ValidateAPIKey(ctx context.Context, plainKey string) (*APIKey, error) {
95 hashedKey := hashKey(plainKey)
96
97 key, err := m.store.GetByHash(ctx, hashedKey)
98 if err != nil {
99 return nil, fmt.Errorf("invalid API key")
100 }
101
102 // Check if key is active
103 if !key.IsActive {
104 return nil, fmt.Errorf("API key has been revoked")
105 }
106
107 // Check expiration
108 if key.ExpiresAt != nil && time.Now().After(*key.ExpiresAt) {
109 return nil, fmt.Errorf("API key has expired")
110 }
111
112 // Update last used timestamp
113 now := time.Now()
114 key.LastUsedAt = &now
115 go m.store.Update(context.Background(), key)
116
117 return key, nil
118}
119
120// ValidateScope checks if the key has required scope
121func (m *APIKeyManager) ValidateScope(key *APIKey, requiredScope string) bool {
122 for _, scope := range key.Scopes {
123 if scope == requiredScope || scope == "*" {
124 return true
125 }
126 }
127 return false
128}
129
130// RevokeAPIKey revokes an API key
131func (m *APIKeyManager) RevokeAPIKey(ctx context.Context, keyID string) error {
132 return m.store.Revoke(ctx, keyID)
133}
134
135// RotateAPIKey creates a new key and revokes the old one
136func (m *APIKeyManager) RotateAPIKey(ctx context.Context, oldKeyID string) (*APIKey, error) {
137 // Get old key
138 oldKey, err := m.store.GetByHash(ctx, oldKeyID)
139 if err != nil {
140 return nil, err
141 }
142
143 // Generate new key with same properties
144 newKey, err := m.GenerateAPIKey(ctx, APIKeyRequest{
145 ClientID: oldKey.ClientID,
146 Name: oldKey.Name + " (rotated)",
147 Scopes: oldKey.Scopes,
148 RateLimit: oldKey.RateLimit,
149 Tier: oldKey.Tier,
150 Metadata: oldKey.Metadata,
151 })
152 if err != nil {
153 return nil, err
154 }
155
156 // Revoke old key
157 if err := m.RevokeAPIKey(ctx, oldKeyID); err != nil {
158 return nil, err
159 }
160
161 return newKey, nil
162}
163
164// Helper functions
165func generateSecureKey() (string, error) {
166 b := make([]byte, 32)
167 if _, err := rand.Read(b); err != nil {
168 return "", err
169 }
170 return base64.URLEncoding.EncodeToString(b), nil
171}
172
173func hashKey(key string) string {
174 hash := sha256.Sum256([]byte(key))
175 return hex.EncodeToString(hash[:])
176}
177
178func generateID() string {
179 b := make([]byte, 16)
180 rand.Read(b)
181 return hex.EncodeToString(b)
182}
183
184// Middleware for API key authentication
185func (m *APIKeyManager) Middleware(next http.Handler) http.Handler {
186 return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
187 // Extract API key from header
188 apiKey := r.Header.Get("X-API-Key")
189 if apiKey == "" {
190 http.Error(w, "Missing API key", http.StatusUnauthorized)
191 return
192 }
193
194 // Validate API key
195 key, err := m.ValidateAPIKey(r.Context(), apiKey)
196 if err != nil {
197 http.Error(w, err.Error(), http.StatusUnauthorized)
198 return
199 }
200
201 // Add key metadata to context
202 ctx := context.WithValue(r.Context(), "api_key", key)
203 next.ServeHTTP(w, r.WithContext(ctx))
204 })
205}
206
207// RequireScope middleware for scope-based authorization
208func (m *APIKeyManager) RequireScope(scope string) func(http.Handler) http.Handler {
209 return func(next http.Handler) http.Handler {
210 return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
211 key, ok := r.Context().Value("api_key").(*APIKey)
212 if !ok {
213 http.Error(w, "API key not found", http.StatusUnauthorized)
214 return
215 }
216
217 if !m.ValidateScope(key, scope) {
218 http.Error(w, fmt.Sprintf("Missing required scope: %s", scope), http.StatusForbidden)
219 return
220 }
221
222 next.ServeHTTP(w, r)
223 })
224 }
225}
Intelligent Routing and Load Balancing
Content-Based Routing
Route requests based on content, headers, or request parameters:
1package routing
2
3import (
4 "net/http"
5 "regexp"
6 "strings"
7)
8
9// ContentBasedRouter routes requests based on content
10type ContentBasedRouter struct {
11 rules []RoutingRule
12}
13
14type RoutingRule struct {
15 Name string
16 Matcher RouteMatcher
17 Backend string
18 Priority int
19 Rewrite *RewriteRule
20}
21
22type RouteMatcher interface {
23 Match(r *http.Request) bool
24}
25
26// Header matcher
27type HeaderMatcher struct {
28 Header string
29 Value *regexp.Regexp
30}
31
32func (h *HeaderMatcher) Match(r *http.Request) bool {
33 headerValue := r.Header.Get(h.Header)
34 return h.Value.MatchString(headerValue)
35}
36
37// Path matcher
38type PathMatcher struct {
39 Pattern *regexp.Regexp
40}
41
42func (p *PathMatcher) Match(r *http.Request) bool {
43 return p.Pattern.MatchString(r.URL.Path)
44}
45
46// Query parameter matcher
47type QueryMatcher struct {
48 Param string
49 Value *regexp.Regexp
50}
51
52func (q *QueryMatcher) Match(r *http.Request) bool {
53 queryValue := r.URL.Query().Get(q.Param)
54 return q.Value.MatchString(queryValue)
55}
56
57// Method matcher
58type MethodMatcher struct {
59 Methods []string
60}
61
62func (m *MethodMatcher) Match(r *http.Request) bool {
63 for _, method := range m.Methods {
64 if r.Method == method {
65 return true
66 }
67 }
68 return false
69}
70
71// Composite matcher
72type CompositeMatcher struct {
73 Matchers []RouteMatcher
74 Operator string // "AND" or "OR"
75}
76
77func (c *CompositeMatcher) Match(r *http.Request) bool {
78 if c.Operator == "AND" {
79 for _, matcher := range c.Matchers {
80 if !matcher.Match(r) {
81 return false
82 }
83 }
84 return true
85 } else { // OR
86 for _, matcher := range c.Matchers {
87 if matcher.Match(r) {
88 return true
89 }
90 }
91 return false
92 }
93}
94
95type RewriteRule struct {
96 PathPattern *regexp.Regexp
97 PathReplacement string
98 AddHeaders map[string]string
99 RemoveHeaders []string
100}
101
102func NewContentBasedRouter() *ContentBasedRouter {
103 return &ContentBasedRouter{
104 rules: make([]RoutingRule, 0),
105 }
106}
107
108// AddRule adds a routing rule
109func (r *ContentBasedRouter) AddRule(rule RoutingRule) {
110 r.rules = append(r.rules, rule)
111
112 // Sort by priority
113 sort.Slice(r.rules, func(i, j int) bool {
114 return r.rules[i].Priority > r.rules[j].Priority
115 })
116}
117
118// SelectBackend selects the appropriate backend
119func (r *ContentBasedRouter) SelectBackend(req *http.Request) (string, *RewriteRule) {
120 for _, rule := range r.rules {
121 if rule.Matcher.Match(req) {
122 return rule.Backend, rule.Rewrite
123 }
124 }
125 return "", nil // Default backend
126}
127
128// Example usage
129func ExampleContentBasedRouting() {
130 router := NewContentBasedRouter()
131
132 // Route mobile traffic to mobile backend
133 router.AddRule(RoutingRule{
134 Name: "mobile-traffic",
135 Matcher: &HeaderMatcher{
136 Header: "User-Agent",
137 Value: regexp.MustCompile(`(?i)mobile|android|iphone`),
138 },
139 Backend: "mobile-backend",
140 Priority: 10,
141 })
142
143 // Route API v2 traffic
144 router.AddRule(RoutingRule{
145 Name: "api-v2",
146 Matcher: &PathMatcher{
147 Pattern: regexp.MustCompile(`^/api/v2/`),
148 },
149 Backend: "api-v2-backend",
150 Priority: 9,
151 Rewrite: &RewriteRule{
152 PathPattern: regexp.MustCompile(`^/api/v2/`),
153 PathReplacement: "/",
154 },
155 })
156
157 // Route beta users to beta backend
158 router.AddRule(RoutingRule{
159 Name: "beta-users",
160 Matcher: &CompositeMatcher{
161 Matchers: []RouteMatcher{
162 &HeaderMatcher{
163 Header: "X-Beta-User",
164 Value: regexp.MustCompile(`true`),
165 },
166 &PathMatcher{
167 Pattern: regexp.MustCompile(`^/api/`),
168 },
169 },
170 Operator: "AND",
171 },
172 Backend: "beta-backend",
173 Priority: 8,
174 })
175}
Practice Exercises
Exercise 1: Implement Advanced Rate Limiting
Build a rate limiter that supports multiple algorithms and can switch between them.
Show Solution
1package exercise
2
3type RateLimiter struct {
4 algorithm string
5 tokenBucket *TokenBucketRateLimiter
6 slidingWindow *SlidingWindowRateLimiter
7}
8
9func Allow(key string) bool {
10 switch rl.algorithm {
11 case "token_bucket":
12 return rl.tokenBucket.Allow(key)
13 case "sliding_window":
14 return rl.slidingWindow.Allow(key)
15 default:
16 return rl.tokenBucket.Allow(key) // Default to token bucket
17 }
18}
Exercise 2: Build Dynamic Configuration System
Implement a configuration system that can hot-reload without restarting the gateway.
Show Solution
1package exercise
2
3type HotReloader struct {
4 configPath string
5 lastMod time.Time
6 config *Config
7 reloadChan chan *Config
8}
9
10func Watch() {
11 ticker := time.NewTicker(5 * time.Second)
12 for range ticker.C {
13 info, err := os.Stat(hr.configPath)
14 if err != nil {
15 continue
16 }
17
18 if info.ModTime().After(hr.lastMod) {
19 if newConfig, err := loadConfig(hr.configPath); err == nil {
20 hr.reloadChan <- newConfig
21 hr.lastMod = info.ModTime()
22 }
23 }
24 }
25}
Exercise 3: Implement Distributed Circuit Breaker
Build a circuit breaker that works across multiple gateway instances using Redis.
Show Solution
1package exercise
2
3import (
4 "context"
5 "fmt"
6 "time"
7
8 "github.com/redis/go-redis/v9"
9)
10
11type DistributedCircuitBreaker struct {
12 name string
13 redis *redis.Client
14 timeout time.Duration
15 maxFailures int
16 resetTimeout time.Duration
17}
18
19func NewDistributedCircuitBreaker(redis *redis.Client, name string, maxFailures int, resetTimeout time.Duration) *DistributedCircuitBreaker {
20 return &DistributedCircuitBreaker{
21 name: name,
22 redis: redis,
23 maxFailures: maxFailures,
24 resetTimeout: resetTimeout,
25 }
26}
27
28func (dcb *DistributedCircuitBreaker) Execute(fn func() error) error {
29 ctx := context.Background()
30
31 // Check circuit state in Redis
32 state, err := dcb.getState(ctx)
33 if err == nil && state == "open" {
34 // Check if reset timeout has passed
35 lastFailure, _ := dcb.redis.Get(ctx, fmt.Sprintf("circuit:%s:last_failure", dcb.name)).Result()
36 if lastFailure != "" {
37 lastTime, _ := time.Parse(time.RFC3339, lastFailure)
38 if time.Since(lastTime) > dcb.resetTimeout {
39 // Try half-open state
40 dcb.setState(ctx, "half-open")
41 } else {
42 return fmt.Errorf("circuit breaker is open")
43 }
44 }
45 }
46
47 // Execute function
48 err = fn()
49
50 // Record result
51 if err != nil {
52 dcb.recordFailure(ctx)
53 } else {
54 dcb.recordSuccess(ctx)
55 }
56
57 return err
58}
59
60func (dcb *DistributedCircuitBreaker) getState(ctx context.Context) (string, error) {
61 return dcb.redis.Get(ctx, fmt.Sprintf("circuit:%s:state", dcb.name)).Result()
62}
63
64func (dcb *DistributedCircuitBreaker) setState(ctx context.Context, state string) error {
65 return dcb.redis.Set(ctx, fmt.Sprintf("circuit:%s:state", dcb.name), state, 24*time.Hour).Err()
66}
67
68func (dcb *DistributedCircuitBreaker) recordFailure(ctx context.Context) {
69 key := fmt.Sprintf("circuit:%s:failures", dcb.name)
70
71 // Increment failure count
72 failures, _ := dcb.redis.Incr(ctx, key).Result()
73 dcb.redis.Expire(ctx, key, dcb.resetTimeout)
74
75 // Record last failure time
76 dcb.redis.Set(ctx, fmt.Sprintf("circuit:%s:last_failure", dcb.name),
77 time.Now().Format(time.RFC3339), dcb.resetTimeout)
78
79 // Open circuit if threshold exceeded
80 if int(failures) >= dcb.maxFailures {
81 dcb.setState(ctx, "open")
82 }
83}
84
85func (dcb *DistributedCircuitBreaker) recordSuccess(ctx context.Context) {
86 state, _ := dcb.getState(ctx)
87
88 if state == "half-open" {
89 // Close circuit on success in half-open state
90 dcb.setState(ctx, "closed")
91 dcb.redis.Del(ctx, fmt.Sprintf("circuit:%s:failures", dcb.name))
92 }
93}
Exercise 4: Multi-Tier Rate Limiting with Redis
Learning Objective: Build a production-ready rate limiting system that supports multiple user tiers (free, pro, enterprise) with different rate limits and quotas.
Real-World Context: Companies like Stripe and Twilio use tiered rate limiting to provide different service levels to customers while protecting their infrastructure. This pattern is essential for any SaaS API platform.
Difficulty: Advanced | Time Estimate: 75-90 minutes
Objective: Implement a distributed rate limiter supporting multiple tiers with daily quotas, burst capacity, and Redis-based coordination.
Show Solution
1package exercise
2
3import (
4 "context"
5 "fmt"
6 "time"
7
8 "github.com/redis/go-redis/v9"
9)
10
11// TieredRateLimiter implements multi-tier rate limiting
12type TieredRateLimiter struct {
13 redis *redis.Client
14 tiers map[string]TierConfig
15}
16
17type TierConfig struct {
18 RequestsPerSecond int
19 RequestsPerDay int64
20 BurstSize int
21}
22
23func NewTieredRateLimiter(redis *redis.Client) *TieredRateLimiter {
24 return &TieredRateLimiter{
25 redis: redis,
26 tiers: map[string]TierConfig{
27 "free": {
28 RequestsPerSecond: 10,
29 RequestsPerDay: 10000,
30 BurstSize: 20,
31 },
32 "pro": {
33 RequestsPerSecond: 100,
34 RequestsPerDay: 1000000,
35 BurstSize: 200,
36 },
37 "enterprise": {
38 RequestsPerSecond: 1000,
39 RequestsPerDay: -1, // Unlimited
40 BurstSize: 2000,
41 },
42 },
43 }
44}
45
46// AllowRequest checks if request is allowed for the given client and tier
47func (rl *TieredRateLimiter) AllowRequest(ctx context.Context, clientID, tier string) (bool, *RateLimitStatus, error) {
48 config, exists := rl.tiers[tier]
49 if !exists {
50 config = rl.tiers["free"] // Default to free tier
51 }
52
53 // Check per-second rate limit using token bucket
54 allowed, err := rl.checkTokenBucket(ctx, clientID, tier, config)
55 if err != nil {
56 return false, nil, err
57 }
58
59 if !allowed {
60 status := rl.getRateLimitStatus(ctx, clientID, tier, config)
61 return false, status, nil
62 }
63
64 // Check daily quota
65 if config.RequestsPerDay > 0 {
66 allowed, err = rl.checkDailyQuota(ctx, clientID, config)
67 if err != nil {
68 return false, nil, err
69 }
70
71 if !allowed {
72 status := rl.getRateLimitStatus(ctx, clientID, tier, config)
73 return false, status, nil
74 }
75 }
76
77 // Increment counters
78 rl.incrementCounters(ctx, clientID)
79
80 status := rl.getRateLimitStatus(ctx, clientID, tier, config)
81 return true, status, nil
82}
83
84type RateLimitStatus struct {
85 Tier string
86 Limit int
87 Remaining int64
88 DailyLimit int64
89 DailyRemaining int64
90 ResetAt time.Time
91 DailyResetAt time.Time
92}
93
94// checkTokenBucket implements token bucket algorithm using Redis Lua script
95func (rl *TieredRateLimiter) checkTokenBucket(ctx context.Context, clientID, tier string, config TierConfig) (bool, error) {
96 script := redis.NewScript(`
97 local key = KEYS[1]
98 local capacity = tonumber(ARGV[1])
99 local refill_rate = tonumber(ARGV[2])
100 local now = tonumber(ARGV[3])
101 local requested = 1
102
103 local bucket = redis.call('HMGET', key, 'tokens', 'last_refill')
104 local tokens = tonumber(bucket[1]) or capacity
105 local last_refill = tonumber(bucket[2]) or now
106
107 -- Calculate tokens to add based on time elapsed
108 local elapsed = now - last_refill
109 local tokens_to_add = math.floor(elapsed * refill_rate)
110
111 tokens = math.min(capacity, tokens + tokens_to_add)
112
113 if tokens >= requested then
114 tokens = tokens - requested
115 redis.call('HMSET', key, 'tokens', tokens, 'last_refill', now)
116 redis.call('EXPIRE', key, 60)
117 return 1
118 else
119 redis.call('HMSET', key, 'tokens', tokens, 'last_refill', now)
120 redis.call('EXPIRE', key, 60)
121 return 0
122 end
123 `)
124
125 key := fmt.Sprintf("ratelimit:token:%s:%s", tier, clientID)
126 now := time.Now().Unix()
127
128 result, err := script.Run(ctx, rl.redis,
129 []string{key},
130 config.BurstSize,
131 config.RequestsPerSecond,
132 now,
133 ).Result()
134
135 if err != nil {
136 return false, err
137 }
138
139 return result.(int64) == 1, nil
140}
141
142// checkDailyQuota checks if client hasn't exceeded daily quota
143func (rl *TieredRateLimiter) checkDailyQuota(ctx context.Context, clientID string, config TierConfig) (bool, error) {
144 key := fmt.Sprintf("ratelimit:daily:%s", clientID)
145
146 count, err := rl.redis.Get(ctx, key).Int64()
147 if err != nil && err != redis.Nil {
148 return false, err
149 }
150
151 return count < config.RequestsPerDay, nil
152}
153
154// incrementCounters increments usage counters
155func (rl *TieredRateLimiter) incrementCounters(ctx context.Context, clientID string) error {
156 dailyKey := fmt.Sprintf("ratelimit:daily:%s", clientID)
157
158 pipe := rl.redis.Pipeline()
159
160 // Increment daily counter
161 pipe.Incr(ctx, dailyKey)
162
163 // Set expiration to end of day if not set
164 pipe.ExpireAt(ctx, dailyKey, getEndOfDay())
165
166 _, err := pipe.Exec(ctx)
167 return err
168}
169
170// getRateLimitStatus returns current rate limit status
171func (rl *TieredRateLimiter) getRateLimitStatus(ctx context.Context, clientID, tier string, config TierConfig) *RateLimitStatus {
172 tokenKey := fmt.Sprintf("ratelimit:token:%s:%s", tier, clientID)
173 dailyKey := fmt.Sprintf("ratelimit:daily:%s", clientID)
174
175 // Get token bucket status
176 bucket, _ := rl.redis.HMGet(ctx, tokenKey, "tokens", "last_refill").Result()
177 tokens := int64(0)
178 if len(bucket) > 0 && bucket[0] != nil {
179 if val, ok := bucket[0].(string); ok {
180 fmt.Sscanf(val, "%d", &tokens)
181 }
182 }
183
184 // Get daily usage
185 dailyUsage, _ := rl.redis.Get(ctx, dailyKey).Int64()
186
187 status := &RateLimitStatus{
188 Tier: tier,
189 Limit: config.RequestsPerSecond,
190 Remaining: tokens,
191 DailyLimit: config.RequestsPerDay,
192 DailyRemaining: config.RequestsPerDay - dailyUsage,
193 ResetAt: time.Now().Add(time.Second),
194 DailyResetAt: getEndOfDay(),
195 }
196
197 if config.RequestsPerDay < 0 {
198 status.DailyRemaining = -1 // Unlimited
199 }
200
201 return status
202}
203
204func getEndOfDay() time.Time {
205 now := time.Now()
206 return time.Date(now.Year(), now.Month(), now.Day(), 23, 59, 59, 0, now.Location())
207}
208
209// Example HTTP middleware
210func (rl *TieredRateLimiter) Middleware(next http.Handler) http.Handler {
211 return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
212 clientID := extractClientID(r)
213 tier := extractTier(r)
214
215 allowed, status, err := rl.AllowRequest(r.Context(), clientID, tier)
216 if err != nil {
217 http.Error(w, "Internal server error", http.StatusInternalServerError)
218 return
219 }
220
221 // Set rate limit headers
222 w.Header().Set("X-RateLimit-Limit", fmt.Sprintf("%d", status.Limit))
223 w.Header().Set("X-RateLimit-Remaining", fmt.Sprintf("%d", status.Remaining))
224 w.Header().Set("X-RateLimit-Reset", fmt.Sprintf("%d", status.ResetAt.Unix()))
225 w.Header().Set("X-RateLimit-Tier", status.Tier)
226
227 if status.DailyLimit > 0 {
228 w.Header().Set("X-RateLimit-Daily-Limit", fmt.Sprintf("%d", status.DailyLimit))
229 w.Header().Set("X-RateLimit-Daily-Remaining", fmt.Sprintf("%d", status.DailyRemaining))
230 w.Header().Set("X-RateLimit-Daily-Reset", fmt.Sprintf("%d", status.DailyResetAt.Unix()))
231 }
232
233 if !allowed {
234 w.WriteHeader(http.StatusTooManyRequests)
235 json.NewEncoder(w).Encode(map[string]interface{}{
236 "error": "Rate limit exceeded",
237 "tier": status.Tier,
238 "reset_at": status.ResetAt,
239 "upgrade_url": "/pricing",
240 })
241 return
242 }
243
244 next.ServeHTTP(w, r)
245 })
246}
247
248func extractClientID(r *http.Request) string {
249 // Extract from API key, JWT, or IP
250 if apiKey := r.Header.Get("X-API-Key"); apiKey != "" {
251 return apiKey
252 }
253 return r.RemoteAddr
254}
255
256func extractTier(r *http.Request) string {
257 // Extract tier from context or header
258 if tier := r.Header.Get("X-Tier"); tier != "" {
259 return tier
260 }
261 return "free"
262}
Key Learning Points:
- Token Bucket Algorithm: Implements smooth rate limiting with burst capacity using Redis Lua scripts for atomic operations
- Multi-Tier Support: Different limits for free, pro, and enterprise tiers with daily quotas
- Distributed Coordination: Redis ensures rate limits work across multiple gateway instances
- Informative Headers: Provides clients with detailed rate limit information
- Graceful Degradation: Falls back to default tier on errors
- Production Patterns: Uses Redis pipelines, Lua scripts, and proper key expiration
Exercise 5: API Gateway with WebSocket Support and Load Balancing
Learning Objective: Build a comprehensive API gateway that handles both HTTP and WebSocket traffic with intelligent load balancing and session affinity.
Real-World Context: Modern applications like Slack and Discord use API gateways that must handle millions of persistent WebSocket connections alongside traditional HTTP requests, routing them efficiently to backend services.
Difficulty: Expert | Time Estimate: 90-110 minutes
Objective: Create a production-ready API gateway supporting WebSocket upgrades, sticky sessions, health checking, and graceful connection draining during deployments.
Show Solution
1package exercise
2
3import (
4 "context"
5 "fmt"
6 "net/http"
7 "net/http/httputil"
8 "net/url"
9 "sync"
10 "sync/atomic"
11 "time"
12
13 "github.com/gorilla/websocket"
14)
15
16// Gateway handles HTTP and WebSocket traffic
17type Gateway struct {
18 backends []*Backend
19 loadBalancer LoadBalancer
20 sessionStore SessionStore
21 healthChecker *HealthChecker
22 upgrader websocket.Upgrader
23 activeConns int64
24 mu sync.RWMutex
25}
26
27type Backend struct {
28 URL *url.URL
29 Proxy *httputil.ReverseProxy
30 Weight int
31 IsHealthy bool
32 ActiveConns int64
33 TotalRequests int64
34 mu sync.RWMutex
35}
36
37type LoadBalancer interface {
38 SelectBackend(req *http.Request) *Backend
39}
40
41type SessionStore interface {
42 Get(sessionID string) (backendURL string, exists bool)
43 Set(sessionID string, backendURL string, ttl time.Duration)
44 Delete(sessionID string)
45}
46
47func NewGateway(backendURLs []string) (*Gateway, error) {
48 backends := make([]*Backend, 0, len(backendURLs))
49
50 for _, urlStr := range backendURLs {
51 u, err := url.Parse(urlStr)
52 if err != nil {
53 return nil, fmt.Errorf("invalid backend URL %s: %w", urlStr, err)
54 }
55
56 proxy := httputil.NewSingleHostReverseProxy(u)
57 proxy.ErrorHandler = customErrorHandler
58
59 backend := &Backend{
60 URL: u,
61 Proxy: proxy,
62 Weight: 1,
63 IsHealthy: true,
64 }
65
66 backends = append(backends, backend)
67 }
68
69 g := &Gateway{
70 backends: backends,
71 loadBalancer: NewWeightedRoundRobin(backends),
72 sessionStore: NewMemorySessionStore(),
73 upgrader: websocket.Upgrader{
74 CheckOrigin: func(r *http.Request) bool {
75 return true // Configure appropriately for production
76 },
77 },
78 }
79
80 // Start health checker
81 g.healthChecker = NewHealthChecker(backends, 10*time.Second)
82 go g.healthChecker.Start(context.Background())
83
84 return g, nil
85}
86
87// ServeHTTP handles incoming requests
88func (g *Gateway) ServeHTTP(w http.ResponseWriter, r *http.Request) {
89 // Check if this is a WebSocket upgrade request
90 if isWebSocketUpgrade(r) {
91 g.handleWebSocket(w, r)
92 return
93 }
94
95 // Handle regular HTTP request
96 g.handleHTTP(w, r)
97}
98
99func (g *Gateway) handleHTTP(w http.ResponseWriter, r *http.Request) {
100 // Check for existing session
101 sessionID := extractSessionID(r)
102 var backend *Backend
103
104 if sessionID != "" {
105 // Try to get sticky session backend
106 if backendURL, exists := g.sessionStore.Get(sessionID); exists {
107 backend = g.findBackendByURL(backendURL)
108 }
109 }
110
111 // Select backend if no session or session backend not found
112 if backend == nil || !backend.IsHealthy {
113 backend = g.loadBalancer.SelectBackend(r)
114 if backend == nil {
115 http.Error(w, "No healthy backends available", http.StatusServiceUnavailable)
116 return
117 }
118
119 // Store session for sticky routing
120 if sessionID != "" {
121 g.sessionStore.Set(sessionID, backend.URL.String(), 1*time.Hour)
122 }
123 }
124
125 // Track connection
126 atomic.AddInt64(&backend.ActiveConns, 1)
127 atomic.AddInt64(&backend.TotalRequests, 1)
128 defer atomic.AddInt64(&backend.ActiveConns, -1)
129
130 // Proxy request
131 backend.Proxy.ServeHTTP(w, r)
132}
133
134func (g *Gateway) handleWebSocket(w http.ResponseWriter, r *http.Request) {
135 // Select backend (with session affinity if available)
136 sessionID := extractSessionID(r)
137 var backend *Backend
138
139 if sessionID != "" {
140 if backendURL, exists := g.sessionStore.Get(sessionID); exists {
141 backend = g.findBackendByURL(backendURL)
142 }
143 }
144
145 if backend == nil || !backend.IsHealthy {
146 backend = g.loadBalancer.SelectBackend(r)
147 if backend == nil {
148 http.Error(w, "No healthy backends available", http.StatusServiceUnavailable)
149 return
150 }
151 }
152
153 // Store session for this WebSocket connection
154 if sessionID == "" {
155 sessionID = generateSessionID()
156 }
157 g.sessionStore.Set(sessionID, backend.URL.String(), 24*time.Hour)
158
159 // Upgrade connection to WebSocket
160 clientConn, err := g.upgrader.Upgrade(w, r, nil)
161 if err != nil {
162 http.Error(w, "Failed to upgrade connection", http.StatusBadRequest)
163 return
164 }
165 defer clientConn.Close()
166
167 // Connect to backend WebSocket
168 backendWSURL := convertToWebSocketURL(backend.URL, r.URL.Path)
169 backendConn, _, err := websocket.DefaultDialer.Dial(backendWSURL, nil)
170 if err != nil {
171 clientConn.WriteMessage(websocket.CloseMessage,
172 websocket.FormatCloseMessage(websocket.CloseInternalServerErr, "Backend unavailable"))
173 return
174 }
175 defer backendConn.Close()
176
177 // Track active connection
178 atomic.AddInt64(&g.activeConns, 1)
179 atomic.AddInt64(&backend.ActiveConns, 1)
180 defer func() {
181 atomic.AddInt64(&g.activeConns, -1)
182 atomic.AddInt64(&backend.ActiveConns, -1)
183 }()
184
185 // Bidirectional message forwarding
186 g.proxyWebSocket(clientConn, backendConn)
187}
188
189// proxyWebSocket forwards messages between client and backend
190func (g *Gateway) proxyWebSocket(client, backend *websocket.Conn) {
191 errChan := make(chan error, 2)
192
193 // Client -> Backend
194 go func() {
195 for {
196 messageType, data, err := client.ReadMessage()
197 if err != nil {
198 errChan <- err
199 return
200 }
201
202 if err := backend.WriteMessage(messageType, data); err != nil {
203 errChan <- err
204 return
205 }
206 }
207 }()
208
209 // Backend -> Client
210 go func() {
211 for {
212 messageType, data, err := backend.ReadMessage()
213 if err != nil {
214 errChan <- err
215 return
216 }
217
218 if err := client.WriteMessage(messageType, data); err != nil {
219 errChan <- err
220 return
221 }
222 }
223 }()
224
225 // Wait for error from either direction
226 <-errChan
227}
228
229// Weighted round-robin load balancer
230type WeightedRoundRobin struct {
231 backends []*Backend
232 currentIndex int64
233 mu sync.Mutex
234}
235
236func NewWeightedRoundRobin(backends []*Backend) *WeightedRoundRobin {
237 return &WeightedRoundRobin{
238 backends: backends,
239 }
240}
241
242func (wrr *WeightedRoundRobin) SelectBackend(req *http.Request) *Backend {
243 wrr.mu.Lock()
244 defer wrr.mu.Unlock()
245
246 // Filter healthy backends
247 healthyBackends := make([]*Backend, 0)
248 for _, backend := range wrr.backends {
249 if backend.IsHealthy {
250 healthyBackends = append(healthyBackends, backend)
251 }
252 }
253
254 if len(healthyBackends) == 0 {
255 return nil
256 }
257
258 // Simple round-robin (can be enhanced with actual weighting)
259 backend := healthyBackends[wrr.currentIndex%int64(len(healthyBackends))]
260 wrr.currentIndex++
261
262 return backend
263}
264
265// Health checker
266type HealthChecker struct {
267 backends []*Backend
268 interval time.Duration
269}
270
271func NewHealthChecker(backends []*Backend, interval time.Duration) *HealthChecker {
272 return &HealthChecker{
273 backends: backends,
274 interval: interval,
275 }
276}
277
278func (hc *HealthChecker) Start(ctx context.Context) {
279 ticker := time.NewTicker(hc.interval)
280 defer ticker.Stop()
281
282 for {
283 select {
284 case <-ctx.Done():
285 return
286 case <-ticker.C:
287 hc.checkHealth()
288 }
289 }
290}
291
292func (hc *HealthChecker) checkHealth() {
293 for _, backend := range hc.backends {
294 go func(b *Backend) {
295 healthURL := b.URL.String() + "/health"
296 resp, err := http.Get(healthURL)
297
298 b.mu.Lock()
299 defer b.mu.Unlock()
300
301 if err != nil || resp.StatusCode != http.StatusOK {
302 b.IsHealthy = false
303 } else {
304 b.IsHealthy = true
305 }
306
307 if resp != nil {
308 resp.Body.Close()
309 }
310 }(backend)
311 }
312}
313
314// Helper functions
315func isWebSocketUpgrade(r *http.Request) bool {
316 return r.Header.Get("Upgrade") == "websocket"
317}
318
319func extractSessionID(r *http.Request) string {
320 cookie, err := r.Cookie("session_id")
321 if err != nil {
322 return ""
323 }
324 return cookie.Value
325}
326
327func (g *Gateway) findBackendByURL(urlStr string) *Backend {
328 g.mu.RLock()
329 defer g.mu.RUnlock()
330
331 for _, backend := range g.backends {
332 if backend.URL.String() == urlStr {
333 return backend
334 }
335 }
336 return nil
337}
338
339func convertToWebSocketURL(baseURL *url.URL, path string) string {
340 scheme := "ws"
341 if baseURL.Scheme == "https" {
342 scheme = "wss"
343 }
344 return fmt.Sprintf("%s://%s%s", scheme, baseURL.Host, path)
345}
346
347func generateSessionID() string {
348 return fmt.Sprintf("%d", time.Now().UnixNano())
349}
350
351func customErrorHandler(w http.ResponseWriter, r *http.Request, err error) {
352 http.Error(w, "Gateway error: "+err.Error(), http.StatusBadGateway)
353}
354
355// Simple memory-based session store
356type MemorySessionStore struct {
357 sessions map[string]sessionEntry
358 mu sync.RWMutex
359}
360
361type sessionEntry struct {
362 backendURL string
363 expiresAt time.Time
364}
365
366func NewMemorySessionStore() *MemorySessionStore {
367 store := &MemorySessionStore{
368 sessions: make(map[string]sessionEntry),
369 }
370
371 // Start cleanup goroutine
372 go store.cleanup()
373
374 return store
375}
376
377func (m *MemorySessionStore) Get(sessionID string) (string, bool) {
378 m.mu.RLock()
379 defer m.mu.RUnlock()
380
381 entry, exists := m.sessions[sessionID]
382 if !exists || time.Now().After(entry.expiresAt) {
383 return "", false
384 }
385
386 return entry.backendURL, true
387}
388
389func (m *MemorySessionStore) Set(sessionID string, backendURL string, ttl time.Duration) {
390 m.mu.Lock()
391 defer m.mu.Unlock()
392
393 m.sessions[sessionID] = sessionEntry{
394 backendURL: backendURL,
395 expiresAt: time.Now().Add(ttl),
396 }
397}
398
399func (m *MemorySessionStore) Delete(sessionID string) {
400 m.mu.Lock()
401 defer m.mu.Unlock()
402
403 delete(m.sessions, sessionID)
404}
405
406func (m *MemorySessionStore) cleanup() {
407 ticker := time.NewTicker(1 * time.Minute)
408 defer ticker.Stop()
409
410 for range ticker.C {
411 m.mu.Lock()
412 now := time.Now()
413 for id, entry := range m.sessions {
414 if now.After(entry.expiresAt) {
415 delete(m.sessions, id)
416 }
417 }
418 m.mu.Unlock()
419 }
420}
Key Learning Points:
- WebSocket Support: Handles WebSocket upgrade requests and maintains persistent connections
- Bidirectional Proxying: Forwards messages in both directions between client and backend
- Sticky Sessions: Maintains session affinity for WebSocket connections
- Health Checking: Monitors backend health and routes away from unhealthy servers
- Load Balancing: Weighted round-robin with automatic failover
- Graceful Handling: Proper connection lifecycle management and cleanup
- Production Patterns: Atomic counters, proper locking, and error handling
Key Takeaways
- Security First: Always implement authentication and authorization before routing requests
- Rate Limiting: Protect your services from abuse and ensure fair usage
- Health Monitoring: Continuously monitor backend health and route away from failed services
- Circuit Breaking: Prevent cascade failures when services become unavailable
- Observability: Comprehensive logging and metrics for troubleshooting and optimization
- Configuration Management: Support hot reloading and environment-specific configurations
- Performance: Use connection pooling, request caching, and proper timeout handling
Production Checklist
- Authentication: Implement JWT, API keys, and OAuth2 support
- Authorization: Add role-based access control
- Rate Limiting: Multiple algorithms with per-client limits
- Load Balancing: Multiple strategies with health checks
- Circuit Breaking: Prevent cascade failures
- Monitoring: Comprehensive metrics and alerting
- Security: HTTPS, CORS headers, input validation
- Testing: Load testing, failure scenarios, and security testing
When You Need an API Gateway
- Multiple microservices that need unified access control
- Different client types with varying requirements
- Cross-cutting concerns that shouldn't be duplicated in each service
- Performance optimization through request aggregation and caching
- Security requirements that need centralized enforcement
- Compliance needs for auditing and access control
Final Insight: API gateways are the backbone of microservices architecture. Invest time in building them correctly with proper security, monitoring, and resilience patterns. A well-designed gateway enables your services to focus on business logic while maintaining security, performance, and reliability at scale.