Why This Matters - Finding Bugs Before They Find You
Think of yourself as a detective investigating a complex case. Your program is the crime scene, and bugs are culprits hiding in shadows. Just like a good detective needs the right tools and techniques, a Go developer needs a solid debugging toolkit to find and fix issues efficiently.
Real-world Impact: A production system with a memory leak can cost thousands in cloud resources before anyone notices. A race condition in a financial application can cause data corruption that's only discovered months later. Effective debugging isn't just about fixing bugs—it's about preventing production disasters.
💡 Key Insight: The best debugging happens before code reaches production. Go's debugging tools are designed to catch issues early when they're cheapest to fix.
In this article, you'll learn:
- How to use Delve debugger for interactive debugging sessions
- Runtime tracing to understand concurrent behavior
- Memory profiling to detect leaks and optimize allocations
- CPU profiling to identify performance bottlenecks
- Race detection to prevent data corruption
- Production debugging patterns that work safely in live systems
Learning Objectives
By the end of this article, you'll be able to:
✅ Use Delve effectively for step-by-step code debugging
✅ Analyze runtime traces to understand goroutine behavior
✅ Profile applications to find memory and CPU bottlenecks
✅ Detect race conditions before they cause production issues
✅ Debug production systems safely using remote debugging
✅ Integrate debugging tools into your development workflow
Core Concepts - The Go Debugging Philosophy
Built-in Observability
Go's runtime is designed with debugging as a first-class citizen. This means debugging capabilities are virtually free when disabled and can be enabled in production without performance impact.
1// Go's runtime provides extensive instrumentation
2package main
3
4import (
5 "fmt"
6 "runtime"
7 "time"
8)
9
10func main() {
11 // Runtime information is always available
12 fmt.Printf("Go version: %s\n", runtime.Version())
13 fmt.Printf("GOMAXPROCS: %d\n", runtime.GOMAXPROCS(0))
14 fmt.Printf("Goroutines: %d\n", runtime.NumGoroutine())
15
16 // These work in any Go program without extra setup
17 demonstrateMemoryUsage()
18}
19
20func demonstrateMemoryUsage() {
21 var m runtime.MemStats
22 runtime.ReadMemStats(&m)
23
24 fmt.Printf("Memory allocated: %d KB\n", m.Alloc/1024)
25 fmt.Printf("Total allocations: %d\n", m.TotalAlloc)
26 fmt.Printf("GC cycles: %d\n", m.NumGC)
27
28 // Allocate some memory to see changes
29 data := make([][]byte, 1000)
30 for i := range data {
31 data[i] = make([]byte, 1024) // 1MB each
32 }
33
34 runtime.ReadMemStats(&m)
35 fmt.Printf("After allocation: %d KB\n", m.Alloc/1024)
36 fmt.Printf("GC cycles after allocation: %d\n", m.NumGC)
37}
Why this matters: You can get basic debugging information from any Go program without additional tools. This makes initial diagnosis incredibly fast.
Zero-Cost Debugging
Go's debugging features are designed to have zero overhead when disabled, allowing you to ship production code with debugging capabilities ready to activate when needed.
1// Production-safe debugging with build tags
2// +build debug
3
4package debug
5
6import (
7 "fmt"
8 "net/http"
9 _ "net/http/pprof"
10)
11
12func init() {
13 // Only start debug server in debug builds
14 go func() {
15 fmt.Println("Debug server running on :6060")
16 fmt.Println("pprof: http://localhost:6060/debug/pprof/")
17 http.ListenAndServe(":6060", nil)
18 }()
19}
Practical Examples - Interactive Debugging with Delve
Understanding Delve's Power
Delve is your high-tech magnifying glass for Go programs. It lets you pause execution, examine variables step by step, and understand exactly what your code is doing at any moment.
Basic Setup and Usage:
1// buggy.go - A program with a subtle bug
2package main
3
4import (
5 "fmt"
6 "strings"
7)
8
9type User struct {
10 ID int
11 Name string
12 Email string
13 Admin bool
14}
15
16func processUser(id int, users []User) User {
17 // Bug: This will panic if id is out of bounds
18 user := users[id-1] // Off-by-one error
19
20 // Business logic that may have issues
21 user.Name = strings.TrimSpace(user.Name)
22
23 // Validation logic
24 if len(user.Name) < 2 {
25 user.Name = "Default User"
26 }
27
28 return user
29}
30
31func main() {
32 users := []User{
33 {ID: 1, Name: " Alice", Email: "alice@example.com", Admin: false},
34 {ID: 2, Name: "Bob", Email: "bob@example.com", Admin: true},
35 {ID: 3, Name: "Charlie", Email: "charlie@example.com", Admin: false},
36 }
37
38 fmt.Println("Processing user 1...")
39 user1 := processUser(1, users)
40 fmt.Printf("User 1: %+v\n", user1)
41
42 fmt.Println("Processing user 2...")
43 user2 := processUser(2, users)
44 fmt.Printf("User 2: %+v\n", user2)
45
46 fmt.Println("Processing user 3...")
47 user3 := processUser(3, users)
48 fmt.Printf("User 3: %+v\n", user3)
49
50 // This will cause a panic due to the off-by-one error
51 fmt.Println("Processing user 4...")
52 user4 := processUser(4, users)
53 fmt.Printf("User 4: %+v\n", user4)
54}
Debugging Session with Delve:
1# Install Delve
2go install github.com/go-delve/delve/cmd/dlv@latest
3
4# Start debugging session
5dlv debug buggy.go
6
7# In the Delve debugger:
8(dlv) break main.processUser
9(dlv) continue
10
11# Program hits breakpoint at processUser function
12(dlv) print id
133
14
15# Step through the function to see the bug
16(dlv) next
17(dlv) print users
18[{ID:1 Name:"Alice" ...} {ID:2 Name:"Bob" ...} {ID:3 Name:"Charlie" ...}]
19
20# Continue stepping
21(dlv) next
22(dlv) print user
23{ID:3 Name:"Charlie" Email:"charlie@example.com" Admin:false}
24
25# The issue is users[id-1] - for id=4, we access users[3]
26# which exists, but for id=1, we access users[0] which also exists
27# However, when id=0 or id > len(users), we'll have issues
28
29(dlv) continue
Advanced Delve Techniques
Conditional Breakpoints:
1// complex.go
2package main
3
4import (
5 "fmt"
6 "math/rand"
7 "time"
8)
9
10type Transaction struct {
11 ID string
12 Amount float64
13 Status string
14}
15
16func processTransaction(tx Transaction) error {
17 // Simulate processing
18 time.Sleep(time.Millisecond * 10)
19
20 if tx.Amount > 1000 {
21 // This only happens for large transactions
22 return fmt.Errorf("amount exceeds limit: %.2f", tx.Amount)
23 }
24
25 if tx.Amount < 0 {
26 // Negative amounts should be rejected
27 return fmt.Errorf("negative amount: %.2f", tx.Amount)
28 }
29
30 tx.Status = "processed"
31 return nil
32}
33
34func main() {
35 transactions := []Transaction{
36 {ID: "tx1", Amount: 100.50},
37 {ID: "tx2", Amount: 1500.00}, // This will fail
38 {ID: "tx3", Amount: -50.00}, // This will fail
39 {ID: "tx4", Amount: 75.25},
40 {ID: "tx5", Amount: 2000.00}, // This will fail
41 }
42
43 for i, tx := range transactions {
44 fmt.Printf("Processing transaction %d/%d: %s\n", i+1, len(transactions), tx.ID)
45
46 if err := processTransaction(tx); err != nil {
47 fmt.Printf("ERROR: %s\n", err)
48 // In real code, we might need special handling here
49 if tx.Amount < 0 {
50 fmt.Println("CRITICAL: Negative amount detected!")
51 }
52 } else {
53 fmt.Printf("SUCCESS: %s processed\n", tx.ID)
54 }
55 }
56}
Debugging with Conditional Breakpoints:
1# Start debugging
2dlv debug complex.go
3
4# Set breakpoint with condition
5(dlv) break main.processTransaction
6(dlv) condition tx.Amount > 1000
7
8# Set another condition for negative amounts
9(dlv) condition tx.Amount < 0
10
11# Continue - will only stop on the large/negative transactions
12(dlv) continue
13
14# When stopped, we can examine the transaction details
15(dlv) print tx
16{ID:"tx2" Amount:1500.00 Status:""}
17
18# We can also modify variables to test fix scenarios
19(dlv) set tx.Amount = 999.99
20(dlv) continue
Integration Patterns - Combining Debugging Tools
Memory Profiling with Live Analysis
Let's create a program that demonstrates memory allocation patterns and how to profile them:
1// memory_profile.go
2package main
3
4import (
5 "fmt"
6 "net/http"
7 _ "net/http/pprof"
8 "runtime"
9 "time"
10)
11
12type DataProcessor struct {
13 cache []string
14 counter int64
15}
16
17// Simulate memory leak with growing cache
18func processData(data string) {
19 // This grows unbounded - a potential memory leak
20 dp.cache = append(dp.cache, data)
21
22 if len(dp.cache) > 10000 {
23 // Reset to prevent actual memory leak in demo
24 dp.cache = dp.cache[len(dp.cache)-1000:]
25 }
26
27 dp.counter++
28
29 // Simulate processing
30 time.Sleep(time.Millisecond)
31}
32
33// Good version with bounded cache
34func processDataBounded(data string) {
35 // Use circular buffer to prevent unbounded growth
36 const maxCache = 1000
37 if len(dp.cache) >= maxCache {
38 // Remove oldest items
39 dp.cache = dp.cache[1:]
40 }
41
42 dp.cache = append(dp.cache, data)
43 dp.counter++
44
45 // Simulate processing
46 time.Sleep(time.Millisecond)
47}
48
49func startProfileServer() {
50 go func() {
51 fmt.Println("Profile server running on :6060")
52 fmt.Println("Memory profile: http://localhost:6060/debug/pprof/heap")
53 fmt.Println("Goroutine profile: http://localhost:6060/debug/pprof/goroutine")
54
55 // Add endpoints for custom profiling
56 http.HandleFunc("/debug/memstats", func(w http.ResponseWriter, r *http.Request) {
57 var m runtime.MemStats
58 runtime.ReadMemStats(&m)
59
60 fmt.Fprintf(w, "Alloc: %d MB\n", m.Alloc/1024/1024)
61 fmt.Fprintf(w, "TotalAlloc: %d MB\n", m.TotalAlloc/1024/1024)
62 fmt.Fprintf(w, "Sys: %d MB\n", m.Sys/1024/1024)
63 fmt.Fprintf(w, "NumGC: %d\n", m.NumGC)
64 fmt.Fprintf(w, "Goroutines: %d\n", runtime.NumGoroutine())
65 })
66
67 http.ListenAndServe(":6060", nil)
68 }()
69}
70
71func main() {
72 startProfileServer()
73
74 processor := &DataProcessor{}
75
76 fmt.Println("Starting memory stress test...")
77 fmt.Println("Check profiles at: http://localhost:6060/debug/pprof/")
78
79 // Simulate data processing over time
80 ticker := time.NewTicker(10 * time.Millisecond)
81 defer ticker.Stop()
82
83 count := 0
84 for {
85 select {
86 case <-ticker.C:
87 count++
88 data := fmt.Sprintf("data-%d", count)
89
90 // Uncomment one of these to test different patterns
91 processor.processData(data) // Potential memory leak
92 // processor.processDataBounded(data) // Fixed version
93
94 if count >= 1000 {
95 fmt.Printf("Processed %d items\n", count)
96 fmt.Printf("Cache size: %d\n", len(processor.cache))
97 return
98 }
99 }
100 }
101}
Analyzing the Memory Profile:
1# Run the program
2go run memory_profile.go
3
4# While running, capture memory profile
5curl http://localhost:6060/debug/pprof/heap > heap.prof
6
7# Analyze with go tool pprof
8go tool pprof heap.prof
9
10# In pprof interactive mode:
11(pprof) top
12(pprof) list processData
13(pprof) web # Opens in browser
14
15# Generate visual graph
16go tool pprof -png heap.prof > heap.png
Race Detection and Debugging
Creating a Race Condition:
1// race_demo.go - Demonstrates race conditions
2package main
3
4import (
5 "fmt"
6 "sync"
7 "time"
8)
9
10type Counter struct {
11 value int64
12 mu sync.Mutex // Uncomment to fix race condition
13}
14
15func Increment() {
16 // RACY VERSION: No synchronization
17 c.value++
18
19 // FIXED VERSION: Use mutex
20 // c.mu.Lock()
21 // c.value++
22 // c.mu.Unlock()
23}
24
25func Value() int64 {
26 // RACY VERSION: No synchronization
27 return c.value
28
29 // FIXED VERSION: Use mutex
30 // c.mu.Lock()
31 // defer c.mu.Unlock()
32 // return c.value
33}
34
35type BankAccount struct {
36 balance float64
37 mu sync.Mutex // Uncomment to fix race condition
38}
39
40func Deposit(amount float64) {
41 // RACY VERSION: Race condition on balance update
42 b.balance += amount
43
44 // FIXED VERSION: Use mutex
45 // b.mu.Lock()
46 // b.balance += amount
47 // b.mu.Unlock()
48}
49
50func GetBalance() float64 {
51 // RACY VERSION: Race condition on balance read
52 return b.balance
53
54 // FIXED VERSION: Use mutex
55 // b.mu.Lock()
56 // defer b.mu.Unlock()
57 // return b.balance
58}
59
60func main() {
61 fmt.Println("Demonstrating race conditions...")
62
63 // Test 1: Simple counter race
64 fmt.Println("\nTest 1: Counter race condition")
65 counter := &Counter{}
66
67 var wg sync.WaitGroup
68 for i := 0; i < 100; i++ {
69 wg.Add(1)
70 go func() {
71 defer wg.Done()
72 for j := 0; j < 1000; j++ {
73 counter.Increment()
74 }
75 }()
76 }
77
78 wg.Wait()
79 fmt.Printf("Final counter value: %d\n", counter.Value())
80
81 // Test 2: Bank account race condition
82 fmt.Println("\nTest 2: Bank account race condition")
83 account := &BankAccount{balance: 1000.0}
84
85 // Simulate concurrent deposits and withdrawals
86 for i := 0; i < 50; i++ {
87 wg.Add(1)
88 go func() {
89 defer wg.Done()
90 account.Deposit(10.0)
91 }()
92
93 wg.Add(1)
94 go func() {
95 defer wg.Done()
96 balance := account.GetBalance()
97 if balance > 1500.0 {
98 // Simulate withdrawal
99 account.Deposit(-10.0) // This creates a race!
100 }
101 }()
102 }
103
104 wg.Wait()
105 fmt.Printf("Final account balance: %.2f\n", account.GetBalance())
106}
Detecting Races with Built-in Race Detector:
1# Run with race detector
2go run -race race_demo.go
3
4# Output will show race warnings:
5==================
6WARNING: DATA RACE
7Write at 0x... by goroutine X:
8main.(*Counter).Increment()
9 /path/to/race_demo.go:22 +0x...
10Previous write at 0x... by goroutine Y:
11main.(*Counter).Increment()
12 /path/to/race_demo.go:22 +0x...
13==================
Production Pattern: Safe Race Detection:
1// race_safe.go
2package main
3
4import (
5 "os"
6 "runtime"
7 "sync/atomic"
8)
9
10type SafeCounter struct {
11 value int64
12}
13
14func Increment() {
15 atomic.AddInt64(&c.value, 1)
16}
17
18func Value() int64 {
19 return atomic.LoadInt64(&c.value)
20}
21
22func main() {
23 // Check if race detection is enabled
24 if os.Getenv("ENABLE_RACE_DETECTION") == "true" {
25 // Enable runtime race detection in development
26 // Note: This requires building with -race flag
27 if !race.Enabled {
28 fmt.Println("WARNING: Race detection requested but not built with -race flag")
29 fmt.Println("Rebuild with: go build -race")
30 }
31 }
32
33 counter := &SafeCounter{}
34
35 // Use atomic operations for high-performance scenarios
36 for i := 0; i < 1000; i++ {
37 go func(id int) {
38 for j := 0; j < 1000; j++ {
39 counter.Increment()
40 }
41 }(i)
42 }
43
44 // Give some time for goroutines to complete
45 runtime.Gosched()
46 fmt.Printf("Final counter value: %d\n", counter.Value())
47}
Common Patterns and Pitfalls
Pattern: Progressive Debugging Approach
When debugging complex issues, use a systematic approach:
1// progressive_debug.go
2package main
3
4import (
5 "fmt"
6 "log"
7 "os"
8 "time"
9)
10
11// Stage 1: Add logging to understand the flow
12func processWithLogging(input string) error {
13 log.Printf("Starting process for input: %s", input)
14
15 if len(input) == 0 {
16 log.Printf("ERROR: Empty input received")
17 return fmt.Errorf("input cannot be empty")
18 }
19
20 log.Printf("Input validation passed")
21
22 // Stage 2: Add intermediate checks
23 processed := strings.ToUpper(input)
24 log.Printf("Processed input: %s", processed)
25
26 // Stage 3: Add timing to detect performance issues
27 start := time.Now()
28 result := heavyComputation(processed)
29 duration := time.Since(start)
30 log.Printf("Heavy computation took: %v", duration)
31
32 if duration > time.Second {
33 log.Printf("WARNING: Computation took longer than expected")
34 }
35
36 log.Printf("Process completed successfully")
37 return nil
38}
39
40func heavyComputation(data string) string {
41 // Simulate work that might have performance issues
42 time.Sleep(time.Millisecond * 100)
43 return fmt.Sprintf("Result: %s", data)
44}
45
46func main() {
47 // Enable debug logging
48 log.SetOutput(os.Stdout)
49
50 fmt.Println("=== Progressive Debugging Demo ===")
51
52 testCases := []string{"", "hello", "performance_test", "error_case"}
53
54 for _, testCase := range testCases {
55 fmt.Printf("\nTesting: %q\n", testCase)
56
57 if err := processWithLogging(testCase); err != nil {
58 fmt.Printf("Process failed: %v\n", err)
59 } else {
60 fmt.Printf("Process succeeded\n")
61 }
62 }
63}
Pitfall: Over-relying on Print Statements
Anti-pattern: Adding print statements everywhere instead of using proper debugging tools.
1// ❌ Bad approach: Debug by print statements
2func badDebug(data []string) {
3 fmt.Printf("Received %d items\n", len(data)) // Debug print
4 for i, item := range data {
5 fmt.Printf("Processing item %d: %s\n", i, item) // Debug print
6 if len(item) > 10 {
7 fmt.Printf("Item %d is too long, skipping\n", i) // Debug print
8 continue
9 }
10 // Process item...
11 fmt.Printf("Processed item %d\n", i) // Debug print
12 }
13 fmt.Printf("Finished processing\n") // Debug print
14}
15
16// ✅ Good approach: Use proper logging and debugging
17import (
18 "github.com/rs/zerolog/log"
19)
20
21func goodDebug(data []string) {
22 log.Info().Int("count", len(data)).Msg("Starting processing")
23
24 for i, item := range data {
25 log.Debug().Int("index", i).Str("item", item).Msg("Processing")
26
27 if len(item) > 10 {
28 log.Warn().Int("index", i).Int("length", len(item)).Msg("Item too long, skipping")
29 continue
30 }
31
32 // Process item...
33 log.Debug().Int("index", i).Msg("Processed item")
34 }
35
36 log.Info().Msg("Processing completed")
37}
Integration and Mastery
Building a Comprehensive Debugging Tool
Let's create a debugging tool that integrates multiple debugging approaches:
1// debug_tool.go
2package main
3
4import (
5 "context"
6 "encoding/json"
7 "fmt"
8 "net/http"
9 _ "net/http/pprof"
10 "os"
11 "os/signal"
12 "runtime"
13 "runtime/debug"
14 "syscall"
15 "time"
16)
17
18type DebugInfo struct {
19 Timestamp time.Time `json:"timestamp"`
20 Goroutines int `json:"goroutines"`
21 Memory MemInfo `json:"memory"`
22 BuildInfo BuildInfo `json:"build_info"`
23}
24
25type MemInfo struct {
26 Alloc uint64 `json:"alloc"` // Currently allocated memory
27 TotalAlloc uint64 `json:"total_alloc"` // Total allocated memory
28 Sys uint64 `json:"sys"` // System memory
29 NumGC uint32 `json:"num_gc"` // Number of GC runs
30}
31
32type BuildInfo struct {
33 GoVersion string `json:"go_version"`
34 Path string `json:"path"`
35 Main struct {
36 Path string `json:"path"`
37 Version string `json:"version"`
38 } `json:"main"`
39}
40
41type DebugServer struct {
42 startTime time.Time
43}
44
45func NewDebugServer() *DebugServer {
46 return &DebugServer{
47 startTime: time.Now(),
48 }
49}
50
51func getDebugInfo() DebugInfo {
52 var m runtime.MemStats
53 runtime.ReadMemStats(&m)
54
55 bi, ok := debug.ReadBuildInfo()
56 if !ok {
57 bi = &debug.BuildInfo{}
58 }
59
60 return DebugInfo{
61 Timestamp: time.Now(),
62 Goroutines: runtime.NumGoroutine(),
63 Memory: MemInfo{
64 Alloc: m.Alloc,
65 TotalAlloc: m.TotalAlloc,
66 Sys: m.Sys,
67 NumGC: m.NumGC,
68 },
69 BuildInfo: BuildInfo{
70 GoVersion: runtime.Version(),
71 Path: bi.Path,
72 Main: struct {
73 Path string `json:"path"`
74 Version string `json:"version"`
75 }{
76 Path: bi.Main.Path,
77 Version: bi.Main.Version,
78 },
79 },
80 }
81}
82
83func setupRoutes() {
84 http.HandleFunc("/debug/info", func(w http.ResponseWriter, r *http.Request) {
85 w.Header().Set("Content-Type", "application/json")
86 json.NewEncoder(w).Encode(ds.getDebugInfo())
87 })
88
89 http.HandleFunc("/debug/health", func(w http.ResponseWriter, r *http.Request) {
90 w.Header().Set("Content-Type", "application/json")
91 json.NewEncoder(w).Encode(map[string]interface{}{
92 "status": "healthy",
93 "uptime": time.Since(ds.startTime).String(),
94 "timestamp": time.Now(),
95 })
96 })
97
98 http.HandleFunc("/debug/stack", func(w http.ResponseWriter, r *http.Request) {
99 stack := make([]byte, 1024*1024)
100 length := runtime.Stack(stack, true)
101 w.Header().Set("Content-Type", "text/plain")
102 w.Write(stack[:length])
103 })
104
105 http.HandleFunc("/debug/gc", func(w http.ResponseWriter, r *http.Request) {
106 runtime.GC()
107 w.Header().Set("Content-Type", "application/json")
108 json.NewEncoder(w).Encode(map[string]interface{}{
109 "message": "GC triggered",
110 "timestamp": time.Now(),
111 })
112 })
113}
114
115func setupPProfEndpoints() {
116 // All standard pprof endpoints
117 http.Handle("/debug/pprof/", http.DefaultServeMux)
118}
119
120func main() {
121 debugServer := NewDebugServer()
122
123 debugServer.setupRoutes()
124 debugServer.setupPProfEndpoints()
125
126 // Add custom middleware for request tracking
127 http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
128 start := time.Now()
129
130 // Simulate some work
131 time.Sleep(time.Millisecond * time.Duration(10+rand.Intn(50)))
132
133 w.Header().Set("Content-Type", "application/json")
134 json.NewEncoder(w).Encode(map[string]interface{}{
135 "message": "Hello, Debug World!",
136 "timestamp": time.Now(),
137 "duration": time.Since(start).String(),
138 "request_id": r.Header.Get("X-Request-ID"),
139 })
140 })
141
142 fmt.Println("Debug server starting...")
143 fmt.Println("Available endpoints:")
144 fmt.Println(" http://localhost:8080/")
145 fmt.Println(" http://localhost:8080/debug/info")
146 fmt.Println(" http://localhost:8080/debug/pprof/")
147 fmt.Println(" http://localhost:8080/debug/pprof/heap")
148 fmt.Println(" http://localhost:8080/debug/pprof/goroutine")
149 fmt.Println(" http://localhost:8080/debug/pprof/profile")
150
151 // Graceful shutdown
152 quit := make(chan os.Signal, 1)
153 signal.Notify(quit, syscall.SIGINT, syscall.SIGTERM)
154
155 go func() {
156 <-quit
157 fmt.Println("\nShutting down debug server...")
158 time.Sleep(time.Second)
159 os.Exit(0)
160 }()
161
162 if err := http.ListenAndServe(":8080", nil); err != nil {
163 fmt.Printf("Server failed: %v\n", err)
164 }
165}
Advanced Debugging Techniques - CPU Profiling and Optimization
Understanding CPU Profiling
CPU profiling helps identify which functions consume the most CPU time. This is critical for optimizing hot paths in your application.
1// cpu_profile_demo.go - Demonstrating CPU profiling
2package main
3
4import (
5 "fmt"
6 "log"
7 "math/rand"
8 "net/http"
9 _ "net/http/pprof"
10 "runtime/pprof"
11 "os"
12 "sort"
13 "time"
14)
15
16// Inefficient sorting algorithm - intentionally slow
17func bubbleSort(arr []int) []int {
18 n := len(arr)
19 result := make([]int, n)
20 copy(result, arr)
21
22 for i := 0; i < n; i++ {
23 for j := 0; j < n-i-1; j++ {
24 if result[j] > result[j+1] {
25 result[j], result[j+1] = result[j+1], result[j]
26 }
27 }
28 }
29 return result
30}
31
32// Efficient sorting using standard library
33func efficientSort(arr []int) []int {
34 result := make([]int, len(arr))
35 copy(result, arr)
36 sort.Ints(result)
37 return result
38}
39
40// CPU-intensive data processing
41func processData(data []int, useEfficient bool) []int {
42 if useEfficient {
43 return efficientSort(data)
44 }
45 return bubbleSort(data)
46}
47
48// Generate large dataset
49func generateTestData(size int) []int {
50 data := make([]int, size)
51 for i := 0; i < size; i++ {
52 data[i] = rand.Intn(10000)
53 }
54 return data
55}
56
57// Benchmark different approaches
58func runBenchmark(dataSize int, iterations int, useEfficient bool) time.Duration {
59 data := generateTestData(dataSize)
60
61 start := time.Now()
62 for i := 0; i < iterations; i++ {
63 _ = processData(data, useEfficient)
64 }
65 return time.Since(start)
66}
67
68func setupHTTPServer() {
69 // Endpoint to trigger CPU-intensive work
70 http.HandleFunc("/process", func(w http.ResponseWriter, r *http.Request) {
71 efficient := r.URL.Query().Get("efficient") == "true"
72 size := 1000
73
74 data := generateTestData(size)
75 start := time.Now()
76 result := processData(data, efficient)
77 duration := time.Since(start)
78
79 fmt.Fprintf(w, "Processed %d items in %v\n", len(result), duration)
80 fmt.Fprintf(w, "Algorithm: ")
81 if efficient {
82 fmt.Fprintf(w, "Efficient (sort.Ints)\n")
83 } else {
84 fmt.Fprintf(w, "Bubble Sort\n")
85 }
86 })
87
88 // Endpoint to start CPU profiling
89 http.HandleFunc("/profile/cpu/start", func(w http.ResponseWriter, r *http.Request) {
90 f, err := os.Create("cpu.prof")
91 if err != nil {
92 http.Error(w, err.Error(), http.StatusInternalServerError)
93 return
94 }
95
96 if err := pprof.StartCPUProfile(f); err != nil {
97 http.Error(w, err.Error(), http.StatusInternalServerError)
98 f.Close()
99 return
100 }
101
102 fmt.Fprint(w, "CPU profiling started - write to cpu.prof\n")
103 })
104
105 // Endpoint to stop CPU profiling
106 http.HandleFunc("/profile/cpu/stop", func(w http.ResponseWriter, r *http.Request) {
107 pprof.StopCPUProfile()
108 fmt.Fprint(w, "CPU profiling stopped - analyze with: go tool pprof cpu.prof\n")
109 })
110
111 fmt.Println("Server starting on :8080")
112 fmt.Println("Endpoints:")
113 fmt.Println(" /process?efficient=false - Use bubble sort (slow)")
114 fmt.Println(" /process?efficient=true - Use efficient sort")
115 fmt.Println(" /profile/cpu/start - Start CPU profiling")
116 fmt.Println(" /profile/cpu/stop - Stop CPU profiling")
117 fmt.Println(" /debug/pprof/ - pprof endpoints")
118}
119
120func main() {
121 // Seed random number generator
122 rand.Seed(time.Now().UnixNano())
123
124 // Run benchmarks
125 fmt.Println("=== Running Benchmarks ===")
126
127 fmt.Println("\nBubble Sort (inefficient):")
128 inefficientTime := runBenchmark(100, 100, false)
129 fmt.Printf("Time: %v\n", inefficientTime)
130
131 fmt.Println("\nStandard Library Sort (efficient):")
132 efficientTime := runBenchmark(100, 100, true)
133 fmt.Printf("Time: %v\n", efficientTime)
134
135 speedup := float64(inefficientTime) / float64(efficientTime)
136 fmt.Printf("\nSpeedup: %.2fx faster\n", speedup)
137
138 // Start HTTP server for profiling
139 fmt.Println("\n=== Starting HTTP Server ===")
140 go setupHTTPServer()
141
142 log.Fatal(http.ListenAndServe(":8080", nil))
143}
Analyzing CPU Profiles:
1# Start the server
2go run cpu_profile_demo.go
3
4# In another terminal, trigger CPU profiling
5curl http://localhost:8080/profile/cpu/start
6
7# Generate load (use inefficient algorithm)
8for i in {1..100}; do
9 curl http://localhost:8080/process?efficient=false &
10done
11wait
12
13# Stop profiling
14curl http://localhost:8080/profile/cpu/stop
15
16# Analyze the profile
17go tool pprof cpu.prof
18
19# In pprof interactive mode:
20(pprof) top10 # Show top 10 CPU consumers
21(pprof) list bubbleSort # Show line-by-line profile
22(pprof) web # Generate visual call graph
23(pprof) pdf > cpu_profile.pdf # Export to PDF
24
25# Compare profiles
26go tool pprof -base=baseline.prof current.prof
Execution Tracing for Concurrency Analysis
Go's execution tracer provides insights into goroutine scheduling, system calls, and GC events.
1// trace_demo.go - Demonstrating execution tracing
2package main
3
4import (
5 "context"
6 "fmt"
7 "os"
8 "runtime/trace"
9 "sync"
10 "time"
11)
12
13// Worker pool pattern with tracing
14type WorkerPool struct {
15 workerCount int
16 jobs chan Job
17 results chan Result
18 wg sync.WaitGroup
19}
20
21type Job struct {
22 ID int
23 Data string
24 Duration time.Duration
25}
26
27type Result struct {
28 JobID int
29 Output string
30 Duration time.Duration
31 Worker int
32}
33
34func NewWorkerPool(workers int) *WorkerPool {
35 return &WorkerPool{
36 workerCount: workers,
37 jobs: make(chan Job, 100),
38 results: make(chan Result, 100),
39 }
40}
41
42func (wp *WorkerPool) Start(ctx context.Context) {
43 // Start worker goroutines
44 for i := 0; i < wp.workerCount; i++ {
45 wp.wg.Add(1)
46 go wp.worker(ctx, i)
47 }
48}
49
50func (wp *WorkerPool) worker(ctx context.Context, id int) {
51 defer wp.wg.Done()
52
53 // Add task annotation for tracing
54 region := trace.StartRegion(ctx, fmt.Sprintf("worker-%d", id))
55 defer region.End()
56
57 for {
58 select {
59 case job, ok := <-wp.jobs:
60 if !ok {
61 return
62 }
63
64 // Process job with tracing
65 result := wp.processJob(ctx, job, id)
66 wp.results <- result
67
68 case <-ctx.Done():
69 return
70 }
71 }
72}
73
74func (wp *WorkerPool) processJob(ctx context.Context, job Job, workerID int) Result {
75 // Create trace region for this job
76 ctx, task := trace.NewTask(ctx, "process-job")
77 defer task.End()
78
79 trace.Logf(ctx, "job", "Processing job %d on worker %d", job.ID, workerID)
80
81 start := time.Now()
82
83 // Simulate work
84 time.Sleep(job.Duration)
85
86 output := fmt.Sprintf("Processed: %s", job.Data)
87
88 return Result{
89 JobID: job.ID,
90 Output: output,
91 Duration: time.Since(start),
92 Worker: workerID,
93 }
94}
95
96func (wp *WorkerPool) Submit(job Job) {
97 wp.jobs <- job
98}
99
100func (wp *WorkerPool) Close() {
101 close(wp.jobs)
102 wp.wg.Wait()
103 close(wp.results)
104}
105
106// Demonstration with different concurrency patterns
107func demonstrateConcurrencyPatterns(ctx context.Context) {
108 // Pattern 1: Fan-out, fan-in
109 fmt.Println("\n=== Pattern 1: Fan-out, Fan-in ===")
110 fanOutFanIn(ctx)
111
112 // Pattern 2: Pipeline
113 fmt.Println("\n=== Pattern 2: Pipeline ===")
114 pipeline(ctx)
115
116 // Pattern 3: Worker pool
117 fmt.Println("\n=== Pattern 3: Worker Pool ===")
118 workerPoolDemo(ctx)
119}
120
121func fanOutFanIn(ctx context.Context) {
122 region := trace.StartRegion(ctx, "fan-out-fan-in")
123 defer region.End()
124
125 const workers = 5
126 input := make(chan int, 10)
127 output := make(chan int, 10)
128
129 // Fan-out: start multiple workers
130 var wg sync.WaitGroup
131 for i := 0; i < workers; i++ {
132 wg.Add(1)
133 go func(id int) {
134 defer wg.Done()
135 for val := range input {
136 // Simulate work
137 time.Sleep(10 * time.Millisecond)
138 output <- val * 2
139 }
140 }(i)
141 }
142
143 // Send work
144 go func() {
145 for i := 0; i < 20; i++ {
146 input <- i
147 }
148 close(input)
149 }()
150
151 // Fan-in: collect results
152 go func() {
153 wg.Wait()
154 close(output)
155 }()
156
157 // Consume results
158 count := 0
159 for range output {
160 count++
161 }
162
163 fmt.Printf("Processed %d items with %d workers\n", count, workers)
164}
165
166func pipeline(ctx context.Context) {
167 region := trace.StartRegion(ctx, "pipeline")
168 defer region.End()
169
170 // Stage 1: Generate numbers
171 generate := func() <-chan int {
172 out := make(chan int)
173 go func() {
174 defer close(out)
175 for i := 0; i < 10; i++ {
176 out <- i
177 }
178 }()
179 return out
180 }
181
182 // Stage 2: Square numbers
183 square := func(in <-chan int) <-chan int {
184 out := make(chan int)
185 go func() {
186 defer close(out)
187 for val := range in {
188 out <- val * val
189 }
190 }()
191 return out
192 }
193
194 // Stage 3: Print results
195 print := func(in <-chan int) {
196 for val := range in {
197 fmt.Printf("%d ", val)
198 }
199 fmt.Println()
200 }
201
202 // Connect pipeline stages
203 numbers := generate()
204 squared := square(numbers)
205 print(squared)
206}
207
208func workerPoolDemo(ctx context.Context) {
209 region := trace.StartRegion(ctx, "worker-pool")
210 defer region.End()
211
212 pool := NewWorkerPool(3)
213 pool.Start(ctx)
214
215 // Submit jobs
216 go func() {
217 for i := 0; i < 10; i++ {
218 pool.Submit(Job{
219 ID: i,
220 Data: fmt.Sprintf("task-%d", i),
221 Duration: time.Duration(10+i*5) * time.Millisecond,
222 })
223 }
224 pool.Close()
225 }()
226
227 // Collect results
228 for result := range pool.results {
229 fmt.Printf("Job %d completed by worker %d in %v\n",
230 result.JobID, result.Worker, result.Duration)
231 }
232}
233
234func main() {
235 // Start tracing
236 f, err := os.Create("trace.out")
237 if err != nil {
238 panic(err)
239 }
240 defer f.Close()
241
242 if err := trace.Start(f); err != nil {
243 panic(err)
244 }
245 defer trace.Stop()
246
247 fmt.Println("Starting execution trace...")
248 fmt.Println("Trace will be written to trace.out")
249
250 ctx := context.Background()
251 demonstrateConcurrencyPatterns(ctx)
252
253 fmt.Println("\nTrace complete!")
254 fmt.Println("View with: go tool trace trace.out")
255}
Analyzing Execution Traces:
1# Run the program to generate trace
2go run trace_demo.go
3
4# View the trace in your browser
5go tool trace trace.out
6
7# The trace viewer shows:
8# - Goroutine analysis
9# - Network blocking profile
10# - Synchronization blocking profile
11# - Syscall blocking profile
12# - Scheduler latency profile
Production Debugging Patterns
Remote Debugging with Delve
Debugging production systems requires special care. Remote debugging lets you attach to running processes safely.
1// remote_debug.go - Production-safe remote debugging setup
2package main
3
4import (
5 "context"
6 "encoding/json"
7 "fmt"
8 "log"
9 "net/http"
10 "os"
11 "os/signal"
12 "sync"
13 "syscall"
14 "time"
15)
16
17// Application represents our production service
18type Application struct {
19 config *Config
20 metrics *MetricsCollector
21 mu sync.RWMutex
22 state map[string]interface{}
23}
24
25type Config struct {
26 Port string
27 DebugEnabled bool
28 DebugPort string
29 Environment string
30}
31
32type MetricsCollector struct {
33 mu sync.RWMutex
34 requestCount int64
35 errorCount int64
36 avgResponseTime time.Duration
37}
38
39func NewApplication(config *Config) *Application {
40 return &Application{
41 config: config,
42 metrics: &MetricsCollector{},
43 state: make(map[string]interface{}),
44 }
45}
46
47// Health check endpoint
48func (app *Application) healthHandler(w http.ResponseWriter, r *http.Request) {
49 health := map[string]interface{}{
50 "status": "healthy",
51 "timestamp": time.Now(),
52 "environment": app.config.Environment,
53 "metrics": app.getMetrics(),
54 }
55
56 w.Header().Set("Content-Type", "application/json")
57 json.NewEncoder(w).Encode(health)
58}
59
60func (app *Application) getMetrics() map[string]interface{} {
61 app.metrics.mu.RLock()
62 defer app.metrics.mu.RUnlock()
63
64 return map[string]interface{}{
65 "request_count": app.metrics.requestCount,
66 "error_count": app.metrics.errorCount,
67 "avg_response_time": app.metrics.avgResponseTime.String(),
68 }
69}
70
71// Debug endpoints - only enabled in debug mode
72func (app *Application) setupDebugEndpoints(mux *http.ServeMux) {
73 if !app.config.DebugEnabled {
74 return
75 }
76
77 log.Println("⚠️ Debug endpoints enabled - NOT for production!")
78
79 // State inspection endpoint
80 mux.HandleFunc("/debug/state", func(w http.ResponseWriter, r *http.Request) {
81 app.mu.RLock()
82 defer app.mu.RUnlock()
83
84 w.Header().Set("Content-Type", "application/json")
85 json.NewEncoder(w).Encode(app.state)
86 })
87
88 // Goroutine dump endpoint
89 mux.HandleFunc("/debug/goroutines", func(w http.ResponseWriter, r *http.Request) {
90 buf := make([]byte, 1<<20) // 1MB buffer
91 stackLen := runtime.Stack(buf, true)
92
93 w.Header().Set("Content-Type", "text/plain")
94 w.Write(buf[:stackLen])
95 })
96
97 // Configuration dump endpoint
98 mux.HandleFunc("/debug/config", func(w http.ResponseWriter, r *http.Request) {
99 w.Header().Set("Content-Type", "application/json")
100 json.NewEncoder(w).Encode(app.config)
101 })
102}
103
104// Production-safe logging with context
105func (app *Application) logWithContext(ctx context.Context, level, message string, fields map[string]interface{}) {
106 logEntry := map[string]interface{}{
107 "timestamp": time.Now(),
108 "level": level,
109 "message": message,
110 "environment": app.config.Environment,
111 }
112
113 // Add context values
114 if requestID := ctx.Value("request_id"); requestID != nil {
115 logEntry["request_id"] = requestID
116 }
117
118 // Add custom fields
119 for k, v := range fields {
120 logEntry[k] = v
121 }
122
123 jsonLog, _ := json.Marshal(logEntry)
124 log.Println(string(jsonLog))
125}
126
127// Middleware for request tracking
128func (app *Application) trackingMiddleware(next http.Handler) http.Handler {
129 return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
130 start := time.Now()
131
132 // Generate request ID
133 requestID := fmt.Sprintf("%d", time.Now().UnixNano())
134 ctx := context.WithValue(r.Context(), "request_id", requestID)
135
136 // Track request
137 app.metrics.mu.Lock()
138 app.metrics.requestCount++
139 app.metrics.mu.Unlock()
140
141 // Add response headers for debugging
142 w.Header().Set("X-Request-ID", requestID)
143
144 // Call next handler
145 next.ServeHTTP(w, r.WithContext(ctx))
146
147 // Update metrics
148 duration := time.Since(start)
149 app.metrics.mu.Lock()
150 app.metrics.avgResponseTime = (app.metrics.avgResponseTime + duration) / 2
151 app.metrics.mu.Unlock()
152
153 // Log request
154 app.logWithContext(ctx, "info", "request completed", map[string]interface{}{
155 "method": r.Method,
156 "path": r.URL.Path,
157 "duration": duration.String(),
158 })
159 })
160}
161
162// Graceful shutdown handler
163func (app *Application) gracefulShutdown(server *http.Server) {
164 quit := make(chan os.Signal, 1)
165 signal.Notify(quit, syscall.SIGINT, syscall.SIGTERM)
166
167 <-quit
168 log.Println("🛑 Shutting down server...")
169
170 ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
171 defer cancel()
172
173 if err := server.Shutdown(ctx); err != nil {
174 log.Printf("❌ Server forced to shutdown: %v", err)
175 }
176
177 log.Println("✅ Server exited")
178}
179
180func main() {
181 // Load configuration
182 config := &Config{
183 Port: getEnv("PORT", "8080"),
184 DebugEnabled: getEnv("DEBUG", "false") == "true",
185 DebugPort: getEnv("DEBUG_PORT", "2345"),
186 Environment: getEnv("ENVIRONMENT", "development"),
187 }
188
189 app := NewApplication(config)
190
191 // Setup routes
192 mux := http.NewServeMux()
193 mux.HandleFunc("/health", app.healthHandler)
194 mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
195 fmt.Fprintf(w, "Production Service - Environment: %s\n", config.Environment)
196 })
197
198 // Setup debug endpoints if enabled
199 app.setupDebugEndpoints(mux)
200
201 // Wrap with middleware
202 handler := app.trackingMiddleware(mux)
203
204 // Create server
205 server := &http.Server{
206 Addr: ":" + config.Port,
207 Handler: handler,
208 ReadTimeout: 15 * time.Second,
209 WriteTimeout: 15 * time.Second,
210 IdleTimeout: 60 * time.Second,
211 }
212
213 // Start server
214 go func() {
215 log.Printf("🚀 Server starting on :%s", config.Port)
216 log.Printf("Environment: %s", config.Environment)
217 if config.DebugEnabled {
218 log.Printf("⚠️ Debug mode enabled")
219 }
220
221 if err := server.ListenAndServe(); err != nil && err != http.ErrServerClosed {
222 log.Fatalf("❌ Server failed: %v", err)
223 }
224 }()
225
226 // Handle graceful shutdown
227 app.gracefulShutdown(server)
228}
229
230func getEnv(key, defaultValue string) string {
231 if value := os.Getenv(key); value != "" {
232 return value
233 }
234 return defaultValue
235}
Remote Debugging Setup:
1# Build with debug symbols
2go build -gcflags="all=-N -l" -o app remote_debug.go
3
4# Start Delve headless server (for remote debugging)
5dlv exec --headless --listen=:2345 --api-version=2 --accept-multiclient ./app
6
7# From another machine, connect to the debug server
8dlv connect <remote-ip>:2345
9
10# Or use VS Code with launch configuration:
11# .vscode/launch.json
12{
13 "version": "0.2.0",
14 "configurations": [
15 {
16 "name": "Connect to Remote Delve",
17 "type": "go",
18 "request": "attach",
19 "mode": "remote",
20 "remotePath": "",
21 "port": 2345,
22 "host": "<remote-ip>"
23 }
24 ]
25}
Debugging Deadlocks and Goroutine Leaks
Goroutine leaks and deadlocks are common in concurrent Go programs. Here's how to detect and fix them.
1// deadlock_debug.go - Detecting and preventing deadlocks
2package main
3
4import (
5 "context"
6 "fmt"
7 "net/http"
8 _ "net/http/pprof"
9 "runtime"
10 "sync"
11 "time"
12)
13
14// Example 1: Classic deadlock scenario
15func classicDeadlock() {
16 fmt.Println("\n=== Classic Deadlock Example ===")
17
18 var mu1, mu2 sync.Mutex
19
20 // Goroutine 1: locks mu1, then tries to lock mu2
21 go func() {
22 mu1.Lock()
23 fmt.Println("Goroutine 1: locked mu1")
24 time.Sleep(100 * time.Millisecond)
25
26 fmt.Println("Goroutine 1: trying to lock mu2...")
27 mu2.Lock()
28 fmt.Println("Goroutine 1: locked mu2")
29 mu2.Unlock()
30 mu1.Unlock()
31 }()
32
33 // Goroutine 2: locks mu2, then tries to lock mu1
34 go func() {
35 mu2.Lock()
36 fmt.Println("Goroutine 2: locked mu2")
37 time.Sleep(100 * time.Millisecond)
38
39 fmt.Println("Goroutine 2: trying to lock mu1...")
40 mu1.Lock()
41 fmt.Println("Goroutine 2: locked mu1")
42 mu1.Unlock()
43 mu2.Unlock()
44 }()
45
46 time.Sleep(2 * time.Second)
47 fmt.Println("⚠️ This will deadlock!")
48}
49
50// Example 2: Channel deadlock
51func channelDeadlock() {
52 fmt.Println("\n=== Channel Deadlock Example ===")
53
54 ch := make(chan int)
55
56 // This will block forever - no one is receiving
57 // Uncomment to see deadlock:
58 // ch <- 1
59
60 // Fixed version: use goroutine
61 go func() {
62 ch <- 1
63 }()
64
65 val := <-ch
66 fmt.Printf("Received: %d\n", val)
67}
68
69// Example 3: Goroutine leak detection
70type LeakDetector struct {
71 mu sync.Mutex
72 initialGoroutines int
73 leakThreshold int
74}
75
76func NewLeakDetector(threshold int) *LeakDetector {
77 return &LeakDetector{
78 initialGoroutines: runtime.NumGoroutine(),
79 leakThreshold: threshold,
80 }
81}
82
83func (ld *LeakDetector) Check() bool {
84 ld.mu.Lock()
85 defer ld.mu.Unlock()
86
87 current := runtime.NumGoroutine()
88 leaked := current - ld.initialGoroutines
89
90 if leaked > ld.leakThreshold {
91 fmt.Printf("⚠️ Goroutine leak detected! Initial: %d, Current: %d, Leaked: %d\n",
92 ld.initialGoroutines, current, leaked)
93
94 // Dump goroutine stack traces
95 buf := make([]byte, 1<<20)
96 stackLen := runtime.Stack(buf, true)
97 fmt.Printf("Goroutine stack trace:\n%s\n", buf[:stackLen])
98
99 return true
100 }
101
102 return false
103}
104
105// Example 4: Preventing deadlocks with timeouts
106func safeOperation(ctx context.Context, mu *sync.Mutex, operation func()) error {
107 // Try to acquire lock with timeout
108 locked := make(chan struct{})
109
110 go func() {
111 mu.Lock()
112 close(locked)
113 }()
114
115 select {
116 case <-locked:
117 defer mu.Unlock()
118 operation()
119 return nil
120
121 case <-time.After(5 * time.Second):
122 return fmt.Errorf("timeout waiting for lock")
123
124 case <-ctx.Done():
125 return ctx.Err()
126 }
127}
128
129// Example 5: Proper context cancellation
130func leakyWorker() {
131 fmt.Println("\n=== Leaky Worker Example ===")
132
133 // BAD: This goroutine will leak if we don't wait for it
134 go func() {
135 for {
136 time.Sleep(100 * time.Millisecond)
137 fmt.Println("Working...")
138 }
139 }()
140
141 time.Sleep(1 * time.Second)
142 fmt.Println("Main exits, but goroutine keeps running!")
143}
144
145func fixedWorker() {
146 fmt.Println("\n=== Fixed Worker Example ===")
147
148 ctx, cancel := context.WithCancel(context.Background())
149 defer cancel()
150
151 // GOOD: Use context for cancellation
152 go func(ctx context.Context) {
153 for {
154 select {
155 case <-ctx.Done():
156 fmt.Println("Worker stopping...")
157 return
158 case <-time.After(100 * time.Millisecond):
159 fmt.Println("Working...")
160 }
161 }
162 }(ctx)
163
164 time.Sleep(1 * time.Second)
165 fmt.Println("Cancelling worker...")
166 cancel()
167 time.Sleep(200 * time.Millisecond)
168 fmt.Println("Worker stopped gracefully!")
169}
170
171// Example 6: Deadlock detector using mutex ordering
172type OrderedMutex struct {
173 id int
174 mutex sync.Mutex
175}
176
177type LockManager struct {
178 mu sync.Mutex
179 lockOrder map[int][]int // goroutine ID -> acquired lock IDs
180 detectedCycle bool
181}
182
183func NewLockManager() *LockManager {
184 return &LockManager{
185 lockOrder: make(map[int][]int),
186 }
187}
188
189func (lm *LockManager) AcquireLock(goroutineID int, lockID int) error {
190 lm.mu.Lock()
191 defer lm.mu.Unlock()
192
193 // Check for potential deadlock
194 currentLocks := lm.lockOrder[goroutineID]
195 for _, heldLock := range currentLocks {
196 if heldLock > lockID {
197 return fmt.Errorf("potential deadlock: attempting to acquire lock %d while holding lock %d", lockID, heldLock)
198 }
199 }
200
201 lm.lockOrder[goroutineID] = append(currentLocks, lockID)
202 return nil
203}
204
205func (lm *LockManager) ReleaseLock(goroutineID int, lockID int) {
206 lm.mu.Lock()
207 defer lm.mu.Unlock()
208
209 locks := lm.lockOrder[goroutineID]
210 for i, id := range locks {
211 if id == lockID {
212 lm.lockOrder[goroutineID] = append(locks[:i], locks[i+1:]...)
213 break
214 }
215 }
216}
217
218func main() {
219 // Start pprof server for runtime analysis
220 go func() {
221 fmt.Println("pprof server running on :6060")
222 fmt.Println("Goroutine profile: http://localhost:6060/debug/pprof/goroutine")
223 http.ListenAndServe(":6060", nil)
224 }()
225
226 // Create leak detector
227 detector := NewLeakDetector(10)
228
229 // Demonstrate different deadlock scenarios
230 fmt.Println("=== Deadlock and Goroutine Leak Detection ===")
231
232 // Example 1: Uncomment to see classic deadlock
233 // classicDeadlock()
234
235 // Example 2: Channel deadlock
236 channelDeadlock()
237
238 // Example 3: Goroutine leak
239 fmt.Println("\nCreating goroutines...")
240 for i := 0; i < 20; i++ {
241 go func() {
242 time.Sleep(10 * time.Second)
243 }()
244 }
245
246 time.Sleep(100 * time.Millisecond)
247 detector.Check()
248
249 // Example 4: Safe operation with timeout
250 fmt.Println("\n=== Safe Operation with Timeout ===")
251 var mu sync.Mutex
252 ctx := context.Background()
253
254 err := safeOperation(ctx, &mu, func() {
255 fmt.Println("Operation completed safely")
256 })
257 if err != nil {
258 fmt.Printf("Error: %v\n", err)
259 }
260
261 // Example 5: Proper context cancellation
262 fixedWorker()
263
264 fmt.Println("\n=== Monitoring Goroutines ===")
265 fmt.Printf("Current goroutines: %d\n", runtime.NumGoroutine())
266 fmt.Println("Check goroutine profile at: http://localhost:6060/debug/pprof/goroutine")
267
268 // Keep main running for monitoring
269 select {}
270}
Performance Debugging Strategies
Identifying Memory Allocation Hotspots
Memory allocations can significantly impact performance. Here's how to find and optimize them.
1// alloc_optimize.go - Finding and fixing allocation hotspots
2package main
3
4import (
5 "fmt"
6 "runtime"
7 "strings"
8 "testing"
9)
10
11// BAD: Excessive allocations in string concatenation
12func inefficientStringConcat(items []string) string {
13 result := ""
14 for _, item := range items {
15 result += item + ", " // Each += creates a new string
16 }
17 return result
18}
19
20// GOOD: Use strings.Builder for efficient concatenation
21func efficientStringConcat(items []string) string {
22 var builder strings.Builder
23 for i, item := range items {
24 builder.WriteString(item)
25 if i < len(items)-1 {
26 builder.WriteString(", ")
27 }
28 }
29 return builder.String()
30}
31
32// BAD: Slice growth causes repeated allocations
33func inefficientSliceGrowth() []int {
34 var result []int
35 for i := 0; i < 10000; i++ {
36 result = append(result, i) // Repeated reallocations
37 }
38 return result
39}
40
41// GOOD: Pre-allocate slice with make
42func efficientSliceGrowth() []int {
43 result := make([]int, 0, 10000) // Pre-allocate capacity
44 for i := 0; i < 10000; i++ {
45 result = append(result, i)
46 }
47 return result
48}
49
50// BAD: Creating many small allocations
51func inefficientSmallAllocs(n int) [][]byte {
52 result := make([][]byte, n)
53 for i := 0; i < n; i++ {
54 result[i] = []byte{byte(i)} // Many small allocations
55 }
56 return result
57}
58
59// GOOD: Use a single large allocation
60func efficientBulkAlloc(n int) []byte {
61 return make([]byte, n) // Single allocation
62}
63
64// Memory usage reporter
65func reportMemStats(label string) {
66 var m runtime.MemStats
67 runtime.ReadMemStats(&m)
68
69 fmt.Printf("\n=== %s ===\n", label)
70 fmt.Printf("Alloc: %d MB\n", m.Alloc/1024/1024)
71 fmt.Printf("TotalAlloc: %d MB\n", m.TotalAlloc/1024/1024)
72 fmt.Printf("Sys: %d MB\n", m.Sys/1024/1024)
73 fmt.Printf("NumGC: %d\n", m.NumGC)
74 fmt.Printf("Mallocs: %d\n", m.Mallocs)
75 fmt.Printf("Frees: %d\n", m.Frees)
76}
77
78func main() {
79 reportMemStats("Initial")
80
81 // Test string concatenation
82 items := make([]string, 1000)
83 for i := 0; i < 1000; i++ {
84 items[i] = fmt.Sprintf("item%d", i)
85 }
86
87 // Inefficient version
88 runtime.GC()
89 reportMemStats("Before Inefficient String Concat")
90 _ = inefficientStringConcat(items)
91 reportMemStats("After Inefficient String Concat")
92
93 // Efficient version
94 runtime.GC()
95 reportMemStats("Before Efficient String Concat")
96 _ = efficientStringConcat(items)
97 reportMemStats("After Efficient String Concat")
98
99 // Test slice growth
100 runtime.GC()
101 reportMemStats("Before Inefficient Slice Growth")
102 _ = inefficientSliceGrowth()
103 reportMemStats("After Inefficient Slice Growth")
104
105 runtime.GC()
106 reportMemStats("Before Efficient Slice Growth")
107 _ = efficientSliceGrowth()
108 reportMemStats("After Efficient Slice Growth")
109}
110
111// Benchmark functions
112func BenchmarkInefficientStringConcat(b *testing.B) {
113 items := make([]string, 100)
114 for i := 0; i < 100; i++ {
115 items[i] = fmt.Sprintf("item%d", i)
116 }
117
118 b.ResetTimer()
119 for i := 0; i < b.N; i++ {
120 _ = inefficientStringConcat(items)
121 }
122}
123
124func BenchmarkEfficientStringConcat(b *testing.B) {
125 items := make([]string, 100)
126 for i := 0; i < 100; i++ {
127 items[i] = fmt.Sprintf("item%d", i)
128 }
129
130 b.ResetTimer()
131 for i := 0; i < b.N; i++ {
132 _ = efficientStringConcat(items)
133 }
134}
135
136func BenchmarkInefficientSliceGrowth(b *testing.B) {
137 for i := 0; i < b.N; i++ {
138 _ = inefficientSliceGrowth()
139 }
140}
141
142func BenchmarkEfficientSliceGrowth(b *testing.B) {
143 for i := 0; i < b.N; i++ {
144 _ = efficientSliceGrowth()
145 }
146}
Running Allocation Benchmarks:
1# Run benchmarks with memory allocation stats
2go test -bench=. -benchmem alloc_optimize.go
3
4# Output shows allocations per operation:
5# BenchmarkInefficientStringConcat-8 5000 300000 ns/op 500000 B/op 1000 allocs/op
6# BenchmarkEfficientStringConcat-8 50000 30000 ns/op 10000 B/op 10 allocs/op
7
8# Profile allocations
9go test -bench=. -memprofile=mem.prof alloc_optimize.go
10go tool pprof mem.prof
11
12# In pprof:
13(pprof) list inefficientStringConcat # See line-by-line allocations
14(pprof) top # Top allocation functions
Practice Exercises
Exercise 1: Debug Memory Leak in Web Service
Build a web service that demonstrates memory leak patterns and practice debugging with pprof.
Requirements:
- HTTP server with multiple endpoints
- Memory allocation that grows over time
- Integration with pprof for analysis
- Identification and fixing of memory leaks
Starter Code:
1// Your task: Complete this implementation and debug memory issues
2package main
3
4import (
5 "net/http"
6 "sync"
7 "time"
8)
9
10type Cache struct {
11 data map[string][]byte
12 mu sync.RWMutex
13}
14
15func NewCache() *Cache {
16 return &Cache{
17 data: make(map[string][]byte),
18 }
19}
20
21func Add(key string, value []byte) {
22 // TODO: Implement with proper cleanup
23 c.mu.Lock()
24 defer c.mu.Unlock()
25 c.data[key] = value
26}
27
28func Get(key string) []byte {
29 c.mu.RLock()
30 defer c.mu.RUnlock()
31 return c.data[key]
32}
33
34// TODO: Add HTTP handlers that use the cache
35// TODO: Add pprof endpoints
36// TODO: Implement memory leak detection
37// TODO: Fix identified issues
38
39func main() {
40 cache := NewCache()
41
42 // TODO: Start HTTP server with cache operations
43 // TODO: Add endpoints that stress test the cache
44 // TODO: Enable pprof for debugging
45}
Debugging Tasks:
- Run the service and monitor memory usage
- Use
curl http://localhost:8080/debug/pprof/heapto capture profiles - Analyze profiles with
go tool pprof - Identify memory allocation patterns
- Implement fixes for identified issues
- Verify fixes by re-profiling
Solution
1package main
2
3import (
4 "fmt"
5 "net/http"
6 _ "net/http/pprof"
7 "runtime"
8 "sync"
9 "time"
10)
11
12type Cache struct {
13 data map[string][]byte
14 mu sync.RWMutex
15 maxSize int
16 hits int64
17 misses int64
18}
19
20func NewCache() *Cache {
21 return &Cache{
22 data: make(map[string][]byte),
23 maxSize: 1000, // Limit cache size
24 }
25}
26
27func Add(key string, value []byte) {
28 c.mu.Lock()
29 defer c.mu.Unlock()
30
31 // Check if we need to evict items
32 if len(c.data) >= c.maxSize {
33 c.evictLRU()
34 }
35
36 c.data[key] = value
37}
38
39func Get(key string) []byte {
40 c.mu.RLock()
41 defer c.mu.RUnlock()
42
43 if value, exists := c.data[key]; exists {
44 atomic.AddInt64(&c.hits, 1)
45 return value
46 }
47
48 atomic.AddInt64(&c.misses, 1)
49 return nil
50}
51
52func evictLRU() {
53 // Simple eviction: remove oldest items
54 // In production, you'd implement proper LRU tracking
55 toRemove := len(c.data) - c.maxSize + 100
56 count := 0
57 for k := range c.data {
58 if count >= toRemove {
59 break
60 }
61 delete(c.data, k)
62 count++
63 }
64}
65
66func Stats() map[string]interface{} {
67 return map[string]interface{}{
68 "size": len(c.data),
69 "max_size": c.maxSize,
70 "hits": c.hits,
71 "misses": c.misses,
72 "hit_rate": float64(c.hits) / float64(c.hits+c.misses) * 100,
73 }
74}
75
76func main() {
77 cache := NewCache()
78
79 // Add metrics handler
80 http.HandleFunc("/cache/stats", func(w http.ResponseWriter, r *http.Request) {
81 w.Header().Set("Content-Type", "application/json")
82 json.NewEncoder(w).Encode(cache.Stats())
83 })
84
85 // Add cache operations
86 http.HandleFunc("/cache/add", func(w http.ResponseWriter, r *http.Request) {
87 key := r.URL.Query().Get("key")
88 value := r.URL.Query().Get("value")
89
90 if key == "" || value == "" {
91 http.Error(w, "key and value required", 400)
92 return
93 }
94
95 cache.Add(key, []byte(value))
96 fmt.Fprintf(w, "Added %s to cache", key)
97 })
98
99 http.HandleFunc("/cache/get", func(w http.ResponseWriter, r *http.Request) {
100 key := r.URL.Query().Get("key")
101 value := cache.Get(key)
102
103 if value == nil {
104 fmt.Fprintf(w, "Cache miss for %s", key)
105 return
106 }
107
108 fmt.Fprintf(w, "Cache hit for %s: %s", key, string(value))
109 })
110
111 // Stress test endpoint
112 http.HandleFunc("/cache/stress", func(w http.ResponseWriter, r *http.Request) {
113 go func() {
114 for i := 0; i < 10000; i++ {
115 key := fmt.Sprintf("key-%d", i%1000)
116 value := make([]byte, 1024) // 1KB values
117 cache.Add(key, value)
118
119 if i%10 == 0 {
120 // Random access pattern
121 randomKey := fmt.Sprintf("key-%d", rand.Intn(1000))
122 cache.Get(randomKey)
123 }
124 }
125 }()
126
127 fmt.Fprint(w, "Stress test started")
128 })
129
130 // Memory usage endpoint
131 http.HandleFunc("/debug/memory", func(w http.ResponseWriter, r *http.Request) {
132 var m runtime.MemStats
133 runtime.ReadMemStats(&m)
134
135 w.Header().Set("Content-Type", "application/json")
136 json.NewEncoder(w).Encode(map[string]interface{}{
137 "alloc_mb": m.Alloc / 1024 / 1024,
138 "total_alloc_mb": m.TotalAlloc / 1024 / 1024,
139 "sys_mb": m.Sys / 1024 / 1024,
140 "num_gc": m.NumGC,
141 "goroutines": runtime.NumGoroutine(),
142 })
143 })
144
145 fmt.Println("Cache server running on :8080")
146 fmt.Println("Endpoints:")
147 fmt.Println(" /cache/stats - Cache statistics")
148 fmt.Println(" /cache/add?key=test&value=data - Add to cache")
149 fmt.Println(" /cache/get?key=test - Get from cache")
150 fmt.Println(" /cache/stress - Stress test cache")
151 fmt.Println(" /debug/memory - Memory usage")
152 fmt.Println(" /debug/pprof/ - pprof endpoints")
153}
Exercise 2: CPU Profiling and Optimization
Profile and optimize a CPU-intensive application.
Requirements:
- Identify CPU hotspots using pprof
- Compare multiple algorithm implementations
- Measure performance improvements
- Generate before/after benchmarks
Starter Code:
1// Your task: Profile and optimize this CPU-intensive code
2package main
3
4import (
5 "fmt"
6 "math"
7)
8
9// TODO: Profile this function and identify bottlenecks
10func calculatePrimes(max int) []int {
11 var primes []int
12 for n := 2; n <= max; n++ {
13 isPrime := true
14 for i := 2; i < n; i++ {
15 if n%i == 0 {
16 isPrime = false
17 break
18 }
19 }
20 if isPrime {
21 primes = append(primes, n)
22 }
23 }
24 return primes
25}
26
27func main() {
28 // TODO: Add CPU profiling
29 // TODO: Run benchmarks
30 // TODO: Optimize based on profile results
31 result := calculatePrimes(10000)
32 fmt.Printf("Found %d primes\n", len(result))
33}
Solution
1package main
2
3import (
4 "fmt"
5 "math"
6 "os"
7 "runtime/pprof"
8 "testing"
9 "time"
10)
11
12// Optimized version using Sieve of Eratosthenes
13func calculatePrimesOptimized(max int) []int {
14 if max < 2 {
15 return []int{}
16 }
17
18 // Create boolean array for sieve
19 isPrime := make([]bool, max+1)
20 for i := 2; i <= max; i++ {
21 isPrime[i] = true
22 }
23
24 // Sieve of Eratosthenes
25 for i := 2; i*i <= max; i++ {
26 if isPrime[i] {
27 for j := i * i; j <= max; j += i {
28 isPrime[j] = false
29 }
30 }
31 }
32
33 // Collect primes
34 primes := make([]int, 0, max/10) // Estimate capacity
35 for i := 2; i <= max; i++ {
36 if isPrime[i] {
37 primes = append(primes, i)
38 }
39 }
40
41 return primes
42}
43
44// Further optimized version with square root optimization
45func calculatePrimesFast(max int) []int {
46 if max < 2 {
47 return []int{}
48 }
49
50 primes := make([]int, 0, max/10)
51 isPrime := make([]bool, max+1)
52 for i := 2; i <= max; i++ {
53 isPrime[i] = true
54 }
55
56 sqrtMax := int(math.Sqrt(float64(max)))
57 for i := 2; i <= sqrtMax; i++ {
58 if isPrime[i] {
59 for j := i * i; j <= max; j += i {
60 isPrime[j] = false
61 }
62 }
63 }
64
65 for i := 2; i <= max; i++ {
66 if isPrime[i] {
67 primes = append(primes, i)
68 }
69 }
70
71 return primes
72}
73
74func main() {
75 // Enable CPU profiling
76 f, err := os.Create("cpu.prof")
77 if err != nil {
78 panic(err)
79 }
80 defer f.Close()
81
82 if err := pprof.StartCPUProfile(f); err != nil {
83 panic(err)
84 }
85 defer pprof.StopCPUProfile()
86
87 // Benchmark original version
88 fmt.Println("Running benchmarks...")
89
90 start := time.Now()
91 primes1 := calculatePrimes(10000)
92 duration1 := time.Since(start)
93 fmt.Printf("Original algorithm: Found %d primes in %v\n", len(primes1), duration1)
94
95 start = time.Now()
96 primes2 := calculatePrimesOptimized(10000)
97 duration2 := time.Since(start)
98 fmt.Printf("Optimized algorithm: Found %d primes in %v\n", len(primes2), duration2)
99
100 start = time.Now()
101 primes3 := calculatePrimesFast(10000)
102 duration3 := time.Since(start)
103 fmt.Printf("Fast algorithm: Found %d primes in %v\n", len(primes3), duration3)
104
105 fmt.Printf("\nSpeedup: %.2fx faster (optimized vs original)\n", float64(duration1)/float64(duration2))
106 fmt.Printf("Speedup: %.2fx faster (fast vs original)\n", float64(duration1)/float64(duration3))
107
108 fmt.Println("\nAnalyze CPU profile with: go tool pprof cpu.prof")
109}
110
111// Benchmark functions
112func BenchmarkOriginal(b *testing.B) {
113 for i := 0; i < b.N; i++ {
114 calculatePrimes(10000)
115 }
116}
117
118func BenchmarkOptimized(b *testing.B) {
119 for i := 0; i < b.N; i++ {
120 calculatePrimesOptimized(10000)
121 }
122}
123
124func BenchmarkFast(b *testing.B) {
125 for i := 0; i < b.N; i++ {
126 calculatePrimesFast(10000)
127 }
128}
Analysis Steps:
- Run with profiling:
go run cpu_profile.go - Analyze profile:
go tool pprof cpu.prof - Run benchmarks:
go test -bench=. -benchmem - Compare results and identify hotspots
Exercise 3: Race Condition Detection and Fix
Find and fix race conditions in concurrent code.
Requirements:
- Use race detector to identify data races
- Fix race conditions using proper synchronization
- Verify fixes with stress testing
- Document the race patterns found
Starter Code:
1// Your task: Find and fix all race conditions
2package main
3
4import (
5 "fmt"
6 "sync"
7)
8
9type Stats struct {
10 count int
11 total int64
12}
13
14var globalStats Stats
15
16func updateStats(value int) {
17 // TODO: This has race conditions - fix them
18 globalStats.count++
19 globalStats.total += int64(value)
20}
21
22func getAverage() float64 {
23 // TODO: This has race conditions - fix them
24 if globalStats.count == 0 {
25 return 0
26 }
27 return float64(globalStats.total) / float64(globalStats.count)
28}
29
30func main() {
31 var wg sync.WaitGroup
32
33 // Concurrent updates
34 for i := 0; i < 100; i++ {
35 wg.Add(1)
36 go func(val int) {
37 defer wg.Done()
38 updateStats(val)
39 }(i)
40 }
41
42 wg.Wait()
43 fmt.Printf("Average: %.2f\n", getAverage())
44}
Solution
1package main
2
3import (
4 "fmt"
5 "sync"
6 "sync/atomic"
7)
8
9// Solution 1: Using mutex
10type StatsMutex struct {
11 mu sync.RWMutex
12 count int
13 total int64
14}
15
16func (s *StatsMutex) Update(value int) {
17 s.mu.Lock()
18 defer s.mu.Unlock()
19 s.count++
20 s.total += int64(value)
21}
22
23func (s *StatsMutex) GetAverage() float64 {
24 s.mu.RLock()
25 defer s.mu.RUnlock()
26
27 if s.count == 0 {
28 return 0
29 }
30 return float64(s.total) / float64(s.count)
31}
32
33// Solution 2: Using atomic operations
34type StatsAtomic struct {
35 count int64
36 total int64
37}
38
39func (s *StatsAtomic) Update(value int) {
40 atomic.AddInt64(&s.count, 1)
41 atomic.AddInt64(&s.total, int64(value))
42}
43
44func (s *StatsAtomic) GetAverage() float64 {
45 count := atomic.LoadInt64(&s.count)
46 if count == 0 {
47 return 0
48 }
49 total := atomic.LoadInt64(&s.total)
50 return float64(total) / float64(count)
51}
52
53// Solution 3: Using channels (actor pattern)
54type StatsChannel struct {
55 updates chan int
56 queries chan chan float64
57 stop chan struct{}
58}
59
60func NewStatsChannel() *StatsChannel {
61 s := &StatsChannel{
62 updates: make(chan int, 100),
63 queries: make(chan chan float64),
64 stop: make(chan struct{}),
65 }
66
67 go s.run()
68 return s
69}
70
71func (s *StatsChannel) run() {
72 var count int
73 var total int64
74
75 for {
76 select {
77 case val := <-s.updates:
78 count++
79 total += int64(val)
80
81 case resp := <-s.queries:
82 var avg float64
83 if count > 0 {
84 avg = float64(total) / float64(count)
85 }
86 resp <- avg
87
88 case <-s.stop:
89 return
90 }
91 }
92}
93
94func (s *StatsChannel) Update(value int) {
95 s.updates <- value
96}
97
98func (s *StatsChannel) GetAverage() float64 {
99 resp := make(chan float64)
100 s.queries <- resp
101 return <-resp
102}
103
104func (s *StatsChannel) Stop() {
105 close(s.stop)
106}
107
108func main() {
109 fmt.Println("=== Testing Race-Free Implementations ===")
110
111 // Test mutex version
112 fmt.Println("\n1. Mutex-based implementation:")
113 testMutex()
114
115 // Test atomic version
116 fmt.Println("\n2. Atomic-based implementation:")
117 testAtomic()
118
119 // Test channel version
120 fmt.Println("\n3. Channel-based implementation:")
121 testChannel()
122}
123
124func testMutex() {
125 stats := &StatsMutex{}
126 var wg sync.WaitGroup
127
128 for i := 0; i < 1000; i++ {
129 wg.Add(1)
130 go func(val int) {
131 defer wg.Done()
132 stats.Update(val)
133 }(i)
134 }
135
136 wg.Wait()
137 fmt.Printf("Average: %.2f\n", stats.GetAverage())
138}
139
140func testAtomic() {
141 stats := &StatsAtomic{}
142 var wg sync.WaitGroup
143
144 for i := 0; i < 1000; i++ {
145 wg.Add(1)
146 go func(val int) {
147 defer wg.Done()
148 stats.Update(val)
149 }(i)
150 }
151
152 wg.Wait()
153 fmt.Printf("Average: %.2f\n", stats.GetAverage())
154}
155
156func testChannel() {
157 stats := NewStatsChannel()
158 defer stats.Stop()
159
160 var wg sync.WaitGroup
161
162 for i := 0; i < 1000; i++ {
163 wg.Add(1)
164 go func(val int) {
165 defer wg.Done()
166 stats.Update(val)
167 }(i)
168 }
169
170 wg.Wait()
171 fmt.Printf("Average: %.2f\n", stats.GetAverage())
172}
Testing for races:
1# Run with race detector
2go run -race race_fix.go
3
4# Should show no race warnings in corrected version
5
6# Stress test
7go test -race -count=100 .
Exercise 4: Goroutine Leak Detection
Build a system to detect and prevent goroutine leaks.
Requirements:
- Implement goroutine leak detector
- Create examples of common leak patterns
- Fix leaks using context and proper shutdown
- Add monitoring for long-running goroutines
Starter Code:
1// Your task: Implement goroutine leak detection and prevention
2package main
3
4import (
5 "time"
6)
7
8// TODO: Implement LeakDetector
9type LeakDetector struct {
10 // Add fields
11}
12
13// TODO: Implement monitoring
14func monitor() {
15 // Start a goroutine that never stops
16 go func() {
17 for {
18 time.Sleep(time.Second)
19 // Doing work...
20 }
21 }()
22}
23
24func main() {
25 // TODO: Detect goroutine leaks
26 // TODO: Fix leaky goroutines
27 monitor()
28 time.Sleep(5 * time.Second)
29}
Solution
1package main
2
3import (
4 "context"
5 "fmt"
6 "runtime"
7 "sync"
8 "time"
9)
10
11// Comprehensive goroutine leak detector
12type LeakDetector struct {
13 mu sync.Mutex
14 baseline int
15 threshold int
16 checkInterval time.Duration
17 goroutineSnapshots []int
18 maxSnapshots int
19}
20
21func NewLeakDetector(threshold int) *LeakDetector {
22 return &LeakDetector{
23 baseline: runtime.NumGoroutine(),
24 threshold: threshold,
25 checkInterval: time.Second,
26 maxSnapshots: 10,
27 }
28}
29
30func (ld *LeakDetector) Start(ctx context.Context) {
31 ticker := time.NewTicker(ld.checkInterval)
32 defer ticker.Stop()
33
34 for {
35 select {
36 case <-ticker.C:
37 ld.check()
38 case <-ctx.Done():
39 fmt.Println("Leak detector stopped")
40 return
41 }
42 }
43}
44
45func (ld *LeakDetector) check() {
46 ld.mu.Lock()
47 defer ld.mu.Unlock()
48
49 current := runtime.NumGoroutine()
50 leaked := current - ld.baseline
51
52 // Record snapshot
53 ld.goroutineSnapshots = append(ld.goroutineSnapshots, current)
54 if len(ld.goroutineSnapshots) > ld.maxSnapshots {
55 ld.goroutineSnapshots = ld.goroutineSnapshots[1:]
56 }
57
58 if leaked > ld.threshold {
59 fmt.Printf("\n⚠️ GOROUTINE LEAK DETECTED!\n")
60 fmt.Printf("Baseline: %d, Current: %d, Leaked: %d\n", ld.baseline, current, leaked)
61 fmt.Printf("Recent snapshots: %v\n", ld.goroutineSnapshots)
62
63 // Dump stack traces
64 buf := make([]byte, 1<<16)
65 stackLen := runtime.Stack(buf, true)
66 fmt.Printf("\nStack trace:\n%s\n", buf[:stackLen])
67 }
68}
69
70func (ld *LeakDetector) GetStats() map[string]interface{} {
71 ld.mu.Lock()
72 defer ld.mu.Unlock()
73
74 current := runtime.NumGoroutine()
75 return map[string]interface{}{
76 "baseline": ld.baseline,
77 "current": current,
78 "leaked": current - ld.baseline,
79 "threshold": ld.threshold,
80 "snapshots": ld.goroutineSnapshots,
81 }
82}
83
84// BAD: Leaky worker without cancellation
85func leakyWorker() {
86 go func() {
87 for {
88 time.Sleep(100 * time.Millisecond)
89 // No way to stop this goroutine!
90 }
91 }()
92}
93
94// GOOD: Properly cancellable worker
95func goodWorker(ctx context.Context, wg *sync.WaitGroup) {
96 wg.Add(1)
97 go func() {
98 defer wg.Done()
99
100 ticker := time.NewTicker(100 * time.Millisecond)
101 defer ticker.Stop()
102
103 for {
104 select {
105 case <-ticker.C:
106 // Do work
107 case <-ctx.Done():
108 fmt.Println("Worker stopping gracefully")
109 return
110 }
111 }
112 }()
113}
114
115// BAD: Leaky HTTP request without timeout
116func leakyHTTPRequest() {
117 go func() {
118 // Blocks forever if server doesn't respond
119 // No timeout, no cancellation
120 }()
121}
122
123// GOOD: HTTP request with context and timeout
124func goodHTTPRequest(ctx context.Context) error {
125 ctx, cancel := context.WithTimeout(ctx, 5*time.Second)
126 defer cancel()
127
128 done := make(chan error, 1)
129
130 go func() {
131 // Simulate HTTP request
132 time.Sleep(3 * time.Second)
133 done <- nil
134 }()
135
136 select {
137 case err := <-done:
138 return err
139 case <-ctx.Done():
140 return ctx.Err()
141 }
142}
143
144// Worker pool with proper lifecycle management
145type WorkerPool struct {
146 workers int
147 jobs chan int
148 wg sync.WaitGroup
149 ctx context.Context
150 cancel context.CancelFunc
151}
152
153func NewWorkerPool(workers int) *WorkerPool {
154 ctx, cancel := context.WithCancel(context.Background())
155
156 pool := &WorkerPool{
157 workers: workers,
158 jobs: make(chan int, 100),
159 ctx: ctx,
160 cancel: cancel,
161 }
162
163 pool.start()
164 return pool
165}
166
167func (wp *WorkerPool) start() {
168 for i := 0; i < wp.workers; i++ {
169 wp.wg.Add(1)
170 go wp.worker(i)
171 }
172}
173
174func (wp *WorkerPool) worker(id int) {
175 defer wp.wg.Done()
176
177 for {
178 select {
179 case job, ok := <-wp.jobs:
180 if !ok {
181 fmt.Printf("Worker %d: job channel closed\n", id)
182 return
183 }
184 // Process job
185 time.Sleep(50 * time.Millisecond)
186 fmt.Printf("Worker %d processed job %d\n", id, job)
187
188 case <-wp.ctx.Done():
189 fmt.Printf("Worker %d: context cancelled\n", id)
190 return
191 }
192 }
193}
194
195func (wp *WorkerPool) Submit(job int) {
196 select {
197 case wp.jobs <- job:
198 case <-wp.ctx.Done():
199 fmt.Println("Cannot submit job: pool is shutting down")
200 }
201}
202
203func (wp *WorkerPool) Shutdown() {
204 fmt.Println("Shutting down worker pool...")
205
206 // Cancel context to signal workers
207 wp.cancel()
208
209 // Close job channel
210 close(wp.jobs)
211
212 // Wait for all workers to finish
213 wp.wg.Wait()
214
215 fmt.Println("Worker pool shut down complete")
216}
217
218func main() {
219 fmt.Println("=== Goroutine Leak Detection Demo ===")
220
221 // Start leak detector
222 ctx, cancel := context.WithCancel(context.Background())
223 defer cancel()
224
225 detector := NewLeakDetector(5)
226 go detector.Start(ctx)
227
228 // Initial state
229 fmt.Printf("Initial goroutines: %d\n", runtime.NumGoroutine())
230
231 // Example 1: Create some leaky goroutines
232 fmt.Println("\n1. Creating leaky goroutines...")
233 for i := 0; i < 10; i++ {
234 leakyWorker()
235 }
236 time.Sleep(2 * time.Second)
237 stats := detector.GetStats()
238 fmt.Printf("After leaky workers: %+v\n", stats)
239
240 // Example 2: Create properly managed goroutines
241 fmt.Println("\n2. Creating properly managed goroutines...")
242 workerCtx, workerCancel := context.WithCancel(context.Background())
243 var wg sync.WaitGroup
244
245 for i := 0; i < 10; i++ {
246 goodWorker(workerCtx, &wg)
247 }
248
249 time.Sleep(1 * time.Second)
250
251 // Shut down good workers
252 fmt.Println("\n3. Shutting down good workers...")
253 workerCancel()
254 wg.Wait()
255 time.Sleep(1 * time.Second)
256
257 // Example 3: Worker pool with proper lifecycle
258 fmt.Println("\n4. Testing worker pool...")
259 pool := NewWorkerPool(3)
260
261 // Submit jobs
262 for i := 0; i < 20; i++ {
263 pool.Submit(i)
264 }
265
266 time.Sleep(2 * time.Second)
267
268 // Shutdown pool
269 pool.Shutdown()
270
271 time.Sleep(1 * time.Second)
272
273 // Final state
274 fmt.Printf("\nFinal goroutines: %d\n", runtime.NumGoroutine())
275 fmt.Printf("Final stats: %+v\n", detector.GetStats())
276
277 // Keep running to observe leaks
278 fmt.Println("\nMonitoring for 5 more seconds...")
279 time.Sleep(5 * time.Second)
280}
Exercise 5: Production Debugging Dashboard
Create a comprehensive debugging dashboard for production applications.
Requirements:
- HTTP endpoints for runtime metrics
- Memory, CPU, and goroutine monitoring
- pprof integration
- Health check endpoints
- Request tracing support
Starter Code:
1// Your task: Build a complete debugging dashboard
2package main
3
4import (
5 "net/http"
6)
7
8func main() {
9 // TODO: Add debug endpoints
10 // TODO: Add metrics collection
11 // TODO: Add pprof endpoints
12 // TODO: Add health checks
13 http.ListenAndServe(":8080", nil)
14}
Solution
1package main
2
3import (
4 "context"
5 "encoding/json"
6 "fmt"
7 "log"
8 "net/http"
9 _ "net/http/pprof"
10 "os"
11 "os/signal"
12 "runtime"
13 "sync"
14 "sync/atomic"
15 "syscall"
16 "time"
17)
18
19// Comprehensive debugging dashboard
20type DebugDashboard struct {
21 startTime time.Time
22 requestCount int64
23 errorCount int64
24 mu sync.RWMutex
25 metrics map[string]interface{}
26}
27
28func NewDebugDashboard() *DebugDashboard {
29 return &DebugDashboard{
30 startTime: time.Now(),
31 metrics: make(map[string]interface{}),
32 }
33}
34
35// Middleware to track requests
36func (dd *DebugDashboard) TrackingMiddleware(next http.Handler) http.Handler {
37 return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
38 atomic.AddInt64(&dd.requestCount, 1)
39 start := time.Now()
40
41 // Wrap response writer to capture status
42 wrapped := &responseWriter{ResponseWriter: w, statusCode: 200}
43 next.ServeHTTP(wrapped, r)
44
45 duration := time.Since(start)
46
47 // Track errors
48 if wrapped.statusCode >= 400 {
49 atomic.AddInt64(&dd.errorCount, 1)
50 }
51
52 // Log request
53 log.Printf("[%s] %s %s - %d (%v)",
54 r.Method, r.URL.Path, r.RemoteAddr, wrapped.statusCode, duration)
55 })
56}
57
58type responseWriter struct {
59 http.ResponseWriter
60 statusCode int
61}
62
63func (rw *responseWriter) WriteHeader(code int) {
64 rw.statusCode = code
65 rw.ResponseWriter.WriteHeader(code)
66}
67
68// Dashboard homepage
69func (dd *DebugDashboard) HomeHandler(w http.ResponseWriter, r *http.Request) {
70 w.Header().Set("Content-Type", "text/html")
71 html := `
72<!DOCTYPE html>
73<html>
74<head>
75 <title>Debug Dashboard</title>
76 <style>
77 body { font-family: Arial, sans-serif; margin: 20px; }
78 h1 { color: #333; }
79 .endpoint { background: #f0f0f0; padding: 10px; margin: 10px 0; border-radius: 5px; }
80 .endpoint a { color: #0066cc; text-decoration: none; }
81 .endpoint a:hover { text-decoration: underline; }
82 </style>
83</head>
84<body>
85 <h1>🔍 Debug Dashboard</h1>
86 <h2>Available Endpoints:</h2>
87
88 <div class="endpoint">
89 <h3><a href="/debug/health">Health Check</a></h3>
90 <p>Application health status and uptime</p>
91 </div>
92
93 <div class="endpoint">
94 <h3><a href="/debug/metrics">Metrics</a></h3>
95 <p>Runtime metrics (goroutines, memory, requests)</p>
96 </div>
97
98 <div class="endpoint">
99 <h3><a href="/debug/memory">Memory Stats</a></h3>
100 <p>Detailed memory allocation statistics</p>
101 </div>
102
103 <div class="endpoint">
104 <h3><a href="/debug/goroutines">Goroutines</a></h3>
105 <p>Current goroutine count and stack traces</p>
106 </div>
107
108 <div class="endpoint">
109 <h3><a href="/debug/pprof/">pprof Index</a></h3>
110 <p>Go pprof profiling endpoints</p>
111 </div>
112
113 <div class="endpoint">
114 <h3><a href="/debug/pprof/heap">Heap Profile</a></h3>
115 <p>Memory allocation profile</p>
116 </div>
117
118 <div class="endpoint">
119 <h3><a href="/debug/pprof/goroutine">Goroutine Profile</a></h3>
120 <p>Detailed goroutine analysis</p>
121 </div>
122
123 <div class="endpoint">
124 <h3><a href="/debug/pprof/profile?seconds=30">CPU Profile</a></h3>
125 <p>30-second CPU profile</p>
126 </div>
127</body>
128</html>
129 `
130 fmt.Fprint(w, html)
131}
132
133// Health check endpoint
134func (dd *DebugDashboard) HealthHandler(w http.ResponseWriter, r *http.Request) {
135 health := map[string]interface{}{
136 "status": "healthy",
137 "timestamp": time.Now(),
138 "uptime": time.Since(dd.startTime).String(),
139 "uptime_seconds": int(time.Since(dd.startTime).Seconds()),
140 "version": "1.0.0",
141 "go_version": runtime.Version(),
142 }
143
144 w.Header().Set("Content-Type", "application/json")
145 json.NewEncoder(w).Encode(health)
146}
147
148// Metrics endpoint
149func (dd *DebugDashboard) MetricsHandler(w http.ResponseWriter, r *http.Request) {
150 var m runtime.MemStats
151 runtime.ReadMemStats(&m)
152
153 metrics := map[string]interface{}{
154 "timestamp": time.Now(),
155 "requests": map[string]interface{}{
156 "total": atomic.LoadInt64(&dd.requestCount),
157 "errors": atomic.LoadInt64(&dd.errorCount),
158 },
159 "runtime": map[string]interface{}{
160 "goroutines": runtime.NumGoroutine(),
161 "cpus": runtime.NumCPU(),
162 "go_version": runtime.Version(),
163 },
164 "memory": map[string]interface{}{
165 "alloc_mb": m.Alloc / 1024 / 1024,
166 "total_alloc_mb": m.TotalAlloc / 1024 / 1024,
167 "sys_mb": m.Sys / 1024 / 1024,
168 "num_gc": m.NumGC,
169 "heap_objects": m.HeapObjects,
170 },
171 }
172
173 w.Header().Set("Content-Type", "application/json")
174 json.NewEncoder(w).Encode(metrics)
175}
176
177// Memory stats endpoint
178func (dd *DebugDashboard) MemoryHandler(w http.ResponseWriter, r *http.Request) {
179 var m runtime.MemStats
180 runtime.ReadMemStats(&m)
181
182 memStats := map[string]interface{}{
183 "general": map[string]interface{}{
184 "alloc": m.Alloc,
185 "total_alloc": m.TotalAlloc,
186 "sys": m.Sys,
187 "lookups": m.Lookups,
188 "mallocs": m.Mallocs,
189 "frees": m.Frees,
190 },
191 "heap": map[string]interface{}{
192 "heap_alloc": m.HeapAlloc,
193 "heap_sys": m.HeapSys,
194 "heap_idle": m.HeapIdle,
195 "heap_in_use": m.HeapInuse,
196 "heap_released": m.HeapReleased,
197 "heap_objects": m.HeapObjects,
198 },
199 "stack": map[string]interface{}{
200 "stack_in_use": m.StackInuse,
201 "stack_sys": m.StackSys,
202 },
203 "gc": map[string]interface{}{
204 "num_gc": m.NumGC,
205 "num_forced_gc": m.NumForcedGC,
206 "gc_cpu_fraction": m.GCCPUFraction,
207 "next_gc": m.NextGC,
208 "last_gc": time.Unix(0, int64(m.LastGC)).Format(time.RFC3339),
209 },
210 }
211
212 w.Header().Set("Content-Type", "application/json")
213 json.NewEncoder(w).Encode(memStats)
214}
215
216// Goroutine stats endpoint
217func (dd *DebugDashboard) GoroutineHandler(w http.ResponseWriter, r *http.Request) {
218 count := runtime.NumGoroutine()
219
220 // Get stack trace if requested
221 includeStack := r.URL.Query().Get("stack") == "true"
222
223 response := map[string]interface{}{
224 "count": count,
225 "timestamp": time.Now(),
226 }
227
228 if includeStack {
229 buf := make([]byte, 1<<20) // 1MB buffer
230 stackLen := runtime.Stack(buf, true)
231 response["stack_trace"] = string(buf[:stackLen])
232 }
233
234 w.Header().Set("Content-Type", "application/json")
235 json.NewEncoder(w).Encode(response)
236}
237
238// Custom metrics endpoint
239func (dd *DebugDashboard) SetMetric(key string, value interface{}) {
240 dd.mu.Lock()
241 defer dd.mu.Unlock()
242 dd.metrics[key] = value
243}
244
245func (dd *DebugDashboard) CustomMetricsHandler(w http.ResponseWriter, r *http.Request) {
246 dd.mu.RLock()
247 defer dd.mu.RUnlock()
248
249 w.Header().Set("Content-Type", "application/json")
250 json.NewEncoder(w).Encode(dd.metrics)
251}
252
253func main() {
254 dashboard := NewDebugDashboard()
255
256 // Setup routes
257 mux := http.NewServeMux()
258
259 // Dashboard homepage
260 mux.HandleFunc("/", dashboard.HomeHandler)
261
262 // Debug endpoints
263 mux.HandleFunc("/debug/health", dashboard.HealthHandler)
264 mux.HandleFunc("/debug/metrics", dashboard.MetricsHandler)
265 mux.HandleFunc("/debug/memory", dashboard.MemoryHandler)
266 mux.HandleFunc("/debug/goroutines", dashboard.GoroutineHandler)
267 mux.HandleFunc("/debug/custom", dashboard.CustomMetricsHandler)
268
269 // pprof endpoints are registered automatically by importing _ "net/http/pprof"
270
271 // Example API endpoints to generate traffic
272 mux.HandleFunc("/api/status", func(w http.ResponseWriter, r *http.Request) {
273 dashboard.SetMetric("last_status_check", time.Now())
274 json.NewEncoder(w).Encode(map[string]string{"status": "ok"})
275 })
276
277 mux.HandleFunc("/api/slow", func(w http.ResponseWriter, r *http.Request) {
278 time.Sleep(2 * time.Second)
279 json.NewEncoder(w).Encode(map[string]string{"message": "slow endpoint"})
280 })
281
282 mux.HandleFunc("/api/error", func(w http.ResponseWriter, r *http.Request) {
283 http.Error(w, "intentional error for testing", http.StatusInternalServerError)
284 })
285
286 // Wrap with tracking middleware
287 handler := dashboard.TrackingMiddleware(mux)
288
289 // Create server
290 server := &http.Server{
291 Addr: ":8080",
292 Handler: handler,
293 ReadTimeout: 15 * time.Second,
294 WriteTimeout: 15 * time.Second,
295 IdleTimeout: 60 * time.Second,
296 }
297
298 // Start server
299 go func() {
300 log.Println("🚀 Debug Dashboard starting on :8080")
301 log.Println("📊 Dashboard: http://localhost:8080/")
302 log.Println("🏥 Health: http://localhost:8080/debug/health")
303 log.Println("📈 Metrics: http://localhost:8080/debug/metrics")
304 log.Println("🔍 pprof: http://localhost:8080/debug/pprof/")
305
306 if err := server.ListenAndServe(); err != nil && err != http.ErrServerClosed {
307 log.Fatalf("Server failed: %v", err)
308 }
309 }()
310
311 // Graceful shutdown
312 quit := make(chan os.Signal, 1)
313 signal.Notify(quit, syscall.SIGINT, syscall.SIGTERM)
314 <-quit
315
316 log.Println("Shutting down server...")
317 ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
318 defer cancel()
319
320 if err := server.Shutdown(ctx); err != nil {
321 log.Printf("Server forced to shutdown: %v", err)
322 }
323
324 log.Println("Server exited")
325}
Summary
Key Takeaways
🔍 Debugging Strategy:
- Start with symptoms, gather evidence systematically
- Use the right tool for the right problem
- Combine multiple debugging approaches for complex issues
- Practice progressive debugging: reproduce, isolate, fix, verify
🛠️ Tool Mastery:
- Delve for interactive debugging and understanding code flow
- pprof for performance analysis and optimization
- Race detector for catching concurrency issues early
- Runtime traces for understanding goroutine behavior
⚡ Production Debugging:
- Enable debug endpoints conditionally in production
- Use structured logging with correlation IDs
- Implement graceful degradation for debugging features
- Monitor resource usage trends over time
Next Steps in Your Learning Journey
📚 Recommended Learning Path:
- Practice debugging techniques on your own projects
- Study Go runtime internals to understand what the tools measure
- Learn advanced pprof analysis for complex performance optimization
- Explore external debugging tools that integrate with Go applications
- Contribute to debugging tools to understand their internals
🛠️ Advanced Topics to Explore:
- Distributed tracing with tools like Jaeger or OpenTelemetry
- Production observability with metrics collection and alerting
- Chaos engineering for testing system resilience
- Memory optimization techniques beyond basic profiling
- Advanced race detection patterns for complex concurrent systems
🚀 Production Readiness:
- Set up comprehensive debugging dashboards
- Implement automated monitoring and alerting
- Create debugging playbooks for common issues
- Establish debugging workflows for your team
Remember: Effective debugging is a skill that improves with practice. Start with simple tools and gradually master advanced techniques as you encounter more complex problems.
Previous: 16 Essential Go Libraries | Next: 18 Go Development Workflow