JSON & Data Encoding

Why JSON and Encoding Matters

Consider writing a letter to a friend who speaks a different language - you need to translate your thoughts into a format they can understand, and they need to translate it back to understand you. Data encoding in Go works exactly like this - it translates your Go data structures into formats that other systems can understand, and back again.

Go's encoding/json package provides powerful tools for working with JSON data. JSON is the de facto standard for web APIs and configuration files, acting as the universal language that allows different programming languages and systems to communicate seamlessly.

💡 Key Takeaway: Think of JSON as a universal translator for your data - it converts Go's strict, typed structures into a flexible text format that any system can understand.

Real-world Impact: Every time you use a mobile app, check the weather, or buy something online, JSON is working behind the scenes. When you post a tweet, your app sends JSON to Twitter's API. When you check your bank balance, the bank's servers send JSON to your app. JSON encoding is the invisible glue that holds the modern internet together.

Understanding JSON in Go isn't just about converting data - it's about:

• Building APIs that millions of users can access
• Integrating services that power modern applications
• Storing configuration that makes apps flexible and maintainable
• Logging structured data that helps you debug production issues
• Communicating between microservices in distributed systems

Learning Objectives

By the end of this article, you'll be able to:

✅ Encode and decode JSON with Go's standard library using Marshal/Unmarshal patterns
✅ Master struct tags for controlling JSON field names, behavior, and API contracts
✅ Handle complex data including nested structures, arrays, maps, and optional fields
✅ Implement custom marshalers for special types like time formats and enums
✅ Stream large datasets efficiently without loading everything into memory
✅ Work with unknown JSON structures using RawMessage and interface{}
✅ Apply production patterns including validation, error handling, and performance optimization
✅ Use multiple formats including XML, CSV, Protocol Buffers, and MessagePack

Core Concepts - Understanding Go's JSON Philosophy

The encoding/json package embodies Go's philosophy of simplicity, clarity, and performance. Unlike some languages that require external libraries or complex frameworks, Go provides production-ready JSON handling right in the standard library.

The JSON Ecosystem in Go

JSON in Production:

Every modern web service uses JSON for:
- REST APIs: Request/response bodies
- Configuration: Config files, feature flags
- Logging: Structured logs for parsing
- Message Queues: Kafka, RabbitMQ payloads
- Databases: MongoDB, PostgreSQL JSONB
- Cache: Redis with JSON values
- Webhooks: GitHub, Stripe, payment processors
- Mobile Apps: iOS/Android API communication

Real-world Example: When you open Netflix, your device sends a JSON request asking for your personalized recommendations. Netflix's servers process millions of these JSON requests per second, encode recommendations as JSON, and send them back. Your app decodes the JSON and displays the shows. All of this happens in milliseconds, thanks to efficient JSON encoding.

Why Go's JSON Package is Special

1. Standard Library - No external dependencies needed. Production-ready JSON handling built-in.

2. Type Safety - Struct-based JSON encoding provides compile-time type safety, unlike JavaScript's dynamic approach.

3. Reflection-Based - Uses Go's reflection to automatically map structs ↔ JSON, reducing boilerplate.

4. Performance - One of the fastest JSON implementations across all languages. Competes with C++ libraries.

5. Flexibility - Supports streaming, custom marshaling, delayed parsing, and multiple formats.

Go JSON vs Other Languages

Feature	Go	Python	Java	JavaScript
Type Safety	✅ Compile-time	❌ Runtime only	✅ Compile-time	❌ Dynamic
Performance	✅ Very fast	⚠️ Moderate	✅ Fast	✅ Fast
Boilerplate	⚠️ Struct definitions	✅ Minimal	❌ Verbose	✅ Native
Streaming	✅ Built-in	✅ Via ijson	⚠️ Via Jackson	⚠️ Via streams
Custom Encoding	✅ Interfaces	✅ __dict__	✅ Annotations	✅ toJSON()
Standard Library	✅ Complete	✅ Complete	❌ Needs Jackson/Gson	✅ Native
Binary Size	~10 MB	~50 MB	~50 MB	~50 MB

Performance Benchmarks:

Encoding 10,000 objects:

Language/Library    | Encode Time | Decode Time | Memory
--------------------|-------------|-------------|--------
Go encoding/json    | 15 ms       | 20 ms       | 10 MB
Go jsoniter         | 8 ms        | 12 ms       | 8 MB
Python json         | 45 ms       | 60 ms       | 25 MB
Java Jackson        | 25 ms       | 30 ms       | 20 MB
Node.js JSON        | 18 ms       | 22 ms       | 15 MB

Note: Go's standard library is fast, but specialized libraries
like jsoniter can be 2x faster for critical paths.

Why This Matters: When you're building an API that handles 10,000 requests per second, every millisecond of JSON encoding/decoding time matters. A 5ms improvement can mean the difference between handling 10,000 or 20,000 requests per second on the same hardware.

Common JSON Use Cases in Production

Use Case	Method	Why	Example
REST API responses	json.NewEncoder(w).Encode()	Streaming to HTTP response	Web servers, microservices
Config files	json.Unmarshal()	Load entire config at startup	App settings, feature flags
Logging	json.Marshal()	Structured logs for parsing	Application logs, audit trails
Message queues	json.Marshal() / Unmarshal()	Serialize messages	Kafka, RabbitMQ, SQS
Large datasets	json.Decoder.Decode()	Stream processing, low memory	Data pipelines, ETL
API clients	json.NewDecoder(resp.Body)	Parse HTTP responses	SDK libraries, integrations

Key Concepts You'll Master

1. Marshal vs Encoder - When to use each approach and why
2. Struct Tags - Control JSON field names and behavior precisely
3. Custom Marshaling - Implement json.Marshaler for special types
4. Streaming - Process large JSON without loading everything into memory
5. Unknown JSON - Handle dynamic/unknown structures safely
6. Performance - Avoid allocations and copies in hot paths
7. Multiple Formats - XML, CSV, Base64, Protobuf for different contexts

⚠️ Important: JSON encoding isn't just about converting data - it's about ensuring your applications can communicate reliably with other systems. Small mistakes in encoding can cause entire systems to fail!

The Two Approaches: Marshal vs Encoder

Go provides two distinct APIs for JSON operations, each optimized for different use cases:

Marshal/Unmarshal Pattern (In-Memory):

1// Encode entire value to []byte
2data, err := json.Marshal(value)
3
4// Decode entire []byte to value
5err := json.Unmarshal(data, &value)

Encoder/Decoder Pattern (Streaming):

1// Stream encode directly to io.Writer
2encoder := json.NewEncoder(writer)
3err := encoder.Encode(value)
4
5// Stream decode directly from io.Reader
6decoder := json.NewDecoder(reader)
7err := decoder.Decode(&value)

When to Use Each:

Scenario	Use Marshal	Use Encoder	Reason
HTTP API response	❌	✅	Stream directly to response writer
Building JSON string	✅	❌	Need []byte for further processing
Reading HTTP request	❌	✅	Stream from request body
Logging	✅	❌	Need string for log message
Large files	❌	✅	Avoid loading entire file in memory
Small data	✅	❌	Simpler API, minimal overhead

💡 Key Takeaway: Use Marshal/Unmarshal for small data where you need byte slices. Use Encoder/Decoder for streaming, especially with HTTP or files. Encoder/Decoder is more efficient for I/O operations because it avoids allocating intermediate byte slices.

Now that we understand why JSON encoding matters and how Go approaches it, let's start with hands-on examples that demonstrate real-world patterns.

Practical Examples - JSON Fundamentals

Let's start with foundational JSON operations, progressing from basic concepts to production-ready patterns with immediate code examples.

Example 1: Basic Marshal and Unmarshal

Learning Goal: Understand fundamental JSON encoding/decoding patterns with structs, maps, and raw messages.

Why This Matters: These are the building blocks for all JSON operations. You'll use these patterns in every Go project that works with JSON data.

 1// run
 2package main
 3
 4import (
 5    "encoding/json"
 6    "fmt"
 7    "log"
 8)
 9
10func main() {
11    // Example 1: Simple struct to JSON
12    type User struct {
13        Name  string `json:"name"`
14        Email string `json:"email"`
15        Age   int    `json:"age"`
16    }
17
18    user := User{
19        Name:  "Alice Johnson",
20        Email: "alice@example.com",
21        Age:   30,
22    }
23
24    // Marshal: Go struct → JSON bytes
25    jsonData, err := json.Marshal(user)
26    if err != nil {
27        log.Fatalf("Failed to marshal user: %v", err)
28    }
29
30    fmt.Printf("JSON Output: %s\n", string(jsonData))
31
32    // Unmarshal: JSON bytes → Go struct
33    var decodedUser User
34    if err := json.Unmarshal(jsonData, &decodedUser); err != nil {
35        log.Fatalf("Failed to unmarshal JSON: %v", err)
36    }
37
38    fmt.Printf("Decoded User: %+v\n", decodedUser)
39
40    // Example 2: Dynamic data with map[string]interface{}
41    // Useful when JSON structure is unknown at compile time
42    dynamicData := map[string]interface{}{
43        "name":   "Bob",
44        "age":    25,
45        "skills": []string{"Go", "JavaScript", "Docker"},
46        "active": true,
47    }
48
49    jsonData2, _ := json.Marshal(dynamicData)
50    fmt.Printf("\nDynamic JSON: %s\n", string(jsonData2))
51
52    var decodedDynamic map[string]interface{}
53    json.Unmarshal(jsonData2, &decodedDynamic)
54
55    // Type assertions required for interface{} values
56    name := decodedDynamic["name"].(string)
57    age := int(decodedDynamic["age"].(float64)) // JSON numbers are float64!
58    active := decodedDynamic["active"].(bool)
59
60    fmt.Printf("Parsed: name=%s age=%d active=%v\n", name, age, active)
61
62    // Example 3: Common pattern - raw JSON for delayed parsing
63    type APIResponse struct {
64        Status string          `json:"status"`
65        Data   json.RawMessage `json:"data"` // Don't parse immediately
66    }
67
68    response := APIResponse{
69        Status: "success",
70        Data:   jsonData2, // Re-use JSON from above
71    }
72
73    responseJSON, _ := json.Marshal(response)
74    fmt.Printf("\nAPI Response: %s\n", string(responseJSON))
75}

Key Concepts Demonstrated:

• ✅ Struct Encoding: Type-safe JSON marshaling with field tags
• ✅ Dynamic Parsing: Using map[string]interface{} for unknown structures
• ✅ Type Assertions: Safe casting from interface{} to concrete types
• ✅ Delayed Parsing: json.RawMessage for conditional processing
• ✅ JSON Number Handling: Numbers decode as float64, need casting

Production Insight: The json.RawMessage pattern is extremely common in production APIs where the response structure varies based on status codes or type fields. Instead of unmarshaling everything immediately, you can peek at status/type fields first and then unmarshal the data field into the appropriate struct.

Example 2: Advanced Struct Tags and Field Control

Learning Goal: Master struct tags for precise JSON control and API contract definition.

Why This Matters: Struct tags are your interface between Go naming conventions and API requirements. They let you create clean Go code while satisfying external API contracts.

 1// run
 2package main
 3
 4import (
 5    "encoding/json"
 6    "fmt"
 7    "os"
 8)
 9
10type Product struct {
11    ID           int      `json:"product_id"`            // Rename field
12    Name         string   `json:"name"`                   // Keep name as-is
13    Description  string   `json:"description,omitempty"`  // Omit if empty
14    Price        float64  `json:"price,string"`           // Encode as string
15    SKU          string   `json:"sku,omitempty"`          // Omit if empty
16    IsActive     bool     `json:"is_active"`              // snake_case
17    Categories   []string `json:"categories,omitempty"`   // Omit empty slice
18    InternalNote string   `json:"-"`                      // Never serialize
19    Discount     *float64 `json:"discount,omitempty"`     // Pointer for optional field
20}
21
22func main() {
23    // Example 1: Full product with all fields
24    fullPrice := 29.99
25    discount := 0.15 // 15% discount
26
27    product := Product{
28        ID:           1,
29        Name:         "Go Programming Book",
30        Description:  "Learn Go programming from scratch",
31        Price:        fullPrice,
32        SKU:          "GO-BOOK-001",
33        IsActive:     true,
34        Categories:   []string{"Programming", "Education", "Technology"},
35        InternalNote: "This is internal only",
36        Discount:     &discount,
37    }
38
39    jsonData, err := json.MarshalIndent(product, "", "  ")
40    if err != nil {
41        fmt.Printf("Error: %v\n", err)
42        os.Exit(1)
43    }
44
45    fmt.Printf("Full Product JSON:\n%s\n\n", string(jsonData))
46
47    // Example 2: Product with optional fields omitted
48    partialProduct := Product{
49        ID:       2,
50        Name:     "Basic Item",
51        Price:    9.99,
52        IsActive: false,
53        // No Description, SKU, Categories, Discount - should be omitted
54    }
55
56    partialJSON, _ := json.MarshalIndent(partialProduct, "", "  ")
57    fmt.Printf("Partial Product JSON (omitempty in action):\n%s\n\n", string(partialJSON))
58
59    // Example 3: Demonstrating pointer vs value for omitempty
60    zeroValue := 0.0
61    withZeroPtr := &zeroValue
62
63    product3 := Product{
64        ID:       3,
65        Name:     "Free Item",
66        Price:    0,
67        Discount: withZeroPtr, // Pointer to zero value - still included!
68    }
69
70    zeroPtrJSON, _ := json.MarshalIndent(product3, "", "  ")
71    fmt.Printf("Product with zero pointer (pointer allows distinguishing null from 0):\n%s\n\n", string(zeroPtrJSON))
72
73    // Example 4: String encoding of numbers
74    type Money struct {
75        Amount   float64 `json:"amount,string"` // As string for precision
76        Currency string  `json:"currency"`
77    }
78
79    money := Money{Amount: 1234.56, Currency: "USD"}
80    moneyJSON, _ := json.MarshalIndent(money, "", "  ")
81    fmt.Printf("Money with string encoding (avoids float precision issues):\n%s\n\n", string(moneyJSON))
82
83    // Parsing back demonstrates string → number conversion
84    var parsedMoney Money
85    json.Unmarshal(moneyJSON, &parsedMoney)
86    fmt.Printf("Parsed Money: %.2f %s\n", parsedMoney.Amount, parsedMoney.Currency)
87}

Advanced Features Explained:

• ✅ Field Renaming: json:"product_id" maps Go field to JSON name
• ✅ Omit Empty: omitempty excludes zero/empty/null values
• ✅ String Encoding: json:"price,string" encodes numbers as JSON strings
• ✅ Exclude Fields: json:"-" never serializes sensitive data
• ✅ Pointer Types: Distinguish "not set" from "zero value"
• ✅ Case Conversion: Automatic camelCase ↔ snake_case handling

Production Insight: The omitempty tag is critical for API bandwidth optimization. Without it, your JSON responses include every field even when they're empty. For large-scale APIs serving millions of requests, removing unnecessary fields can reduce bandwidth costs by 30-50%.

Understanding Struct Tags in Depth

Struct tags are like name tags for your data fields - they tell Go exactly how to name each field when converting to and from JSON. They're one of Go's most powerful features for API design, allowing you to bridge the gap between Go naming conventions and API requirements.

Real-world Example: Imagine you're building an API for a JavaScript frontend. Your Go code uses PascalCase (UserID, FirstName) but JavaScript prefers camelCase (userId, firstName). Struct tags let you satisfy both conventions without compromising either.

Why Struct Tags Matter in Production:

1. API Contract - Define exact JSON field names independently of Go naming conventions
2. Flexibility - Support snake_case JSON while using PascalCase in Go
3. Optional Fields - Control which fields appear in JSON responses
4. Security - Prevent sensitive fields from being exposed in JSON
5. Compatibility - Match existing JSON APIs without changing Go code
6. Bandwidth - Reduce JSON size by omitting empty fields

Struct Tag Syntax Explained:

1type Field Type `json:"json_name,option1,option2"`
2                      ↑          ↑
3                field name    options

All JSON Tag Options:

Tag	Effect	Use Case	Example
json:"field_name"	Rename field in JSON	API uses snake_case, Go uses PascalCase	json:"user_id"
json:",omitempty"	Omit if zero value	Optional fields, reduce JSON size	json:"age,omitempty"
json:"-"	Never serialize	Sensitive data, internal fields	json:"-"
json:",string"	Force string encoding	Numbers as strings, precision	json:"price,string"
No tag	Use field name as-is	Simple cases, when names match	(no tag)

Under the Hood - How Struct Tags Work:

 1// Go uses reflection to read struct tags at runtime
 2type User struct {
 3    ID   int    `json:"id"`
 4    Name string `json:"name"`
 5}
 6
 7// At encoding time:
 81. Reflect on User struct
 92. For each exported field, read the `json` tag
103. Use tag value as JSON key name
114. Apply options (omitempty, string, etc.)
125. Encode value to JSON
13
14// Without tags:
15{"ID": 1, "Name": "Alice"}  // Exact Go field names
16
17// With tags:
18{"id": 1, "name": "Alice"}  // Tag-specified names

Real-World Pattern - Versioned API Structs:

 1// Support multiple API versions with different field names
 2type UserV1 struct {
 3    ID        int    `json:"id"`
 4    Username  string `json:"username"`
 5}
 6
 7type UserV2 struct {
 8    ID        int    `json:"user_id"`      // Renamed in v2
 9    Username  string `json:"user_name"`    // Renamed in v2
10    Email     string `json:"email"`        // New in v2
11}
12
13// Convert between versions as needed
14func (u UserV1) ToV2(email string) UserV2 {
15    return UserV2{
16        ID:       u.ID,
17        Username: u.Username,
18        Email:    email,
19    }
20}

Production Pattern - Sensitive Data Filtering:

 1type User struct {
 2    ID           int    `json:"id"`
 3    Username     string `json:"username"`
 4    Email        string `json:"email"`
 5
 6    // Never expose in API responses
 7    PasswordHash string `json:"-"`
 8    APIKey       string `json:"-"`
 9    InternalID   string `json:"-"`
10
11    // Optional in responses
12    LastLogin    *time.Time `json:"last_login,omitempty"`
13    Bio          string     `json:"bio,omitempty"`
14}

Structs and JSON - The Type-Safe Approach

While maps are flexible, structs are where Go's type system really shines. They provide compile-time safety, better performance, and make your code more maintainable. Think of structs as custom-designed containers for your data, perfectly fit for what you're storing.

Why Structs Over Maps:

Aspect	Structs	Maps
Type Safety	✅ Compile-time checks	❌ Runtime errors
Performance	✅ ~20% faster	⚠️ Slower
IDE Support	✅ Autocomplete, refactoring	❌ Limited
Documentation	✅ Self-documenting	❌ Must read code
Validation	✅ Struct tags, methods	❌ Manual checks
Maintenance	✅ Easy refactoring	❌ String keys fragile

Basic Struct Marshaling

 1// run
 2package main
 3
 4import (
 5    "encoding/json"
 6    "fmt"
 7)
 8
 9type Person struct {
10    Name string
11    Age  int
12    City string
13}
14
15func main() {
16    person := Person{
17        Name: "Alice",
18        Age:  25,
19        City: "NYC",
20    }
21
22    jsonData, _ := json.Marshal(person)
23    fmt.Println(string(jsonData))
24    // Output: {"Name":"Alice","Age":25,"City":"NYC"}
25
26    // Notice: Go struct field names become JSON keys
27    // Exported fields (capitalized) are included
28    // Unexported fields (lowercase) are ignored
29}

Production Tip: Always use struct tags even if field names match your desired JSON output. This makes your code more maintainable and protects against future changes.

Nested Structures - Modeling Real-World Relationships

Real-world objects often contain other objects - a person has an address, an order has items, a company has employees. Nested structs in Go perfectly model these real-world relationships.

💡 Key Takeaway: Nested structs let you model complex real-world relationships in your code. When you encode to JSON, these relationships are preserved automatically. This is how modern APIs represent complex domain models.

 1// run
 2package main
 3
 4import (
 5    "encoding/json"
 6    "fmt"
 7)
 8
 9type Address struct {
10    Street  string `json:"street"`
11    City    string `json:"city"`
12    ZipCode string `json:"zip_code"`
13    Country string `json:"country"`
14}
15
16type Person struct {
17    Name    string  `json:"name"`
18    Age     int     `json:"age"`
19    Address Address `json:"address"`
20    Email   string  `json:"email"`
21}
22
23func main() {
24    person := Person{
25        Name: "Alice",
26        Age:  25,
27        Address: Address{
28            Street:  "123 Main St",
29            City:    "NYC",
30            ZipCode: "10001",
31            Country: "USA",
32        },
33        Email: "alice@example.com",
34    }
35
36    jsonData, _ := json.MarshalIndent(person, "", "  ")
37    fmt.Println(string(jsonData))
38
39    // Unmarshal back to verify bidirectional conversion
40    var decoded Person
41    json.Unmarshal(jsonData, &decoded)
42    fmt.Printf("\nDecoded: %s lives in %s\n", decoded.Name, decoded.Address.City)
43}

Real-World Pattern - Complex E-Commerce Order:

 1type Order struct {
 2    OrderID      string    `json:"order_id"`
 3    CustomerID   string    `json:"customer_id"`
 4    Items        []OrderItem `json:"items"`
 5    ShippingAddr Address   `json:"shipping_address"`
 6    BillingAddr  Address   `json:"billing_address,omitempty"`
 7    TotalAmount  float64   `json:"total_amount,string"`
 8    Status       string    `json:"status"`
 9    CreatedAt    time.Time `json:"created_at"`
10}
11
12type OrderItem struct {
13    ProductID string  `json:"product_id"`
14    Name      string  `json:"name"`
15    Quantity  int     `json:"quantity"`
16    Price     float64 `json:"price,string"`
17}

Arrays and Slices - Collections in JSON

Arrays and slices represent collections of items - think of them as lists that can hold multiple values. In JSON, these become arrays, which are universally understood across all programming languages.

Real-world Example: A shopping cart contains multiple items, a user has multiple hobbies, and a blog post has multiple comments. All of these are perfect use cases for slices in Go. When you encode them to JSON, they become JSON arrays that any client can process.

 1// run
 2package main
 3
 4import (
 5    "encoding/json"
 6    "fmt"
 7)
 8
 9type Team struct {
10    Name    string   `json:"name"`
11    Members []string `json:"members"`
12}
13
14func main() {
15    team := Team{
16        Name:    "Developers",
17        Members: []string{"Alice", "Bob", "Carol"},
18    }
19
20    jsonData, _ := json.MarshalIndent(team, "", "  ")
21    fmt.Println(string(jsonData))
22
23    // Unmarshal back
24    var decoded Team
25    json.Unmarshal(jsonData, &decoded)
26    fmt.Printf("\nTeam: %s has %d members\n", decoded.Name, len(decoded.Members))
27
28    // Nil vs empty slice difference
29    type Response struct {
30        Items []string `json:"items"`
31    }
32
33    // nil slice encodes as null
34    r1 := Response{Items: nil}
35    j1, _ := json.Marshal(r1)
36    fmt.Printf("\nnil slice: %s\n", j1)  // {"items":null}
37
38    // empty slice encodes as []
39    r2 := Response{Items: []string{}}
40    j2, _ := json.Marshal(r2)
41    fmt.Printf("empty slice: %s\n", j2)  // {"items":[]}
42}

Production Best Practice: Always initialize slices to empty rather than nil for consistent JSON output. Use make([]Type, 0) or []Type{} instead of nil to ensure empty arrays in JSON rather than null.

Working with Unknown JSON - Dynamic Data Handling

Sometimes you don't know the structure of JSON data in advance - like when you're building a generic API client or processing data from external services. Go provides powerful tools to handle these dynamic scenarios safely.

json.RawMessage - Delayed Parsing Pattern

Think of json.RawMessage as a "delayed opening" for JSON data. It lets you peek at part of the JSON to decide how to handle the rest, perfect for APIs that return different data types based on status codes or type fields.

Real-world Example: An API might return {"status": "success", "data": {...}} or {"status": "error", "error": {...}}. You need to check the status first before deciding how to parse the data field. This pattern is used by GitHub API, Stripe API, and most modern REST APIs.

 1// run
 2package main
 3
 4import (
 5    "encoding/json"
 6    "fmt"
 7)
 8
 9type Response struct {
10    Status string          `json:"status"`
11    Data   json.RawMessage `json:"data"` // Delay parsing until we know the status
12}
13
14type SuccessData struct {
15    Name string `json:"name"`
16    Age  int    `json:"age"`
17}
18
19type ErrorData struct {
20    Code    string `json:"code"`
21    Message string `json:"message"`
22}
23
24func main() {
25    // Success case
26    successJSON := `{"status":"success","data":{"name":"Alice","age":25}}`
27
28    var resp Response
29    json.Unmarshal([]byte(successJSON), &resp)
30
31    fmt.Println("Status:", resp.Status)
32
33    // Now parse the data field based on status
34    if resp.Status == "success" {
35        var user SuccessData
36        json.Unmarshal(resp.Data, &user)
37        fmt.Printf("User: name=%s age=%d\n", user.Name, user.Age)
38    }
39
40    // Error case
41    errorJSON := `{"status":"error","data":{"code":"NOT_FOUND","message":"User not found"}}`
42
43    var resp2 Response
44    json.Unmarshal([]byte(errorJSON), &resp2)
45
46    if resp2.Status == "error" {
47        var errData ErrorData
48        json.Unmarshal(resp2.Data, &errData)
49        fmt.Printf("Error: code=%s message=%s\n", errData.Code, errData.Message)
50    }
51}

When to Use RawMessage:

• ✅ API responses with variable data shapes based on status/type
• ✅ Webhook payloads with different event types
• ✅ GraphQL responses with fragments
• ✅ Plugin systems where data format varies
• ✅ Performance optimization - parse only what you need

map[string]interface{} - Maximum Flexibility

When you need maximum flexibility, map[string]interface{} is your go-to solution. It can hold any JSON structure, but remember that you lose compile-time type safety and pay a performance cost.

⚠️ Important: Use map[string]interface{} sparingly. While flexible, it requires runtime type checking and can lead to runtime errors if not handled carefully. Prefer structs when possible for better performance and type safety.

 1// run
 2package main
 3
 4import (
 5    "encoding/json"
 6    "fmt"
 7)
 8
 9func main() {
10    jsonStr := `{
11        "name": "Alice",
12        "age": 25,
13        "hobbies": ["reading", "coding"],
14        "address": {"city": "NYC"},
15        "active": true,
16        "balance": 1234.56
17    }`
18
19    var data map[string]interface{}
20    json.Unmarshal([]byte(jsonStr), &data)
21
22    // Access simple types
23    fmt.Println("Name:", data["name"].(string))
24    fmt.Println("Active:", data["active"].(bool))
25
26    // JSON numbers are always float64!
27    age := int(data["age"].(float64))
28    fmt.Println("Age:", age)
29
30    // Arrays become []interface{}
31    hobbies := data["hobbies"].([]interface{})
32    fmt.Println("First hobby:", hobbies[0].(string))
33
34    // Nested objects become map[string]interface{}
35    address := data["address"].(map[string]interface{})
36    fmt.Println("City:", address["city"].(string))
37
38    // Safe type assertion with comma-ok idiom
39    if balance, ok := data["balance"].(float64); ok {
40        fmt.Printf("Balance: $%.2f\n", balance)
41    }
42}

Production Pattern - Safe Dynamic Access:

 1// Helper function for safe map access
 2func getString(m map[string]interface{}, key string) (string, bool) {
 3    val, exists := m[key]
 4    if !exists {
 5        return "", false
 6    }
 7    str, ok := val.(string)
 8    return str, ok
 9}
10
11func getInt(m map[string]interface{}, key string) (int, bool) {
12    val, exists := m[key]
13    if !exists {
14        return 0, false
15    }
16    // JSON numbers are float64
17    if f, ok := val.(float64); ok {
18        return int(f), true
19    }
20    return 0, false
21}

Custom JSON Encoding - Full Control

Sometimes the default JSON encoding isn't what you need. Maybe you want emails always lowercase, dates in a specific format, or enums as strings. Implementing json.Marshaler and json.Unmarshaler interfaces gives you complete control.

When to Implement Custom Marshalers:

• ✅ Custom time/date formats for API compatibility
• ✅ Enum types as strings instead of integers
• ✅ Data normalization (lowercase emails, trimmed strings)
• ✅ Encryption/decryption during encoding
• ✅ Complex business logic during serialization
• ✅ Legacy format compatibility

 1// run
 2package main
 3
 4import (
 5    "encoding/json"
 6    "fmt"
 7    "strings"
 8)
 9
10// Custom Email type that normalizes to lowercase
11type Email string
12
13func (e Email) MarshalJSON() ([]byte, error) {
14    // Always lowercase emails when serializing
15    return json.Marshal(strings.ToLower(string(e)))
16}
17
18func (e *Email) UnmarshalJSON(data []byte) error {
19    var s string
20    if err := json.Unmarshal(data, &s); err != nil {
21        return err
22    }
23    *e = Email(strings.ToLower(s))
24    return nil
25}
26
27type User struct {
28    Name  string `json:"name"`
29    Email Email  `json:"email"`
30}
31
32func main() {
33    user := User{
34        Name:  "Alice",
35        Email: "ALICE@EXAMPLE.COM",  // Mixed case input
36    }
37
38    jsonData, _ := json.MarshalIndent(user, "", "  ")
39    fmt.Println("Encoded (email normalized to lowercase):")
40    fmt.Println(string(jsonData))
41    // Email will be "alice@example.com" in JSON
42
43    // Unmarshal with different case
44    jsonStr := `{"name":"Bob","email":"BOB@EXAMPLE.COM"}`
45    var user2 User
46    json.Unmarshal([]byte(jsonStr), &user2)
47    fmt.Printf("\nDecoded email (normalized): %s\n", user2.Email)
48}

Production Example - Custom Status Enum:

 1type OrderStatus int
 2
 3const (
 4    OrderPending OrderStatus = iota
 5    OrderProcessing
 6    OrderShipped
 7    OrderDelivered
 8    OrderCancelled
 9)
10
11func (s OrderStatus) MarshalJSON() ([]byte, error) {
12    statuses := []string{
13        "pending",
14        "processing",
15        "shipped",
16        "delivered",
17        "cancelled",
18    }
19    if int(s) < 0 || int(s) >= len(statuses) {
20        return nil, fmt.Errorf("invalid status: %d", s)
21    }
22    return json.Marshal(statuses[s])
23}
24
25func (s *OrderStatus) UnmarshalJSON(data []byte) error {
26    var str string
27    if err := json.Unmarshal(data, &str); err != nil {
28        return err
29    }
30
31    statuses := map[string]OrderStatus{
32        "pending":    OrderPending,
33        "processing": OrderProcessing,
34        "shipped":    OrderShipped,
35        "delivered":  OrderDelivered,
36        "cancelled":  OrderCancelled,
37    }
38
39    status, ok := statuses[strings.ToLower(str)]
40    if !ok {
41        return fmt.Errorf("invalid status: %s", str)
42    }
43    *s = status
44    return nil
45}

Handling Time - Date and Timestamp Encoding

Go's time.Time type has built-in JSON support, but you often need custom formats for API compatibility or human readability.

 1// run
 2package main
 3
 4import (
 5    "encoding/json"
 6    "fmt"
 7    "time"
 8)
 9
10type Event struct {
11    Name      string    `json:"name"`
12    Timestamp time.Time `json:"timestamp"`  // RFC3339 format by default
13}
14
15func main() {
16    event := Event{
17        Name:      "Meeting",
18        Timestamp: time.Now(),
19    }
20
21    jsonData, _ := json.MarshalIndent(event, "", "  ")
22    fmt.Println("Default time encoding (RFC3339):")
23    fmt.Println(string(jsonData))
24
25    // Unmarshal back
26    var decoded Event
27    json.Unmarshal(jsonData, &decoded)
28    fmt.Printf("\nEvent: %s at %v\n", decoded.Name, decoded.Timestamp)
29
30    // Custom time format example
31    type CustomDate time.Time
32
33    func (d CustomDate) MarshalJSON() ([]byte, error) {
34        formatted := time.Time(d).Format("2006-01-02")
35        return json.Marshal(formatted)
36    }
37
38    type EventWithCustomDate struct {
39        Name string     `json:"name"`
40        Date CustomDate `json:"date"`
41    }
42
43    evt := EventWithCustomDate{
44        Name: "Birthday",
45        Date: CustomDate(time.Now()),
46    }
47
48    customJSON, _ := json.MarshalIndent(evt, "", "  ")
49    fmt.Println("\nCustom date format (YYYY-MM-DD):")
50    fmt.Println(string(customJSON))
51}

Common Time Format Patterns:

 1// Date only: "2006-01-02"
 2const DateFormat = "2006-01-02"
 3
 4// DateTime: "2006-01-02 15:04:05"
 5const DateTimeFormat = "2006-01-02 15:04:05"
 6
 7// Unix timestamp (seconds)
 8func (t Time) MarshalJSON() ([]byte, error) {
 9    return json.Marshal(time.Time(t).Unix())
10}
11
12// ISO 8601 with timezone: "2006-01-02T15:04:05Z07:00"
13const ISO8601 = time.RFC3339

Streaming JSON - Handling Large Datasets

For large datasets, use json.Encoder and json.Decoder to stream data without loading everything into memory. This is critical for production systems processing gigabytes of JSON.

Why Streaming Matters:

• ✅ Process files larger than available memory
• ✅ Lower memory footprint (~90% less)
• ✅ Faster startup time - begin processing immediately
• ✅ Better for HTTP - stream responses as they're generated
• ✅ Efficient for logs - process line by line

 1// run
 2package main
 3
 4import (
 5    "encoding/json"
 6    "fmt"
 7    "os"
 8)
 9
10type Person struct {
11    Name string `json:"name"`
12    Age  int    `json:"age"`
13}
14
15func main() {
16    // Writing JSON with Encoder - streams directly to file
17    file, _ := os.Create("people.json")
18    defer file.Close()
19
20    encoder := json.NewEncoder(file)
21    encoder.SetIndent("", "  ")
22
23    people := []Person{
24        {Name: "Alice", Age: 25},
25        {Name: "Bob", Age: 30},
26        {Name: "Carol", Age: 35},
27    }
28
29    for _, person := range people {
30        encoder.Encode(person)  // Writes directly to file, no buffering
31    }
32
33    fmt.Println("✓ Streamed", len(people), "people to file")
34
35    // Reading JSON with Decoder - streams from file
36    file2, _ := os.Open("people.json")
37    defer file2.Close()
38
39    decoder := json.NewDecoder(file2)
40
41    count := 0
42    for {
43        var person Person
44        err := decoder.Decode(&person)
45        if err != nil {
46            break  // EOF or error
47        }
48        count++
49        fmt.Printf("Read: %s (age %d)\n", person.Name, person.Age)
50    }
51
52    fmt.Printf("\n✓ Streamed %d people from file\n", count)
53
54    // Cleanup
55    os.Remove("people.json")
56}

Production Pattern - Processing Large Log Files:

 1func processLogFile(filename string) error {
 2    file, err := os.Open(filename)
 3    if err != nil {
 4        return err
 5    }
 6    defer file.Close()
 7
 8    decoder := json.NewDecoder(file)
 9
10    // Read opening bracket of array
11    _, err = decoder.Token()
12    if err != nil {
13        return err
14    }
15
16    // Process each log entry
17    for decoder.More() {
18        var entry LogEntry
19        if err := decoder.Decode(&entry); err != nil {
20            return err
21        }
22
23        // Process entry - memory is freed after each iteration
24        processLogEntry(entry)
25    }
26
27    // Read closing bracket
28    _, err = decoder.Token()
29    return err
30}

Other Encoding Formats - Beyond JSON

While JSON is ubiquitous, Go's standard library supports multiple formats for different use cases.

XML - Legacy Systems and Structured Documents

XML is still common in enterprise systems, SOAP APIs, and configuration files.

 1// run
 2package main
 3
 4import (
 5    "encoding/xml"
 6    "fmt"
 7)
 8
 9type Book struct {
10    XMLName xml.Name `xml:"book"`
11    Title   string   `xml:"title"`
12    Author  string   `xml:"author"`
13    Year    int      `xml:"year"`
14    ISBN    string   `xml:"isbn,attr"`  // Attribute, not element
15}
16
17func main() {
18    book := Book{
19        Title:  "The Go Programming Language",
20        Author: "Donovan & Kernighan",
21        Year:   2015,
22        ISBN:   "978-0-134190-44-4",
23    }
24
25    xmlData, _ := xml.MarshalIndent(book, "", "  ")
26    fmt.Println(xml.Header + string(xmlData))
27
28    // Unmarshal
29    var decoded Book
30    xml.Unmarshal(xmlData, &decoded)
31    fmt.Printf("\nBook: %s by %s (%d)\n", decoded.Title, decoded.Author, decoded.Year)
32}

CSV - Tabular Data and Excel Compatibility

CSV is perfect for data exports, reports, and Excel integration.

 1// run
 2package main
 3
 4import (
 5    "encoding/csv"
 6    "fmt"
 7    "os"
 8)
 9
10func main() {
11    records := [][]string{
12        {"Name", "Age", "City"},
13        {"Alice", "25", "NYC"},
14        {"Bob", "30", "LA"},
15        {"Carol", "35", "SF"},
16    }
17
18    writer := csv.NewWriter(os.Stdout)
19    defer writer.Flush()
20
21    for _, record := range records {
22        writer.Write(record)
23    }
24
25    fmt.Println("\n✓ CSV written to stdout")
26}

Base64 - Binary Data Encoding

Base64 encodes binary data as ASCII text for JSON/XML transmission.

 1// run
 2package main
 3
 4import (
 5    "encoding/base64"
 6    "fmt"
 7)
 8
 9func main() {
10    data := "Hello, World! 🌍"
11
12    // Encode
13    encoded := base64.StdEncoding.EncodeToString([]byte(data))
14    fmt.Println("Encoded:", encoded)
15
16    // Decode
17    decoded, _ := base64.StdEncoding.DecodeString(encoded)
18    fmt.Println("Decoded:", string(decoded))
19
20    // URL-safe encoding (for URLs and filenames)
21    urlSafe := base64.URLEncoding.EncodeToString([]byte(data))
22    fmt.Println("\nURL-safe:", urlSafe)
23}

Best Practices - Production-Ready JSON Code

These best practices will save you from the most common and dangerous JSON pitfalls in production systems.

1. Always Use Struct Tags for API Contracts

Why: API field names should be independent of Go naming conventions and stable over time.

 1// ❌ BAD: Go names leak into API
 2type User struct {
 3    UserID    int    // JSON: "UserID" - breaks if you refactor
 4    FirstName string // JSON: "FirstName"
 5}
 6
 7// ✅ GOOD: Explicit API contract
 8type User struct {
 9    UserID    int    `json:"user_id"`      // Stable API field name
10    FirstName string `json:"first_name"`   // Won't change with refactoring
11}

2. Use omitempty for Optional Fields

Why: Reduces JSON size by 20-50%, improves API clarity, reduces bandwidth costs.

 1// ❌ BAD: Always sends zero values
 2type User struct {
 3    Name     string `json:"name"`
 4    Age      int    `json:"age"`      // Sends "age": 0 even if not set
 5    Website  string `json:"website"`  // Sends "website": "" always
 6}
 7// Result: {"name":"Alice","age":0,"website":""}  ← Waste of bandwidth
 8
 9// ✅ GOOD: Only send set fields
10type User struct {
11    Name    string `json:"name"`
12    Age     int    `json:"age,omitempty"`      // Omits if 0
13    Website string `json:"website,omitempty"`  // Omits if ""
14}
15// Result: {"name":"Alice"}  ← Clean and efficient

Important omitempty Behavior:

 1// Zero values that trigger omission:
 2false, 0, "", nil, nil pointer, empty slice, empty map, empty struct
 3
 4// ⚠️ Be careful with booleans:
 5type User struct {
 6    IsActive bool `json:"is_active,omitempty"`
 7}
 8
 9user := User{IsActive: false}
10// JSON: {}  ← Missing field! Client can't tell false from unset
11
12// Solution: Use pointer for truly optional booleans
13type User struct {
14    IsActive *bool `json:"is_active,omitempty"`
15}

3. Use Pointers for Optional Fields

Why: Distinguish between "not set" (nil) vs "set to zero value" (0, false, "").

 1// Problem: Can't tell if Age is unset or actually 0
 2type User struct {
 3    Name string `json:"name"`
 4    Age  int    `json:"age,omitempty"`  // 0 might be meaningful!
 5}
 6
 7// ✅ SOLUTION: Use pointer
 8type User struct {
 9    Name string `json:"name"`
10    Age  *int   `json:"age,omitempty"`  // nil = not set, &0 = zero
11}
12
13// Usage:
14age := 25
15user1 := User{Name: "Alice", Age: &age}  // Age is set to 25
16user2 := User{Name: "Bob"}               // Age is nil (not set)
17
18// In PATCH requests, this lets you distinguish:
19// {"age": null}  → Set age to null (clear it)
20// {}             → Don't change age
21// {"age": 0}     → Set age to 0

4. Always Check Unmarshal Errors

Why: Invalid JSON causes silent failures or panics that crash production systems.

 1// ❌ BAD: Ignoring errors
 2var user User
 3json.Unmarshal(data, &user)  // What if data is invalid?
 4useUser(user)  // Might use zero-value struct!
 5
 6// ✅ GOOD: Handle errors
 7var user User
 8if err := json.Unmarshal(data, &user); err != nil {
 9    return fmt.Errorf("invalid JSON: %w", err)
10}
11// user is guaranteed valid here

5. Use json.RawMessage for Delayed Parsing

Why: Parse different data shapes based on other fields, improve performance by parsing only what you need.

 1// Example: API response where "data" shape depends on "type"
 2type Response struct {
 3    Type string          `json:"type"`
 4    Data json.RawMessage `json:"data"`  // Parse later!
 5}
 6
 7func handleResponse(data []byte) error {
 8    var resp Response
 9    if err := json.Unmarshal(data, &resp); err != nil {
10        return err
11    }
12
13    // Parse Data based on Type
14    switch resp.Type {
15    case "user":
16        var user User
17        return json.Unmarshal(resp.Data, &user)
18    case "order":
19        var order Order
20        return json.Unmarshal(resp.Data, &order)
21    default:
22        return fmt.Errorf("unknown type: %s", resp.Type)
23    }
24}

6. Validate After Unmarshaling

Why: JSON being valid doesn't mean data is valid. Protect your system from malicious or malformed data.

 1type User struct {
 2    Email string `json:"email"`
 3    Age   int    `json:"age"`
 4}
 5
 6func (u *User) Validate() error {
 7    if !strings.Contains(u.Email, "@") {
 8        return errors.New("invalid email")
 9    }
10    if u.Age < 0 || u.Age > 150 {
11        return errors.New("invalid age")
12    }
13    return nil
14}
15
16// Usage:
17var user User
18if err := json.Unmarshal(data, &user); err != nil {
19    return err
20}
21if err := user.Validate(); err != nil {  // Always validate!
22    return fmt.Errorf("validation failed: %w", err)
23}

7. Implement Custom Marshalers for Special Types

Why: Control exact encoding for timestamps, enums, money, sensitive data, etc.

 1// Example: Always lowercase and validate emails
 2type Email string
 3
 4func (e Email) MarshalJSON() ([]byte, error) {
 5    return json.Marshal(strings.ToLower(string(e)))
 6}
 7
 8func (e *Email) UnmarshalJSON(data []byte) error {
 9    var s string
10    if err := json.Unmarshal(data, &s); err != nil {
11        return err
12    }
13    if !strings.Contains(s, "@") {
14        return errors.New("invalid email format")
15    }
16    *e = Email(strings.ToLower(s))
17    return nil
18}

8. Use Encoder/Decoder for Streaming

Why: Avoid loading entire payload in memory. Critical for large files, HTTP responses, and high-traffic systems.

 1// ❌ BAD: Loads 100MB into memory
 2data, _ := io.ReadAll(resp.Body)
 3var result BigStruct
 4json.Unmarshal(data, &result)
 5
 6// ✅ GOOD: Streams data, uses ~10MB
 7var result BigStruct
 8if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
 9    return err
10}

9. Never Marshal Recursive Structures

Problem: Causes infinite loop and stack overflow, crashing your application.

 1// ❌ BAD: Infinite recursion if Next points to parent
 2type Node struct {
 3    Value int   `json:"value"`
 4    Next  *Node `json:"next"`  // Cycles cause infinite loop!
 5}
 6
 7// ✅ SOLUTION: Use IDs for references
 8type Node struct {
 9    Value  int    `json:"value"`
10    NextID string `json:"next_id,omitempty"`  // Reference by ID
11}

10. Use MarshalIndent for Debugging Only

Why: MarshalIndent is 2-3x slower and creates 30-50% larger output.

1// ✅ PRODUCTION: Compact JSON
2data, _ := json.Marshal(user)  // Fast, small
3
4// ✅ DEBUGGING/LOGS: Pretty JSON
5data, _ := json.MarshalIndent(user, "", "  ")  // Readable, slower

Common Pitfalls - What NOT to Do

Learn from these common mistakes that cause production incidents.

1. Forgetting to Export Fields

Problem: Unexported fields are silently ignored, causing data loss.

 1// ❌ WRONG: Fields not exported
 2type User struct {
 3    id   int    // lowercase - not marshaled!
 4    name string // lowercase - not marshaled!
 5}
 6
 7user := User{id: 1, name: "Alice"}
 8data, _ := json.Marshal(user)
 9fmt.Println(string(data))  // Output: {}  ← Empty! Silent data loss
10
11// ✅ CORRECT: Export fields (capitalize)
12type User struct {
13    ID   int    `json:"id"`    // Exported
14    Name string `json:"name"`  // Exported
15}

Why: Go's reflection can only access exported fields. The JSON package uses reflection internally, so unexported fields are invisible to it.

2. Not Handling Unmarshal Errors

Problem: Invalid JSON produces zero values, leading to silent bugs and corrupted data.

 1// ❌ WRONG: Ignoring errors
 2var user User
 3json.Unmarshal([]byte(`invalid json`), &user)
 4fmt.Println(user.Name)  // Prints "", no error! Dangerous in production
 5
 6// ✅ CORRECT: Check errors
 7var user User
 8if err := json.Unmarshal([]byte(`{"name":"Alice"}`), &user); err != nil {
 9    log.Fatal("Invalid JSON:", err)
10}
11// Now you know user is valid

3. Unsafe Type Assertions

Problem: Type assertions on interface{} cause panics if wrong type, crashing your application.

 1// ❌ DANGEROUS: Panic if wrong type
 2var data map[string]interface{}
 3json.Unmarshal(jsonBytes, &data)
 4
 5name := data["name"].(string)  // Panics if name is not a string!
 6age := data["age"].(int)       // Panics! JSON numbers are float64, not int!
 7
 8// ✅ SAFE: Check type assertion
 9name, ok := data["name"].(string)
10if !ok {
11    return errors.New("name is not a string")
12}
13
14// ✅ BETTER: Use structs for type safety
15type User struct {
16    Name string `json:"name"`
17    Age  int    `json:"age"`
18}
19var user User
20json.Unmarshal(jsonBytes, &user)  // Type-safe, no assertions needed

JSON Number Type Gotcha:

 1// JSON numbers are ALWAYS decoded as float64!
 2jsonData := `{"count": 42}`
 3
 4var data map[string]interface{}
 5json.Unmarshal([]byte(jsonData), &data)
 6
 7// ❌ WRONG: Assumes int - PANIC!
 8count := data["count"].(int)
 9
10// ✅ CORRECT: Numbers are float64
11count := int(data["count"].(float64))

4. Wrong Struct Tag Syntax

Problem: Incorrect tag syntax causes silent failures - your tags are ignored.

 1// ❌ WRONG: Various syntax errors
 2type User struct {
 3    Name string `json:"name omitempty"`   // Missing comma!
 4    Age  int    `json: "age"`             // Space after colon!
 5    City string `json:city`               // Missing quotes!
 6    ID   int    `json:"id,omit_empty"`    // Typo: omit_empty
 7}
 8
 9// ✅ CORRECT: Proper syntax
10type User struct {
11    Name string `json:"name,omitempty"`   // Comma separates options
12    Age  int    `json:"age"`              // No space after colon
13    City string `json:"city"`             // Quotes required
14    ID   int    `json:"id,omitempty"`     // Correct option name
15}

5. Misunderstanding omitempty

Problem: omitempty omits zero values, which isn't always what you want.

 1// Problem: false is omitted, losing important data
 2type User struct {
 3    IsAdmin bool `json:"is_admin,omitempty"`
 4}
 5
 6user := User{IsAdmin: false}
 7data, _ := json.Marshal(user)
 8// JSON: {}  ← Field missing! Client can't tell false from unset
 9
10// ✅ SOLUTION 1: Don't use omitempty for booleans
11type User struct {
12    IsAdmin bool `json:"is_admin"`  // Always included
13}
14// JSON: {"is_admin":false}
15
16// ✅ SOLUTION 2: Use pointer for optional booleans
17type User struct {
18    IsAdmin *bool `json:"is_admin,omitempty"`
19}
20
21isAdmin := false
22user := User{IsAdmin: &isAdmin}
23// JSON: {"is_admin":false}  ← Explicitly set to false

6. Not Using Pointers for Optional Fields

Problem: Can't distinguish "not provided" from "provided as zero value" in PATCH requests.

 1// Problem: How to tell "age not provided" from "age is 0"?
 2type User struct {
 3    Name string `json:"name"`
 4    Age  int    `json:"age,omitempty"`
 5}
 6
 7// PATCH /users/123 with {"name": "Alice"}
 8// Expected: Don't update age
 9// Actual: Age becomes 0! (zero value for int)
10
11// ✅ SOLUTION: Use pointers for PATCH updates
12type User struct {
13    Name string `json:"name"`
14    Age  *int   `json:"age,omitempty"`
15}
16
17// Now you can distinguish:
18// {"age": null}  → Set age to null (clear it)
19// {}             → Don't change age (nil pointer)
20// {"age": 0}     → Set age to 0 (&0)

7. Forgetting to Pass Pointer to Unmarshal

Problem: Unmarshal requires a pointer to modify the value. Passing a value does nothing.

1// ❌ WRONG: Passing value, not pointer
2var user User
3json.Unmarshal(data, user)  // Does nothing! user is still zero value
4
5// ✅ CORRECT: Pass pointer (note the &)
6var user User
7json.Unmarshal(data, &user)  // ← Ampersand!

8. Marshal/Unmarshal in Loops Without Pooling

Problem: Excessive allocations in hot paths cause memory pressure and slow performance.

 1// ❌ SLOW: Allocates for every iteration
 2for _, item := range items {
 3    data, _ := json.Marshal(item)  // Allocates new buffer each time!
 4    send(data)
 5}
 6
 7// ✅ FASTER: Reuse encoder (50% faster)
 8var buf bytes.Buffer
 9encoder := json.NewEncoder(&buf)
10
11for _, item := range items {
12    buf.Reset()  // Reuse buffer
13    encoder.Encode(item)
14    send(buf.Bytes())
15}

9. Unmarshaling Numbers into Wrong Types

Problem: JSON numbers are float64, conversions can lose precision for large integers.

 1// JSON with large integer (Twitter ID, timestamp, etc.)
 2jsonData := `{"id": 9007199254740993}`  // > 2^53 (float64 precision limit)
 3
 4// ❌ WRONG: Loses precision
 5var data map[string]interface{}
 6json.Unmarshal([]byte(jsonData), &data)
 7id := int64(data["id"].(float64))  // Precision loss!
 8
 9// ✅ CORRECT: Use json.Number for large integers
10decoder := json.NewDecoder(bytes.NewReader([]byte(jsonData)))
11decoder.UseNumber()  // Preserve large integers as strings internally
12
13var data map[string]interface{}
14decoder.Decode(&data)
15
16idNum := data["id"].(json.Number)
17id, _ := idNum.Int64()  // Preserves full precision

10. Encoding Nil Slices vs Empty Slices Inconsistently

Problem: nil slice encodes as null, empty slice as []. This inconsistency confuses API clients.

 1type Response struct {
 2    Items []string `json:"items"`
 3}
 4
 5// nil slice → null
 6resp1 := Response{Items: nil}
 7data1, _ := json.Marshal(resp1)
 8fmt.Println(string(data1))  // {"items":null}  ← Inconsistent!
 9
10// Empty slice → []
11resp2 := Response{Items: []string{}}
12data2, _ := json.Marshal(resp2)
13fmt.Println(string(data2))  // {"items":[]}  ← What we usually want
14
15// ✅ SOLUTION: Always initialize slices to empty, never leave as nil
16type Response struct {
17    Items []string `json:"items"`
18}
19
20func NewResponse() Response {
21    return Response{
22        Items: []string{},  // or make([]string, 0)
23    }
24}
25
26// Or use field initialization
27resp := Response{Items: []string{}}  // Always empty array in JSON

Why This Matters: Many JSON parsers and API clients treat null and [] differently. Some clients crash on null when expecting an array. Consistency is critical for API reliability.

JSON Performance Optimization

Performance matters when your API handles thousands of requests per second. Understanding JSON performance characteristics helps you build systems that scale efficiently.

Benchmarking JSON Operations

Understanding the Cost of JSON:

 1// run
 2package main
 3
 4import (
 5    "encoding/json"
 6    "fmt"
 7    "time"
 8)
 9
10type User struct {
11    ID       int64  `json:"id"`
12    Name     string `json:"name"`
13    Email    string `json:"email"`
14    IsActive bool   `json:"is_active"`
15}
16
17func main() {
18    user := User{
19        ID:       12345,
20        Name:     "Alice Johnson",
21        Email:    "alice@example.com",
22        IsActive: true,
23    }
24
25    // Benchmark Marshal
26    iterations := 100000
27    start := time.Now()
28
29    for i := 0; i < iterations; i++ {
30        json.Marshal(user)
31    }
32
33    marshalTime := time.Since(start)
34    fmt.Printf("Marshal %d times: %v (%.2f ns/op)\n",
35        iterations, marshalTime, float64(marshalTime.Nanoseconds())/float64(iterations))
36
37    // Benchmark Unmarshal
38    data, _ := json.Marshal(user)
39    start = time.Now()
40
41    for i := 0; i < iterations; i++ {
42        var u User
43        json.Unmarshal(data, &u)
44    }
45
46    unmarshalTime := time.Since(start)
47    fmt.Printf("Unmarshal %d times: %v (%.2f ns/op)\n",
48        iterations, unmarshalTime, float64(unmarshalTime.Nanoseconds())/float64(iterations))
49
50    // Memory comparison
51    fmt.Printf("\nJSON size: %d bytes\n", len(data))
52}

Typical Performance Numbers:

Operation              | Time      | Allocations | Notes
-----------------------|-----------|-------------|------------------
Marshal small struct   | 500 ns    | 1 alloc     | Typical REST response
Unmarshal small struct | 800 ns    | 2-3 allocs  | Reflection overhead
Marshal large struct   | 5-10 µs   | 3-5 allocs  | 1000+ fields
Stream decode          | 50-100 µs | Minimal     | Large JSON arrays

Optimization Techniques

1. Reuse Buffers to Reduce Allocations:

 1// run
 2package main
 3
 4import (
 5    "bytes"
 6    "encoding/json"
 7    "fmt"
 8    "sync"
 9)
10
11type User struct {
12    Name  string `json:"name"`
13    Email string `json:"email"`
14}
15
16// Global buffer pool for reuse
17var bufferPool = sync.Pool{
18    New: func() interface{} {
19        return new(bytes.Buffer)
20    },
21}
22
23// ❌ SLOW: Allocates new buffer each time
24func marshalSlow(user User) []byte {
25    data, _ := json.Marshal(user)
26    return data
27}
28
29// ✅ FAST: Reuses buffers (2x faster, 50% less memory)
30func marshalFast(user User) []byte {
31    buf := bufferPool.Get().(*bytes.Buffer)
32    buf.Reset()
33    defer bufferPool.Put(buf)
34
35    encoder := json.NewEncoder(buf)
36    encoder.Encode(user)
37
38    // Copy data before returning buffer to pool
39    result := make([]byte, buf.Len())
40    copy(result, buf.Bytes())
41    return result
42}
43
44func main() {
45    user := User{Name: "Alice", Email: "alice@example.com"}
46
47    // Using pooled buffers
48    data := marshalFast(user)
49    fmt.Printf("Encoded: %s\n", data)
50
51    // Buffer is returned to pool and reused on next call
52    data2 := marshalFast(User{Name: "Bob", Email: "bob@example.com"})
53    fmt.Printf("Encoded: %s\n", data2)
54}

2. Use Encoder/Decoder for Streaming:

 1// ❌ BAD: Loads entire response into memory
 2func fetchUsers() ([]User, error) {
 3    resp, _ := http.Get("https://api.example.com/users")
 4    defer resp.Body.Close()
 5
 6    data, _ := io.ReadAll(resp.Body)  // 100 MB in memory!
 7
 8    var users []User
 9    json.Unmarshal(data, &users)
10    return users, nil
11}
12
13// ✅ GOOD: Streams data, uses constant memory
14func fetchUsersStreaming() ([]User, error) {
15    resp, _ := http.Get("https://api.example.com/users")
16    defer resp.Body.Close()
17
18    var users []User
19    decoder := json.NewDecoder(resp.Body)  // Streams from network!
20    decoder.Decode(&users)
21    return users, nil
22}

3. Avoid Interface{} When Possible:

 1// ❌ SLOW: Interface{} forces runtime type checks
 2func processGeneric(data []byte) error {
 3    var result map[string]interface{}
 4    json.Unmarshal(data, &result)
 5
 6    // Every field access requires type assertion (slow!)
 7    name := result["name"].(string)
 8    age := int(result["age"].(float64))
 9    return nil
10}
11
12// ✅ FAST: Struct provides compile-time types (2-3x faster)
13type User struct {
14    Name string `json:"name"`
15    Age  int    `json:"age"`
16}
17
18func processTyped(data []byte) error {
19    var user User
20    json.Unmarshal(data, &user)
21
22    // Direct field access, no type assertions
23    name := user.Name
24    age := user.Age
25    return nil
26}

4. Use json.RawMessage for Partial Parsing:

 1// Only parse what you need for 3-5x speedup
 2type Response struct {
 3    Type string          `json:"type"`
 4    Data json.RawMessage `json:"data"`  // Don't parse yet!
 5}
 6
 7func handleResponse(payload []byte) error {
 8    var resp Response
 9    if err := json.Unmarshal(payload, &resp); err != nil {
10        return err
11    }
12
13    // Only parse Data if Type matches what we need
14    if resp.Type == "user" {
15        var user User
16        return json.Unmarshal(resp.Data, &user)
17    }
18
19    // Skip parsing Data entirely for other types
20    return nil
21}

5. Pre-allocate Slices When Size is Known:

 1// ❌ SLOW: Multiple reallocations as slice grows
 2var users []User
 3for decoder.More() {
 4    var user User
 5    decoder.Decode(&user)
 6    users = append(users, user)  // Reallocs when capacity exceeded
 7}
 8
 9// ✅ FAST: Allocate once if you know the size
10users := make([]User, 0, expectedCount)  // Pre-allocate capacity
11for decoder.More() {
12    var user User
13    decoder.Decode(&user)
14    users = append(users, user)  // No reallocations!
15}

When to Optimize JSON Performance

Profile before optimizing. Use Go's profiling tools to identify bottlenecks:

1# CPU profiling
2go test -bench=. -cpuprofile=cpu.prof
3go tool pprof cpu.prof
4
5# Memory profiling
6go test -bench=. -memprofile=mem.prof
7go tool pprof mem.prof

Optimize when:

• ✅ Handling >1000 requests/second
• ✅ Processing large JSON files (>10 MB)
• ✅ JSON encoding/decoding appears in CPU profile
• ✅ Memory allocations are causing GC pressure

Don't optimize when:

• ❌ Handling <100 requests/second
• ❌ JSON size is <1 KB
• ❌ Network latency dominates (no point optimizing 1ms JSON when network is 100ms)
• ❌ Profiling shows JSON is <5% of CPU time

Error Handling Patterns

Robust error handling is critical for production JSON processing. Invalid JSON, unexpected formats, and malicious payloads must be handled gracefully.

Comprehensive Error Handling

Basic Error Handling Pattern:

 1// run
 2package main
 3
 4import (
 5    "encoding/json"
 6    "fmt"
 7    "errors"
 8)
 9
10type User struct {
11    Name  string `json:"name"`
12    Email string `json:"email"`
13    Age   int    `json:"age"`
14}
15
16func (u *User) Validate() error {
17    if u.Name == "" {
18        return errors.New("name is required")
19    }
20    if u.Email == "" {
21        return errors.New("email is required")
22    }
23    if u.Age < 0 || u.Age > 150 {
24        return fmt.Errorf("invalid age: %d", u.Age)
25    }
26    return nil
27}
28
29func parseUser(data []byte) (*User, error) {
30    var user User
31
32    // Step 1: Check JSON syntax
33    if err := json.Unmarshal(data, &user); err != nil {
34        return nil, fmt.Errorf("invalid JSON syntax: %w", err)
35    }
36
37    // Step 2: Validate business rules
38    if err := user.Validate(); err != nil {
39        return nil, fmt.Errorf("validation failed: %w", err)
40    }
41
42    return &user, nil
43}
44
45func main() {
46    // Test valid JSON
47    validJSON := []byte(`{"name":"Alice","email":"alice@example.com","age":25}`)
48    user, err := parseUser(validJSON)
49    if err != nil {
50        fmt.Printf("Error: %v\n", err)
51    } else {
52        fmt.Printf("✓ Valid user: %s\n", user.Name)
53    }
54
55    // Test invalid JSON syntax
56    invalidJSON := []byte(`{"name":"Bob"`)
57    _, err = parseUser(invalidJSON)
58    if err != nil {
59        fmt.Printf("✗ %v\n", err)
60    }
61
62    // Test validation failure
63    invalidAge := []byte(`{"name":"Carol","email":"carol@example.com","age":200}`)
64    _, err = parseUser(invalidAge)
65    if err != nil {
66        fmt.Printf("✗ %v\n", err)
67    }
68}

Handling Specific JSON Errors

Identifying Error Types:

 1func handleJSONError(err error) {
 2    var syntaxErr *json.SyntaxError
 3    var unmarshalTypeErr *json.UnmarshalTypeError
 4
 5    switch {
 6    case errors.As(err, &syntaxErr):
 7        fmt.Printf("JSON syntax error at byte offset %d\n", syntaxErr.Offset)
 8
 9    case errors.As(err, &unmarshalTypeErr):
10        fmt.Printf("Type mismatch: expected %v but got %v for field %q\n",
11            unmarshalTypeErr.Type, unmarshalTypeErr.Value, unmarshalTypeErr.Field)
12
13    case errors.Is(err, io.EOF):
14        fmt.Println("Unexpected end of JSON input")
15
16    default:
17        fmt.Printf("Unknown JSON error: %v\n", err)
18    }
19}

Production Error Response Pattern

HTTP API Error Handling:

 1type APIError struct {
 2    Code    string `json:"code"`
 3    Message string `json:"message"`
 4    Details string `json:"details,omitempty"`
 5}
 6
 7func handleRequest(w http.ResponseWriter, r *http.Request) {
 8    var user User
 9
10    // Decode request body
11    decoder := json.NewDecoder(r.Body)
12    decoder.DisallowUnknownFields()  // Reject extra fields
13
14    if err := decoder.Decode(&user); err != nil {
15        var syntaxErr *json.SyntaxError
16        var unmarshalTypeErr *json.UnmarshalTypeError
17
18        switch {
19        case errors.As(err, &syntaxErr):
20            sendError(w, http.StatusBadRequest, APIError{
21                Code:    "invalid_json",
22                Message: "Request body contains malformed JSON",
23                Details: fmt.Sprintf("at position %d", syntaxErr.Offset),
24            })
25            return
26
27        case errors.As(err, &unmarshalTypeErr):
28            sendError(w, http.StatusBadRequest, APIError{
29                Code:    "type_mismatch",
30                Message: fmt.Sprintf("Invalid type for field %q", unmarshalTypeErr.Field),
31                Details: fmt.Sprintf("expected %v", unmarshalTypeErr.Type),
32            })
33            return
34
35        case errors.Is(err, io.EOF):
36            sendError(w, http.StatusBadRequest, APIError{
37                Code:    "empty_body",
38                Message: "Request body is empty",
39            })
40            return
41
42        default:
43            sendError(w, http.StatusBadRequest, APIError{
44                Code:    "invalid_request",
45                Message: "Could not decode request body",
46            })
47            return
48        }
49    }
50
51    // Validate business rules
52    if err := user.Validate(); err != nil {
53        sendError(w, http.StatusUnprocessableEntity, APIError{
54            Code:    "validation_failed",
55            Message: "User data validation failed",
56            Details: err.Error(),
57        })
58        return
59    }
60
61    // Success response
62    w.WriteHeader(http.StatusOK)
63    json.NewEncoder(w).Encode(user)
64}
65
66func sendError(w http.ResponseWriter, status int, apiErr APIError) {
67    w.Header().Set("Content-Type", "application/json")
68    w.WriteHeader(status)
69    json.NewEncoder(w).Encode(apiErr)
70}

Limiting Input Size

Protect Against Large Payloads:

 1// Limit request body size to prevent DoS attacks
 2func limitedDecoder(r *http.Request, maxBytes int64) *json.Decoder {
 3    // Create limited reader (prevents reading beyond maxBytes)
 4    limitedReader := io.LimitReader(r.Body, maxBytes)
 5    return json.NewDecoder(limitedReader)
 6}
 7
 8func handleRequest(w http.ResponseWriter, r *http.Request) {
 9    const maxBodySize = 1 << 20  // 1 MB limit
10
11    decoder := limitedDecoder(r, maxBodySize)
12
13    var user User
14    if err := decoder.Decode(&user); err != nil {
15        if err.Error() == "http: request body too large" {
16            http.Error(w, "Request body too large (max 1 MB)", http.StatusRequestEntityTooLarge)
17            return
18        }
19        http.Error(w, "Invalid JSON", http.StatusBadRequest)
20        return
21    }
22
23    // Process user...
24}

Testing JSON Code

Testing JSON encoding/decoding is essential for reliable APIs. Here are patterns for comprehensive JSON testing.

Unit Testing Marshal/Unmarshal

Basic Test Pattern:

 1// run
 2package main
 3
 4import (
 5    "encoding/json"
 6    "testing"
 7    "reflect"
 8)
 9
10type User struct {
11    Name  string `json:"name"`
12    Email string `json:"email"`
13    Age   int    `json:"age"`
14}
15
16func TestUserMarshal(t *testing.T) {
17    user := User{
18        Name:  "Alice",
19        Email: "alice@example.com",
20        Age:   25,
21    }
22
23    data, err := json.Marshal(user)
24    if err != nil {
25        t.Fatalf("Marshal failed: %v", err)
26    }
27
28    expected := `{"name":"Alice","email":"alice@example.com","age":25}`
29    if string(data) != expected {
30        t.Errorf("Got %s, want %s", data, expected)
31    }
32}
33
34func TestUserUnmarshal(t *testing.T) {
35    jsonData := []byte(`{"name":"Bob","email":"bob@example.com","age":30}`)
36
37    var user User
38    err := json.Unmarshal(jsonData, &user)
39    if err != nil {
40        t.Fatalf("Unmarshal failed: %v", err)
41    }
42
43    if user.Name != "Bob" {
44        t.Errorf("Name = %q, want %q", user.Name, "Bob")
45    }
46    if user.Email != "bob@example.com" {
47        t.Errorf("Email = %q, want %q", user.Email, "bob@example.com")
48    }
49    if user.Age != 30 {
50        t.Errorf("Age = %d, want %d", user.Age, 30)
51    }
52}
53
54func TestRoundTrip(t *testing.T) {
55    original := User{Name: "Carol", Email: "carol@example.com", Age: 35}
56
57    // Marshal
58    data, err := json.Marshal(original)
59    if err != nil {
60        t.Fatalf("Marshal failed: %v", err)
61    }
62
63    // Unmarshal
64    var decoded User
65    err = json.Unmarshal(data, &decoded)
66    if err != nil {
67        t.Fatalf("Unmarshal failed: %v", err)
68    }
69
70    // Compare
71    if !reflect.DeepEqual(original, decoded) {
72        t.Errorf("Round trip failed:\nOriginal: %+v\nDecoded:  %+v", original, decoded)
73    }
74}
75
76func main() {
77    // Run tests manually for demonstration
78    t := &testing.T{}
79    TestUserMarshal(t)
80    TestUserUnmarshal(t)
81    TestRoundTrip(t)
82    println("All tests would pass")
83}

Table-Driven Tests

Testing Multiple Cases:

 1func TestUserValidation(t *testing.T) {
 2    tests := []struct {
 3        name    string
 4        json    string
 5        wantErr bool
 6        errMsg  string
 7    }{
 8        {
 9            name:    "valid user",
10            json:    `{"name":"Alice","email":"alice@example.com","age":25}`,
11            wantErr: false,
12        },
13        {
14            name:    "missing name",
15            json:    `{"email":"alice@example.com","age":25}`,
16            wantErr: true,
17            errMsg:  "name is required",
18        },
19        {
20            name:    "invalid age",
21            json:    `{"name":"Alice","email":"alice@example.com","age":200}`,
22            wantErr: true,
23            errMsg:  "invalid age",
24        },
25        {
26            name:    "malformed JSON",
27            json:    `{"name":"Alice"`,
28            wantErr: true,
29            errMsg:  "invalid JSON",
30        },
31    }
32
33    for _, tt := range tests {
34        t.Run(tt.name, func(t *testing.T) {
35            var user User
36            err := json.Unmarshal([]byte(tt.json), &user)
37
38            if tt.wantErr {
39                if err == nil {
40                    t.Errorf("Expected error containing %q, got nil", tt.errMsg)
41                }
42            } else {
43                if err != nil {
44                    t.Errorf("Unexpected error: %v", err)
45                }
46            }
47        })
48    }
49}

Testing Custom Marshalers

Testing MarshalJSON/UnmarshalJSON:

 1type Timestamp time.Time
 2
 3func (t Timestamp) MarshalJSON() ([]byte, error) {
 4    return json.Marshal(time.Time(t).Unix())
 5}
 6
 7func (t *Timestamp) UnmarshalJSON(data []byte) error {
 8    var unix int64
 9    if err := json.Unmarshal(data, &unix); err != nil {
10        return err
11    }
12    *t = Timestamp(time.Unix(unix, 0))
13    return nil
14}
15
16func TestTimestampMarshal(t *testing.T) {
17    ts := Timestamp(time.Unix(1609459200, 0))  // 2021-01-01 00:00:00 UTC
18
19    data, err := json.Marshal(ts)
20    if err != nil {
21        t.Fatalf("Marshal failed: %v", err)
22    }
23
24    if string(data) != "1609459200" {
25        t.Errorf("Got %s, want 1609459200", data)
26    }
27}
28
29func TestTimestampUnmarshal(t *testing.T) {
30    data := []byte("1609459200")
31
32    var ts Timestamp
33    err := json.Unmarshal(data, &ts)
34    if err != nil {
35        t.Fatalf("Unmarshal failed: %v", err)
36    }
37
38    expected := time.Unix(1609459200, 0)
39    if !time.Time(ts).Equal(expected) {
40        t.Errorf("Got %v, want %v", time.Time(ts), expected)
41    }
42}

Golden File Testing

Testing Against Expected JSON:

 1func TestUserJSON(t *testing.T) {
 2    user := User{Name: "Alice", Email: "alice@example.com", Age: 25}
 3
 4    data, _ := json.MarshalIndent(user, "", "  ")
 5
 6    goldenFile := "testdata/user.golden.json"
 7
 8    // Update golden file: go test -update
 9    if *update {
10        os.WriteFile(goldenFile, data, 0644)
11    }
12
13    // Compare with golden file
14    expected, err := os.ReadFile(goldenFile)
15    if err != nil {
16        t.Fatalf("Failed to read golden file: %v", err)
17    }
18
19    if !bytes.Equal(data, expected) {
20        t.Errorf("JSON mismatch:\nGot:\n%s\nWant:\n%s", data, expected)
21    }
22}

Security Considerations

JSON processing has several security implications that must be handled properly in production systems.

Protecting Against JSON Attacks

1. Prevent Denial of Service (DoS) via Large Payloads:

 1// ❌ VULNERABLE: No size limit
 2func handleRequest(w http.ResponseWriter, r *http.Request) {
 3    var data map[string]interface{}
 4    json.NewDecoder(r.Body).Decode(&data)  // Attacker can send 1 GB!
 5}
 6
 7// ✅ PROTECTED: Enforce size limit
 8func handleRequestSafe(w http.ResponseWriter, r *http.Request) {
 9    const maxBytes = 1 << 20  // 1 MB limit
10
11    r.Body = http.MaxBytesReader(w, r.Body, maxBytes)
12
13    var data map[string]interface{}
14    if err := json.NewDecoder(r.Body).Decode(&data); err != nil {
15        http.Error(w, "Payload too large", http.StatusRequestEntityTooLarge)
16        return
17    }
18}

2. Reject Unknown Fields:

 1// ❌ PERMISSIVE: Accepts any fields
 2decoder := json.NewDecoder(r.Body)
 3decoder.Decode(&user)
 4
 5// ✅ STRICT: Rejects unexpected fields
 6decoder := json.NewDecoder(r.Body)
 7decoder.DisallowUnknownFields()  // Returns error if extra fields present
 8if err := decoder.Decode(&user); err != nil {
 9    return fmt.Errorf("unexpected fields in JSON: %w", err)
10}

3. Prevent Deeply Nested JSON (Stack Overflow):

 1// Malicious payload: {"a":{"a":{"a":{"a":...}}}} (10,000 levels deep)
 2// Can cause stack overflow and crash
 3
 4// ✅ PROTECTION: Limit nesting depth
 5func decodeWithDepthLimit(r io.Reader, v interface{}, maxDepth int) error {
 6    dec := json.NewDecoder(r)
 7
 8    // Track depth during decoding
 9    depth := 0
10    for {
11        t, err := dec.Token()
12        if err == io.EOF {
13            break
14        }
15        if err != nil {
16            return err
17        }
18
19        switch t {
20        case json.Delim('{'), json.Delim('['):
21            depth++
22            if depth > maxDepth {
23                return fmt.Errorf("JSON nesting depth exceeds limit of %d", maxDepth)
24            }
25        case json.Delim('}'), json.Delim(']'):
26            depth--
27        }
28    }
29
30    return dec.Decode(v)
31}

Sanitizing Sensitive Data

Never Log Sensitive JSON Fields:

 1type User struct {
 2    Email    string `json:"email"`
 3    Password string `json:"password"`  // Sensitive!
 4    CardNum  string `json:"card_number"`  // Sensitive!
 5}
 6
 7// ❌ BAD: Logs password and card number
 8log.Printf("User data: %+v", user)
 9
10// ✅ GOOD: Custom String() method masks sensitive fields
11func (u User) String() string {
12    return fmt.Sprintf("User{Email: %s, Password: [REDACTED], CardNum: [REDACTED]}", u.Email)
13}
14
15// ✅ BETTER: Separate structs for logging vs storage
16type UserSafe struct {
17    Email string `json:"email"`
18    // No password or card number!
19}
20
21func (u User) Safe() UserSafe {
22    return UserSafe{Email: u.Email}
23}
24
25log.Printf("User: %+v", user.Safe())  // Only logs safe fields

Custom Marshaler to Redact Sensitive Fields:

 1type CreditCard struct {
 2    Number     string `json:"-"`  // Never marshal
 3    LastFour   string `json:"last_four"`
 4    ExpiryDate string `json:"expiry_date"`
 5}
 6
 7func (cc CreditCard) MarshalJSON() ([]byte, error) {
 8    // Only expose last 4 digits
 9    return json.Marshal(map[string]string{
10        "last_four":   cc.Number[len(cc.Number)-4:],
11        "expiry_date": cc.ExpiryDate,
12    })
13}

Preventing Injection Attacks

Validate JSON Content:

 1type SearchQuery struct {
 2    Query string `json:"query"`
 3}
 4
 5func (sq *SearchQuery) Validate() error {
 6    // Prevent SQL injection via JSON
 7    if strings.Contains(sq.Query, ";") ||
 8       strings.Contains(sq.Query, "--") ||
 9       strings.Contains(sq.Query, "/*") {
10        return errors.New("invalid characters in query")
11    }
12
13    // Limit query length
14    if len(sq.Query) > 100 {
15        return errors.New("query too long")
16    }
17
18    return nil
19}

Content-Type Validation

Always Verify Content-Type Header:

 1func handleJSONRequest(w http.ResponseWriter, r *http.Request) {
 2    // ✅ Verify Content-Type before decoding
 3    contentType := r.Header.Get("Content-Type")
 4    if contentType != "application/json" {
 5        http.Error(w, "Content-Type must be application/json", http.StatusUnsupportedMediaType)
 6        return
 7    }
 8
 9    var data map[string]interface{}
10    json.NewDecoder(r.Body).Decode(&data)
11}

Rate Limiting JSON Endpoints

Protect Against Abuse:

 1// Simple rate limiter using token bucket
 2var rateLimiter = rate.NewLimiter(10, 100)  // 10 req/sec, burst of 100
 3
 4func rateLimitedHandler(w http.ResponseWriter, r *http.Request) {
 5    if !rateLimiter.Allow() {
 6        http.Error(w, "Rate limit exceeded", http.StatusTooManyRequests)
 7        return
 8    }
 9
10    // Process JSON request...
11}

Advanced Encoding Formats - Beyond JSON

While JSON is the most common format, production systems often use specialized formats for specific needs like performance, schema validation, or binary efficiency.

Protocol Buffers - High Performance Binary Encoding

Protocol Buffers (protobuf) provide efficient binary serialization with strong schema validation, used by Google, Uber, Netflix, and other high-scale systems.

Why Use Protobuf:

• ✅ 3-10x smaller than JSON (30-70% size reduction)
• ✅ 2-5x faster encoding/decoding
• ✅ Strong schema validation and backwards compatibility
• ✅ Cross-language support (Go, Java, Python, C++, etc.)
• ✅ Code generation for type safety
• ❌ Not human-readable (binary format)
• ❌ Requires schema definition and compilation

Installation:

1go install google.golang.org/protobuf/cmd/protoc-gen-go@latest

Define Schema (user.proto):

 1syntax = "proto3";
 2
 3package pb;
 4
 5option go_package = "./pb";
 6
 7message User {
 8  int64 id = 1;
 9  string name = 2;
10  string email = 3;
11  repeated string tags = 4;
12  map<string, string> metadata = 5;
13  bool is_active = 6;
14}
15
16message UserList {
17  repeated User users = 1;
18  int32 total_count = 2;
19}

Generate Go Code:

1protoc --go_out=. --go_opt=paths=source_relative user.proto

Using Protobuf in Go:

 1// run
 2package main
 3
 4import (
 5	"fmt"
 6	"log"
 7
 8	"google.golang.org/protobuf/proto"
 9	pb "yourproject/pb"  // Generated code
10)
11
12func main() {
13	// Create message
14	user := &pb.User{
15		Id:    1,
16		Name:  "Alice",
17		Email: "alice@example.com",
18		Tags:  []string{"admin", "developer"},
19		Metadata: map[string]string{
20			"role":       "engineer",
21			"department": "platform",
22		},
23		IsActive: true,
24	}
25
26	// Marshal to binary (much smaller than JSON)
27	data, err := proto.Marshal(user)
28	if err != nil {
29		log.Fatal(err)
30	}
31	fmt.Printf("Protobuf size: %d bytes\n", len(data))
32
33	// Unmarshal
34	var decoded pb.User
35	if err := proto.Unmarshal(data, &decoded); err != nil {
36		log.Fatal(err)
37	}
38	fmt.Printf("Name: %s, Email: %s, Active: %v\n",
39		decoded.Name, decoded.Email, decoded.IsActive)
40}

Protobuf vs JSON Size Comparison:

 1// Typical 30-50% size reduction with protobuf
 2User data:
 3- JSON:     120 bytes
 4- Protobuf:  60 bytes (50% smaller)
 5
 61000 users:
 7- JSON:     120 KB
 8- Protobuf:  60 KB (saves 60 KB per request!)
 9
101M requests/day: Saves 60 GB bandwidth/day

MessagePack - Binary JSON Alternative

MessagePack is a binary format similar to JSON but more compact and faster. It's easier to adopt than protobuf (no schema required) but still provides significant benefits.

Installation:

1go get github.com/vmihailenco/msgpack/v5

Basic Usage:

 1// run
 2package main
 3
 4import (
 5	"fmt"
 6	"log"
 7
 8	"github.com/vmihailenco/msgpack/v5"
 9)
10
11type User struct {
12	ID    int64             `msgpack:"id"`
13	Name  string            `msgpack:"name"`
14	Email string            `msgpack:"email"`
15	Meta  map[string]string `msgpack:"meta,omitempty"`
16}
17
18func main() {
19	user := User{
20		ID:    1,
21		Name:  "Alice",
22		Email: "alice@example.com",
23		Meta: map[string]string{
24			"role": "admin",
25		},
26	}
27
28	// Encode (similar to json.Marshal)
29	data, err := msgpack.Marshal(&user)
30	if err != nil {
31		log.Fatal(err)
32	}
33	fmt.Printf("MessagePack size: %d bytes\n", len(data))
34
35	// Decode (similar to json.Unmarshal)
36	var decoded User
37	if err := msgpack.Unmarshal(data, &decoded); err != nil {
38		log.Fatal(err)
39	}
40	fmt.Printf("%+v\n", decoded)
41}

MessagePack vs JSON Performance:

Operation        | JSON     | MessagePack | Speedup
-----------------|----------|-------------|--------
Encode (10k)     | 15 ms    | 8 ms        | 1.9x
Decode (10k)     | 20 ms    | 12 ms       | 1.7x
Size (1 object)  | 100 bytes| 75 bytes    | 25% smaller
Memory (10k)     | 10 MB    | 7 MB        | 30% less

When to Use Each Format:

Format	Best For	Pros	Cons
JSON	REST APIs, configs, human-readable	Universal, debuggable, text	Verbose, slower
Protobuf	High-performance RPCs, microservices	Fastest, smallest, schema	Binary, requires compilation
MessagePack	Redis cache, message queues	Faster than JSON, no schema	Binary, less universal
XML	Legacy systems, SOAP APIs	Enterprise standard	Very verbose, complex
CSV	Data exports, Excel	Simple, human-readable	No nesting, limited types

Practice Exercises

Exercise 1: Student Record Encoder

Learning Objectives: Master basic JSON struct creation, implement encoding/decoding operations, work with data slices, and understand JSON struct tags and serialization fundamentals.

Difficulty: ⭐⭐ Beginner
Time Estimate: 15 minutes

Create a student record system that can serialize and deserialize student data including personal information and academic performance metrics. This exercise teaches you to design proper Go structs for JSON representation, use struct tags for field mapping, handle nested data structures like grade arrays, and implement bidirectional JSON conversion.

Real-World Context: Student record systems are fundamental in educational institutions for managing academic data, tracking performance, and generating reports. Understanding JSON encoding/decoding patterns is crucial for building any data management system that needs to persist or transmit structured information, from user profiles to inventory systems.

Solution

 1// run
 2package main
 3
 4import (
 5    "encoding/json"
 6    "fmt"
 7)
 8
 9type Student struct {
10    Name   string `json:"name"`
11    Age    int    `json:"age"`
12    Grades []int  `json:"grades"`
13}
14
15func main() {
16    student := Student{
17        Name:   "Alice",
18        Age:    20,
19        Grades: []int{85, 90, 92, 88},
20    }
21
22    // Encode to JSON
23    jsonData, err := json.MarshalIndent(student, "", "  ")
24    if err != nil {
25        fmt.Println("Error:", err)
26        return
27    }
28
29    fmt.Println("JSON:")
30    fmt.Println(string(jsonData))
31
32    // Decode from JSON
33    var decoded Student
34    err = json.Unmarshal(jsonData, &decoded)
35    if err != nil {
36        fmt.Println("Error:", err)
37        return
38    }
39
40    fmt.Printf("\nDecoded: %+v\n", decoded)
41}

Exercise 2: Config File Parser

Learning Objectives: Design configuration data structures, parse JSON from strings, handle different data types including booleans and arrays, and implement robust configuration parsing with validation.

Difficulty: ⭐⭐ Beginner
Time Estimate: 20 minutes

Build a configuration parser that can load and validate application settings from JSON strings, handling various data types and providing type-safe access to configuration values. This exercise teaches you to work with mixed data types, implement configuration validation, handle array data like IP lists, and create reusable parsing functions for configuration management.

Real-World Context: Configuration management is essential in virtually every software application. From web servers to microservices, applications need to load settings like database connections, API endpoints, feature flags, and security configurations. Understanding configuration parsing patterns is fundamental for building flexible, deployable software systems.

Solution

 1// run
 2package main
 3
 4import (
 5    "encoding/json"
 6    "fmt"
 7)
 8
 9type Config struct {
10    Host       string   `json:"host"`
11    Port       int      `json:"port"`
12    Debug      bool     `json:"debug"`
13    AllowedIPs []string `json:"allowed_ips"`
14}
15
16func parseConfig(jsonStr string) (*Config, error) {
17    var config Config
18    err := json.Unmarshal([]byte(jsonStr), &config)
19    if err != nil {
20        return nil, err
21    }
22    return &config, nil
23}
24
25func main() {
26    configJSON := `{
27        "host": "localhost",
28        "port": 8080,
29        "debug": true,
30        "allowed_ips": ["127.0.0.1", "192.168.1.1"]
31    }`
32
33    config, err := parseConfig(configJSON)
34    if err != nil {
35        fmt.Println("Error:", err)
36        return
37    }
38
39    fmt.Printf("Server: %s:%d\n", config.Host, config.Port)
40    fmt.Printf("Debug mode: %v\n", config.Debug)
41    fmt.Printf("Allowed IPs: %v\n", config.AllowedIPs)
42}

Exercise 3: API Response Handler

Learning Objectives: Parse complex API responses, handle dynamic data fields with json.RawMessage, implement conditional parsing based on response status, and work with nested JSON structures from external APIs.

Difficulty: ⭐⭐⭐ Intermediate
Time Estimate: 25 minutes

Create an API response handler that can parse and process structured responses from web APIs, handling both success and error cases with different data payloads. This exercise teaches you to work with json.RawMessage for deferred parsing, implement status-based conditional logic, handle dynamic content types, and build robust API client response processing.

Real-World Context: API response handling is fundamental for client applications that consume web services. Every mobile app, web frontend, or microservice needs to parse API responses that contain status information, error messages, and variable data payloads. Understanding these patterns is crucial for building reliable API integrations and error handling.

Solution

 1// run
 2package main
 3
 4import (
 5    "encoding/json"
 6    "fmt"
 7)
 8
 9type APIResponse struct {
10    Status  string          `json:"status"`
11    Message string          `json:"message"`
12    Data    json.RawMessage `json:"data"`
13}
14
15type UserData struct {
16    ID    int    `json:"id"`
17    Name  string `json:"name"`
18    Email string `json:"email"`
19}
20
21func main() {
22    responseJSON := `{
23        "status": "success",
24        "message": "User found",
25        "data": {
26            "id": 123,
27            "name": "Alice",
28            "email": "alice@example.com"
29        }
30    }`
31
32    var resp APIResponse
33    err := json.Unmarshal([]byte(responseJSON), &resp)
34    if err != nil {
35        fmt.Println("Error:", err)
36        return
37    }
38
39    fmt.Println("Status:", resp.Status)
40    fmt.Println("Message:", resp.Message)
41
42    if resp.Status == "success" {
43        var user UserData
44        json.Unmarshal(resp.Data, &user)
45        fmt.Printf("User: %s (%s)\n", user.Name, user.Email)
46    }
47}

Exercise 4: Custom Time Format

Learning Objectives: Implement custom JSON marshaling/unmarshaling, create custom time formats, work with Go's time package, and understand how to control JSON serialization for specific data types.

Difficulty: ⭐⭐⭐ Intermediate
Time Estimate: 20 minutes

Build a custom date type that handles JSON serialization with a specific date format instead of Go's default RFC3339 format. This exercise teaches you to implement MarshalJSON and UnmarshalJSON methods, work with Go's time formatting patterns, handle custom type conversions, and control exactly how your data appears in JSON.

Real-World Context: Custom time formatting is essential when integrating with external systems that expect specific date formats. APIs, databases, and legacy systems often use different date standards, and your Go applications need to adapt. Understanding custom serialization patterns is crucial for building compatible APIs and data exchange systems.

Solution

 1// run
 2package main
 3
 4import (
 5    "encoding/json"
 6    "fmt"
 7    "time"
 8)
 9
10type Date time.Time
11
12const dateFormat = "2006-01-02"
13
14func (d Date) MarshalJSON() ([]byte, error) {
15    return json.Marshal(time.Time(d).Format(dateFormat))
16}
17
18func (d *Date) UnmarshalJSON(data []byte) error {
19    var s string
20    if err := json.Unmarshal(data, &s); err != nil {
21        return err
22    }
23
24    t, err := time.Parse(dateFormat, s)
25    if err != nil {
26        return err
27    }
28
29    *d = Date(t)
30    return nil
31}
32
33type Event struct {
34    Name string `json:"name"`
35    Date Date   `json:"date"`
36}
37
38func main() {
39    event := Event{
40        Name: "Conference",
41        Date: Date(time.Now()),
42    }
43
44    jsonData, _ := json.MarshalIndent(event, "", "  ")
45    fmt.Println(string(jsonData))
46
47    // Unmarshal
48    var decoded Event
49    json.Unmarshal(jsonData, &decoded)
50    fmt.Printf("Event: %s on %v\n", decoded.Name, time.Time(decoded.Date))
51}

Exercise 5: Nested JSON to Flat Struct

Learning Objectives: Extract data from complex nested JSON structures, work with interface{} types for dynamic parsing, navigate nested object hierarchies, and transform complex data into simplified, usable structs.

Difficulty: ⭐⭐⭐ Intermediate
Time Estimate: 25 minutes

Build a data extractor that can navigate complex nested JSON structures and extract relevant fields into flat, easy-to-use Go structs. This exercise teaches you to work with map[string]interface{} for dynamic JSON parsing, perform type assertions safely, navigate nested object hierarchies, and transform complex data into simplified structures for practical use.

Real-World Context: Real-world APIs often return deeply nested JSON structures with more data than you need. Extracting specific fields into flat structures is essential for building clean, efficient applications. This pattern is used in data processing pipelines, API client libraries, and systems that need to normalize data from various sources.

Solution

 1// run
 2package main
 3
 4import (
 5    "encoding/json"
 6    "fmt"
 7)
 8
 9type UserProfile struct {
10    Username string
11    Email    string
12    City     string
13}
14
15func main() {
16    jsonStr := `{
17        "user": {
18            "username": "alice",
19            "email": "alice@example.com",
20            "profile": {
21                "location": {
22                    "city": "NYC"
23                }
24            }
25        }
26    }`
27
28    var raw map[string]interface{}
29    json.Unmarshal([]byte(jsonStr), &raw)
30
31    user := raw["user"].(map[string]interface{})
32    profile := user["profile"].(map[string]interface{})
33    location := profile["location"].(map[string]interface{})
34
35    result := UserProfile{
36        Username: user["username"].(string),
37        Email:    user["email"].(string),
38        City:     location["city"].(string),
39    }
40
41    fmt.Printf("%+v\n", result)
42}

Exercise 6: Settings Manager

Learning Objectives: Implement complete configuration management workflows, handle JSON file I/O operations, build data modification interfaces, and create persistent settings systems with save/load functionality.

Difficulty: ⭐⭐⭐ Intermediate
Time Estimate: 30 minutes

Create a comprehensive settings manager that can load configuration from JSON, modify values programmatically, validate changes, and persist updated settings back to files. This exercise teaches you to build complete data lifecycle management, implement file-based persistence, create fluent APIs for configuration updates, and handle error scenarios in configuration management.

Real-World Context: Settings managers are fundamental components of applications that need to maintain user preferences, system configurations, or feature flags. From desktop applications to web services, the ability to load, modify, and save configuration data is essential for creating customizable, user-friendly software that remembers settings between sessions.

Solution

 1// run
 2package main
 3
 4import (
 5    "encoding/json"
 6    "fmt"
 7)
 8
 9type Settings struct {
10    Theme      string            `json:"theme"`
11    Language   string            `json:"language"`
12    Notifs     bool              `json:"notifications"`
13    CustomVars map[string]string `json:"custom_vars,omitempty"`
14}
15
16func LoadSettings(jsonStr string) (*Settings, error) {
17    var settings Settings
18    err := json.Unmarshal([]byte(jsonStr), &settings)
19    return &settings, err
20}
21
22func (s *Settings) ToJSON() (string, error) {
23    data, err := json.MarshalIndent(s, "", "  ")
24    return string(data), err
25}
26
27func (s *Settings) SetCustomVar(key, value string) {
28    if s.CustomVars == nil {
29        s.CustomVars = make(map[string]string)
30    }
31    s.CustomVars[key] = value
32}
33
34func main() {
35    configJSON := `{
36        "theme": "dark",
37        "language": "en",
38        "notifications": true
39    }`
40
41    settings, err := LoadSettings(configJSON)
42    if err != nil {
43        fmt.Println("Error:", err)
44        return
45    }
46
47    fmt.Println("Loaded settings:", settings.Theme, settings.Language)
48
49    // Modify
50    settings.Theme = "light"
51    settings.SetCustomVar("api_key", "abc123")
52
53    // Save
54    output, _ := settings.ToJSON()
55    fmt.Println("\nUpdated settings:")
56    fmt.Println(output)
57}

Exercise 7: JSON API Data Processor

Learning Objectives: Process complex API response structures, handle nested data arrays and objects, implement data transformation pipelines, and build robust JSON processing for real-world API integrations.

Difficulty: ⭐⭐⭐ Intermediate
Time Estimate: 35 minutes

Build a comprehensive JSON data processor that can handle complex nested API responses, transform data structures, filter information, and convert between different data formats. This exercise teaches you to work with deeply nested JSON structures, implement data transformation logic, handle arrays of objects, and build processing pipelines for API data normalization.

Real-World Context: API data processing is essential for building applications that integrate with external services. Social media platforms, payment processors, analytics services, and data providers all return complex nested JSON that needs to be processed, filtered, and transformed for use in your application. Understanding these patterns is crucial for building data-driven applications.

Solution

  1// run
  2package main
  3
  4import (
  5	"encoding/json"
  6	"fmt"
  7	"time"
  8)
  9
 10type User struct {
 11	ID        int       `json:"id"`
 12	Username  string    `json:"username"`
 13	Email     string    `json:"email"`
 14	FullName  string    `json:"full_name,omitempty"`
 15	IsActive  bool      `json:"is_active"`
 16	CreatedAt time.Time `json:"created_at"`
 17	Profile   *Profile  `json:"profile,omitempty"`
 18}
 19
 20type Profile struct {
 21	Bio         string            `json:"bio,omitempty"`
 22	Website     string            `json:"website,omitempty"`
 23	Location    string            `json:"location,omitempty"`
 24	Skills      []string          `json:"skills,omitempty"`
 25	SocialLinks map[string]string `json:"social_links,omitempty"`
 26}
 27
 28type APIResponse struct {
 29	Success bool        `json:"success"`
 30	Data    interface{} `json:"data,omitempty"`
 31	Error   *APIError   `json:"error,omitempty"`
 32	Meta    *Meta       `json:"meta,omitempty"`
 33}
 34
 35type APIError struct {
 36	Code    string `json:"code"`
 37	Message string `json:"message"`
 38}
 39
 40type Meta struct {
 41	Page       int `json:"page"`
 42	PerPage    int `json:"per_page"`
 43	TotalPages int `json:"total_pages"`
 44	TotalItems int `json:"total_items"`
 45}
 46
 47func main() {
 48	fmt.Println("=== JSON API Data Processor ===\n")
 49
 50	// Create sample user
 51	user := User{
 52		ID:        1,
 53		Username:  "alice",
 54		Email:     "alice@example.com",
 55		FullName:  "Alice Johnson",
 56		IsActive:  true,
 57		CreatedAt: time.Now(),
 58		Profile: &Profile{
 59			Bio:      "Software Engineer at TechCorp",
 60			Website:  "https://alice.dev",
 61			Location: "San Francisco, CA",
 62			Skills:   []string{"Go", "Python", "JavaScript", "Docker"},
 63			SocialLinks: map[string]string{
 64				"github":   "https://github.com/alice",
 65				"twitter":  "https://twitter.com/alice",
 66				"linkedin": "https://linkedin.com/in/alice",
 67			},
 68		},
 69	}
 70
 71	// Marshal to JSON
 72	fmt.Println("1. Marshaling User to JSON:")
 73	userJSON, err := json.MarshalIndent(user, "", "  ")
 74	if err != nil {
 75		fmt.Println("Error:", err)
 76		return
 77	}
 78	fmt.Println(string(userJSON))
 79
 80	// Create success response
 81	successResp := APIResponse{
 82		Success: true,
 83		Data:    user,
 84		Meta: &Meta{
 85			Page:       1,
 86			PerPage:    10,
 87			TotalPages: 5,
 88			TotalItems: 42,
 89		},
 90	}
 91
 92	fmt.Println("\n2. Success API Response:")
 93	respJSON, _ := json.MarshalIndent(successResp, "", "  ")
 94	fmt.Println(string(respJSON))
 95
 96	// Create error response
 97	errorResp := APIResponse{
 98		Success: false,
 99		Error: &APIError{
100			Code:    "USER_NOT_FOUND",
101			Message: "User with ID 999 does not exist",
102		},
103	}
104
105	fmt.Println("\n3. Error API Response:")
106	errJSON, _ := json.MarshalIndent(errorResp, "", "  ")
107	fmt.Println(string(errJSON))
108
109	// Unmarshal from JSON string
110	fmt.Println("\n4. Unmarshaling JSON to struct:")
111	jsonData := `{
112		"id": 2,
113		"username": "bob",
114		"email": "bob@example.com",
115		"is_active": true,
116		"created_at": "2025-01-15T10:00:00Z"
117	}`
118
119	var parsedUser User
120	if err := json.Unmarshal([]byte(jsonData), &parsedUser); err != nil {
121		fmt.Println("Error:", err)
122		return
123	}
124
125	fmt.Printf("Parsed User: %s (%s) - Active: %v\n",
126		parsedUser.Username, parsedUser.Email, parsedUser.IsActive)
127
128	// Handle missing optional fields
129	fmt.Println("\n5. User without profile:")
130	userNoProfile := User{
131		ID:        3,
132		Username:  "charlie",
133		Email:     "charlie@example.com",
134		IsActive:  false,
135		CreatedAt: time.Now(),
136	}
137
138	noProfileJSON, _ := json.MarshalIndent(userNoProfile, "", "  ")
139	fmt.Println(string(noProfileJSON))
140}

Exercise 8: Custom JSON Marshaling

Implement custom JSON marshaling for special data types like time formats and enums.

Solution

  1// run
  2package main
  3
  4import (
  5	"encoding/json"
  6	"fmt"
  7	"strconv"
  8	"strings"
  9	"time"
 10)
 11
 12// Custom time format
 13type CustomTime time.Time
 14
 15func (ct CustomTime) MarshalJSON() ([]byte, error) {
 16	t := time.Time(ct)
 17	formatted := t.Format("2006-01-02 15:04:05")
 18	return json.Marshal(formatted)
 19}
 20
 21func (ct *CustomTime) UnmarshalJSON(data []byte) error {
 22	var s string
 23	if err := json.Unmarshal(data, &s); err != nil {
 24		return err
 25	}
 26
 27	t, err := time.Parse("2006-01-02 15:04:05", s)
 28	if err != nil {
 29		return err
 30	}
 31
 32	*ct = CustomTime(t)
 33	return nil
 34}
 35
 36// Status enum
 37type Status int
 38
 39const (
 40	StatusPending Status = iota
 41	StatusInProgress
 42	StatusCompleted
 43	StatusCancelled
 44)
 45
 46func (s Status) String() string {
 47	return [...]string{"pending", "in_progress", "completed", "cancelled"}[s]
 48}
 49
 50func (s Status) MarshalJSON() ([]byte, error) {
 51	return json.Marshal(s.String())
 52}
 53
 54func (s *Status) UnmarshalJSON(data []byte) error {
 55	var str string
 56	if err := json.Unmarshal(data, &str); err != nil {
 57		return err
 58	}
 59
 60	switch strings.ToLower(str) {
 61	case "pending":
 62		*s = StatusPending
 63	case "in_progress":
 64		*s = StatusInProgress
 65	case "completed":
 66		*s = StatusCompleted
 67	case "cancelled":
 68		*s = StatusCancelled
 69	default:
 70		return fmt.Errorf("invalid status: %s", str)
 71	}
 72
 73	return nil
 74}
 75
 76// Money type
 77type Money int64
 78
 79func (m Money) MarshalJSON() ([]byte, error) {
 80	dollars := float64(m) / 100.0
 81	return json.Marshal(fmt.Sprintf("%.2f", dollars))
 82}
 83
 84func (m *Money) UnmarshalJSON(data []byte) error {
 85	var s string
 86	if err := json.Unmarshal(data, &s); err != nil {
 87		// Try as number
 88		var f float64
 89		if err := json.Unmarshal(data, &f); err != nil {
 90			return err
 91		}
 92		*m = Money(f * 100)
 93		return nil
 94	}
 95
 96	// Parse string like "123.45"
 97	s = strings.TrimPrefix(s, "$")
 98	f, err := strconv.ParseFloat(s, 64)
 99	if err != nil {
100		return err
101	}
102
103	*m = Money(f * 100)
104	return nil
105}
106
107func (m Money) String() string {
108	return fmt.Sprintf("$%.2f", float64(m)/100.0)
109}
110
111// Task with custom types
112type Task struct {
113	ID          int        `json:"id"`
114	Title       string     `json:"title"`
115	Description string     `json:"description,omitempty"`
116	Status      Status     `json:"status"`
117	Budget      Money      `json:"budget"`
118	CreatedAt   CustomTime `json:"created_at"`
119	UpdatedAt   CustomTime `json:"updated_at"`
120}
121
122func main() {
123	fmt.Println("=== Custom JSON Marshaling ===\n")
124
125	// Create task with custom types
126	task := Task{
127		ID:          1,
128		Title:       "Build REST API",
129		Description: "Create a RESTful API using Go",
130		Status:      StatusInProgress,
131		Budget:      Money(250000), // $2500.00
132		CreatedAt:   CustomTime(time.Now().Add(-24 * time.Hour)),
133		UpdatedAt:   CustomTime(time.Now()),
134	}
135
136	// Marshal to JSON
137	fmt.Println("1. Task with custom types:")
138	taskJSON, err := json.MarshalIndent(task, "", "  ")
139	if err != nil {
140		fmt.Println("Error:", err)
141		return
142	}
143	fmt.Println(string(taskJSON))
144
145	// Unmarshal from JSON
146	fmt.Println("\n2. Parsing JSON with custom types:")
147	jsonData := `{
148		"id": 2,
149		"title": "Deploy Application",
150		"status": "completed",
151		"budget": "1500.00",
152		"created_at": "2025-01-14 09:00:00",
153		"updated_at": "2025-01-15 17:30:00"
154	}`
155
156	var parsedTask Task
157	if err := json.Unmarshal([]byte(jsonData), &parsedTask); err != nil {
158		fmt.Println("Error:", err)
159		return
160	}
161
162	fmt.Printf("Task: %s\n", parsedTask.Title)
163	fmt.Printf("Status: %s\n", parsedTask.Status)
164	fmt.Printf("Budget: %s\n", parsedTask.Budget)
165	fmt.Printf("Created: %v\n", time.Time(parsedTask.CreatedAt))
166
167	// Test different status values
168	fmt.Println("\n3. Testing Status enum:")
169	statuses := []string{
170		`{"status": "pending"}`,
171		`{"status": "in_progress"}`,
172		`{"status": "completed"}`,
173		`{"status": "cancelled"}`,
174	}
175
176	for _, s := range statuses {
177		var t Task
178		json.Unmarshal([]byte(s), &t)
179		fmt.Printf("Status: %s\n", t.Status)
180	}
181
182	// Test money parsing
183	fmt.Println("\n4. Testing Money type:")
184	moneyTests := []string{
185		`{"budget": "99.99"}`,
186		`{"budget": 150.50}`,
187		`{"budget": "$250.00"}`,
188	}
189
190	for _, m := range moneyTests {
191		var t Task
192		json.Unmarshal([]byte(m), &t)
193		fmt.Printf("Budget: %s (%d cents)\n", t.Budget, t.Budget)
194	}
195}

Exercise 9: JSON Streaming and Large Data

Process large JSON data using streaming to handle files that don't fit in memory.

Solution

  1// run
  2package main
  3
  4import (
  5	"encoding/json"
  6	"fmt"
  7	"io"
  8	"os"
  9	"strings"
 10)
 11
 12type LogEntry struct {
 13	Timestamp string `json:"timestamp"`
 14	Level     string `json:"level"`
 15	Message   string `json:"message"`
 16	UserID    int    `json:"user_id,omitempty"`
 17	RequestID string `json:"request_id,omitempty"`
 18}
 19
 20type LogStats struct {
 21	TotalEntries int            `json:"total_entries"`
 22	ByLevel      map[string]int `json:"by_level"`
 23	ByUser       map[int]int    `json:"by_user"`
 24	Errors       []LogEntry     `json:"recent_errors,omitempty"`
 25}
 26
 27// Stream process logs with json.Decoder
 28func processLogsStreaming(filename string) (*LogStats, error) {
 29	file, err := os.Open(filename)
 30	if err != nil {
 31		return nil, err
 32	}
 33	defer file.Close()
 34
 35	decoder := json.NewDecoder(file)
 36
 37	stats := &LogStats{
 38		ByLevel: make(map[string]int),
 39		ByUser:  make(map[int]int),
 40		Errors:  make([]LogEntry, 0),
 41	}
 42
 43	// Read array opening bracket
 44	if _, err := decoder.Token(); err != nil {
 45		return nil, err
 46	}
 47
 48	// Process each log entry
 49	for decoder.More() {
 50		var entry LogEntry
 51		if err := decoder.Decode(&entry); err != nil {
 52			return nil, err
 53		}
 54
 55		stats.TotalEntries++
 56		stats.ByLevel[entry.Level]++
 57
 58		if entry.UserID > 0 {
 59			stats.ByUser[entry.UserID]++
 60		}
 61
 62		// Keep last 5 errors
 63		if entry.Level == "ERROR" {
 64			stats.Errors = append(stats.Errors, entry)
 65			if len(stats.Errors) > 5 {
 66				stats.Errors = stats.Errors[1:]
 67			}
 68		}
 69	}
 70
 71	// Read array closing bracket
 72	if _, err := decoder.Token(); err != nil {
 73		return nil, err
 74	}
 75
 76	return stats, nil
 77}
 78
 79// Write logs using json.Encoder
 80func writeLogsStreaming(filename string, entries []LogEntry) error {
 81	file, err := os.Create(filename)
 82	if err != nil {
 83		return err
 84	}
 85	defer file.Close()
 86
 87	encoder := json.NewEncoder(file)
 88	encoder.SetIndent("", "  ")
 89
 90	// Write array
 91	return encoder.Encode(entries)
 92}
 93
 94// Process JSONL format (JSON Lines - one JSON object per line)
 95func processJSONLines(filename string) (*LogStats, error) {
 96	file, err := os.Open(filename)
 97	if err != nil {
 98		return nil, err
 99	}
100	defer file.Close()
101
102	decoder := json.NewDecoder(file)
103
104	stats := &LogStats{
105		ByLevel: make(map[string]int),
106		ByUser:  make(map[int]int),
107		Errors:  make([]LogEntry, 0),
108	}
109
110	for {
111		var entry LogEntry
112		err := decoder.Decode(&entry)
113		if err == io.EOF {
114			break
115		}
116		if err != nil {
117			return nil, err
118		}
119
120		stats.TotalEntries++
121		stats.ByLevel[entry.Level]++
122
123		if entry.UserID > 0 {
124			stats.ByUser[entry.UserID]++
125		}
126
127		if entry.Level == "ERROR" {
128			stats.Errors = append(stats.Errors, entry)
129			if len(stats.Errors) > 5 {
130				stats.Errors = stats.Errors[1:]
131			}
132		}
133	}
134
135	return stats, nil
136}
137
138// Transform logs
139func transformLogs(inputFile, outputFile, levelFilter string) error {
140	input, err := os.Open(inputFile)
141	if err != nil {
142		return err
143	}
144	defer input.Close()
145
146	output, err := os.Create(outputFile)
147	if err != nil {
148		return err
149	}
150	defer output.Close()
151
152	decoder := json.NewDecoder(input)
153	encoder := json.NewEncoder(output)
154	encoder.SetIndent("", "  ")
155
156	// Read and write array brackets
157	if _, err := decoder.Token(); err != nil {
158		return err
159	}
160
161	output.WriteString("[\n")
162
163	first := true
164	for decoder.More() {
165		var entry LogEntry
166		if err := decoder.Decode(&entry); err != nil {
167			return err
168		}
169
170		// Filter by level
171		if levelFilter == "" || strings.EqualFold(entry.Level, levelFilter) {
172			if !first {
173				output.WriteString(",\n")
174			}
175			// Transform: uppercase level
176			entry.Level = strings.ToUpper(entry.Level)
177
178			// Encode without array brackets
179			data, _ := json.MarshalIndent(entry, "  ", "  ")
180			output.Write(data)
181			first = false
182		}
183	}
184
185	output.WriteString("\n]\n")
186
187	if _, err := decoder.Token(); err != nil {
188		return err
189	}
190
191	return nil
192}
193
194func main() {
195	fmt.Println("=== JSON Streaming and Large Data ===\n")
196
197	// Create sample log entries
198	logs := []LogEntry{
199		{Timestamp: "2025-01-15T10:00:00Z", Level: "INFO", Message: "Application started", UserID: 1},
200		{Timestamp: "2025-01-15T10:00:05Z", Level: "DEBUG", Message: "Loading config", UserID: 1},
201		{Timestamp: "2025-01-15T10:00:10Z", Level: "INFO", Message: "Database connected", UserID: 2},
202		{Timestamp: "2025-01-15T10:00:15Z", Level: "WARN", Message: "High memory usage", UserID: 2},
203		{Timestamp: "2025-01-15T10:00:20Z", Level: "ERROR", Message: "Failed to process request", UserID: 3, RequestID: "req-123"},
204		{Timestamp: "2025-01-15T10:00:25Z", Level: "INFO", Message: "Request completed", UserID: 1},
205		{Timestamp: "2025-01-15T10:00:30Z", Level: "ERROR", Message: "Database timeout", UserID: 2, RequestID: "req-124"},
206		{Timestamp: "2025-01-15T10:00:35Z", Level: "INFO", Message: "Shutting down", UserID: 1},
207	}
208
209	// Write logs to file
210	fmt.Println("1. Writing logs to file...")
211	err := writeLogsStreaming("logs.json", logs)
212	if err != nil {
213		fmt.Println("Error:", err)
214		return
215	}
216	fmt.Println("✓ Logs written to logs.json")
217
218	// Process logs with streaming
219	fmt.Println("\n2. Processing logs with streaming...")
220	stats, err := processLogsStreaming("logs.json")
221	if err != nil {
222		fmt.Println("Error:", err)
223		return
224	}
225
226	statsJSON, _ := json.MarshalIndent(stats, "", "  ")
227	fmt.Println(string(statsJSON))
228
229	// Transform logs
230	fmt.Println("\n3. Filtering ERROR logs...")
231	err = transformLogs("logs.json", "errors.json", "ERROR")
232	if err != nil {
233		fmt.Println("Error:", err)
234		return
235	}
236	fmt.Println("✓ Errors written to errors.json")
237
238	// Read filtered logs
239	data, _ := os.ReadFile("errors.json")
240	fmt.Println(string(data))
241
242	// Create JSONL format
243	fmt.Println("\n4. Writing JSONL format...")
244	jsonlFile, _ := os.Create("logs.jsonl")
245	for _, entry := range logs {
246		json.NewEncoder(jsonlFile).Encode(entry)
247	}
248	jsonlFile.Close()
249	fmt.Println("✓ JSONL written to logs.jsonl")
250
251	// Process JSONL
252	fmt.Println("\n5. Processing JSONL format...")
253	jsonlStats, err := processJSONLines("logs.jsonl")
254	if err != nil {
255		fmt.Println("Error:", err)
256		return
257	}
258	fmt.Printf("JSONL Stats: %d total entries\n", jsonlStats.TotalEntries)
259
260	// Cleanup
261	os.Remove("logs.json")
262	os.Remove("errors.json")
263	os.Remove("logs.jsonl")
264	fmt.Println("\n✓ Cleanup complete")
265}

Exercise 10: XML and CSV Encoding

Work with XML and CSV formats for data interchange.

Solution

  1// run
  2package main
  3
  4import (
  5	"encoding/csv"
  6	"encoding/xml"
  7	"fmt"
  8	"os"
  9	"strconv"
 10	"time"
 11)
 12
 13// XML structures
 14type Library struct {
 15	XMLName xml.Name `xml:"library"`
 16	Name    string   `xml:"name,attr"`
 17	Books   []Book   `xml:"book"`
 18}
 19
 20type Book struct {
 21	XMLName     xml.Name `xml:"book"`
 22	ISBN        string   `xml:"isbn,attr"`
 23	Title       string   `xml:"title"`
 24	Author      string   `xml:"author"`
 25	PublishYear int      `xml:"publish_year"`
 26	Price       float64  `xml:"price"`
 27	Available   bool     `xml:"available"`
 28	Tags        []string `xml:"tags>tag"`
 29}
 30
 31// CSV-friendly structure
 32type Employee struct {
 33	ID         int
 34	Name       string
 35	Department string
 36	Salary     float64
 37	HireDate   time.Time
 38	Active     bool
 39}
 40
 41func main() {
 42	fmt.Println("=== XML and CSV Encoding ===\n")
 43
 44	// XML Marshaling
 45	fmt.Println("1. XML Marshaling:")
 46	library := Library{
 47		Name: "City Library",
 48		Books: []Book{
 49			{
 50				ISBN:        "978-0-123456-78-9",
 51				Title:       "The Go Programming Language",
 52				Author:      "Alan Donovan",
 53				PublishYear: 2015,
 54				Price:       44.99,
 55				Available:   true,
 56				Tags:        []string{"programming", "go", "computer-science"},
 57			},
 58			{
 59				ISBN:        "978-0-987654-32-1",
 60				Title:       "Concurrency in Go",
 61				Author:      "Katherine Cox-Buday",
 62				PublishYear: 2017,
 63				Price:       39.99,
 64				Available:   false,
 65				Tags:        []string{"programming", "concurrency", "go"},
 66			},
 67		},
 68	}
 69
 70	xmlData, err := xml.MarshalIndent(library, "", "  ")
 71	if err != nil {
 72		fmt.Println("Error:", err)
 73		return
 74	}
 75
 76	fmt.Println(xml.Header + string(xmlData))
 77
 78	// Write XML to file
 79	xmlFile, _ := os.Create("library.xml")
 80	xmlFile.WriteString(xml.Header)
 81	xmlFile.Write(xmlData)
 82	xmlFile.Close()
 83
 84	// XML Unmarshaling
 85	fmt.Println("\n2. XML Unmarshaling:")
 86	xmlData2, _ := os.ReadFile("library.xml")
 87
 88	var parsedLibrary Library
 89	if err := xml.Unmarshal(xmlData2, &parsedLibrary); err != nil {
 90		fmt.Println("Error:", err)
 91		return
 92	}
 93
 94	fmt.Printf("Library: %s\n", parsedLibrary.Name)
 95	fmt.Printf("Books: %d\n", len(parsedLibrary.Books))
 96	for i, book := range parsedLibrary.Books {
 97		fmt.Printf("%d. %s by %s ($%.2f)\n", i+1, book.Title, book.Author, book.Price)
 98	}
 99
100	// CSV Writing
101	fmt.Println("\n3. CSV Writing:")
102	employees := []Employee{
103		{1, "Alice Johnson", "Engineering", 120000, time.Date(2020, 1, 15, 0, 0, 0, 0, time.UTC), true},
104		{2, "Bob Smith", "Marketing", 85000, time.Date(2021, 3, 20, 0, 0, 0, 0, time.UTC), true},
105		{3, "Charlie Brown", "Sales", 95000, time.Date(2019, 7, 10, 0, 0, 0, 0, time.UTC), false},
106		{4, "Diana Prince", "Engineering", 130000, time.Date(2018, 11, 5, 0, 0, 0, 0, time.UTC), true},
107	}
108
109	csvFile, err := os.Create("employees.csv")
110	if err != nil {
111		fmt.Println("Error:", err)
112		return
113	}
114	defer csvFile.Close()
115
116	writer := csv.NewWriter(csvFile)
117	defer writer.Flush()
118
119	// Write header
120	writer.Write([]string{"ID", "Name", "Department", "Salary", "HireDate", "Active"})
121
122	// Write data
123	for _, emp := range employees {
124		record := []string{
125			strconv.Itoa(emp.ID),
126			emp.Name,
127			emp.Department,
128			fmt.Sprintf("%.2f", emp.Salary),
129			emp.HireDate.Format("2006-01-02"),
130			strconv.FormatBool(emp.Active),
131		}
132		writer.Write(record)
133	}
134
135	fmt.Println("✓ CSV written to employees.csv")
136
137	// CSV Reading
138	fmt.Println("\n4. CSV Reading:")
139	csvFile2, err := os.Open("employees.csv")
140	if err != nil {
141		fmt.Println("Error:", err)
142		return
143	}
144	defer csvFile2.Close()
145
146	reader := csv.NewReader(csvFile2)
147
148	// Read header
149	header, _ := reader.Read()
150	fmt.Printf("Headers: %v\n", header)
151
152	// Read records
153	records, err := reader.ReadAll()
154	if err != nil {
155		fmt.Println("Error:", err)
156		return
157	}
158
159	fmt.Printf("\nEmployees (%d):\n", len(records))
160	for _, record := range records {
161		fmt.Printf("ID: %s, Name: %-20s Dept: %-15s Salary: $%s\n",
162			record[0], record[1], record[2], record[3])
163	}
164
165	// Advanced CSV - custom separator
166	fmt.Println("\n5. Custom CSV (TSV):")
167	tsvFile, _ := os.Create("employees.tsv")
168	defer tsvFile.Close()
169
170	tsvWriter := csv.NewWriter(tsvFile)
171	tsvWriter.Comma = '\t' // Use tab as separator
172	defer tsvWriter.Flush()
173
174	tsvWriter.Write([]string{"ID", "Name", "Department"})
175	for _, emp := range employees {
176		tsvWriter.Write([]string{
177			strconv.Itoa(emp.ID),
178			emp.Name,
179			emp.Department,
180		})
181	}
182
183	fmt.Println("✓ TSV written to employees.tsv")
184
185	// Read and display TSV
186	tsvData, _ := os.ReadFile("employees.tsv")
187	fmt.Println("\nTSV Content:")
188	fmt.Println(string(tsvData))
189
190	// Cleanup
191	os.Remove("library.xml")
192	os.Remove("employees.csv")
193	os.Remove("employees.tsv")
194	fmt.Println("✓ Cleanup complete")
195}

Exercise 11: Base64 and Data Encoding

Handle binary data encoding for APIs and file storage.

Solution

  1// run
  2package main
  3
  4import (
  5	"bytes"
  6	"encoding/base64"
  7	"encoding/json"
  8	"fmt"
  9	"image"
 10	"image/color"
 11	"image/png"
 12	"os"
 13)
 14
 15type FileUpload struct {
 16	Filename    string `json:"filename"`
 17	ContentType string `json:"content_type"`
 18	Size        int    `json:"size"`
 19	Data        string `json:"data"` // Base64 encoded
 20}
 21
 22type ImageData struct {
 23	Width  int    `json:"width"`
 24	Height int    `json:"height"`
 25	Format string `json:"format"`
 26	Data   []byte `json:"-"` // Not serialized directly
 27}
 28
 29// Custom marshaling for ImageData
 30func (img *ImageData) MarshalJSON() ([]byte, error) {
 31	type Alias ImageData
 32	return json.Marshal(&struct {
 33		*Alias
 34		Data string `json:"data"`
 35	}{
 36		Alias: (*Alias)(img),
 37		Data:  base64.StdEncoding.EncodeToString(img.Data),
 38	})
 39}
 40
 41func (img *ImageData) UnmarshalJSON(data []byte) error {
 42	type Alias ImageData
 43	aux := &struct {
 44		*Alias
 45		Data string `json:"data"`
 46	}{
 47		Alias: (*Alias)(img),
 48	}
 49
 50	if err := json.Unmarshal(data, &aux); err != nil {
 51		return err
 52	}
 53
 54	decoded, err := base64.StdEncoding.DecodeString(aux.Data)
 55	if err != nil {
 56		return err
 57	}
 58
 59	img.Data = decoded
 60	return nil
 61}
 62
 63// Create sample image
 64func createSampleImage(width, height int) ([]byte, error) {
 65	img := image.NewRGBA(image.Rect(0, 0, width, height))
 66
 67	// Draw gradient
 68	for y := 0; y < height; y++ {
 69		for x := 0; x < width; x++ {
 70			r := uint8(x * 255 / width)
 71			g := uint8(y * 255 / height)
 72			b := uint8((x + y) * 255 / (width + height))
 73			img.Set(x, y, color.RGBA{r, g, b, 255})
 74		}
 75	}
 76
 77	var buf bytes.Buffer
 78	if err := png.Encode(&buf, img); err != nil {
 79		return nil, err
 80	}
 81
 82	return buf.Bytes(), nil
 83}
 84
 85func main() {
 86	fmt.Println("=== Base64 and Data Encoding ===\n")
 87
 88	// 1. Basic Base64 encoding
 89	fmt.Println("1. Basic Base64 Encoding:")
 90	message := "Hello, World! This is a secret message."
 91	encoded := base64.StdEncoding.EncodeToString([]byte(message))
 92	fmt.Printf("Original: %s\n", message)
 93	fmt.Printf("Encoded:  %s\n", encoded)
 94
 95	decoded, _ := base64.StdEncoding.DecodeString(encoded)
 96	fmt.Printf("Decoded:  %s\n", string(decoded))
 97
 98	// 2. URL-safe Base64
 99	fmt.Println("\n2. URL-safe Base64:")
100	urlData := "user@example.com?token=abc123&id=456"
101	urlEncoded := base64.URLEncoding.EncodeToString([]byte(urlData))
102	fmt.Printf("Original:    %s\n", urlData)
103	fmt.Printf("URL-encoded: %s\n", urlEncoded)
104
105	urlDecoded, _ := base64.URLEncoding.DecodeString(urlEncoded)
106	fmt.Printf("Decoded:     %s\n", string(urlDecoded))
107
108	// 3. File upload simulation
109	fmt.Println("\n3. File Upload with Base64:")
110	fileContent := []byte("This is the content of my file.\nIt has multiple lines.\nAnd some data: " + string([]byte{0x01, 0x02, 0x03, 0xFF}))
111
112	upload := FileUpload{
113		Filename:    "document.txt",
114		ContentType: "text/plain",
115		Size:        len(fileContent),
116		Data:        base64.StdEncoding.EncodeToString(fileContent),
117	}
118
119	uploadJSON, _ := json.MarshalIndent(upload, "", "  ")
120	fmt.Println("Upload JSON:")
121	fmt.Println(string(uploadJSON))
122
123	// Parse and extract
124	var parsedUpload FileUpload
125	json.Unmarshal(uploadJSON, &parsedUpload)
126
127	extractedData, _ := base64.StdEncoding.DecodeString(parsedUpload.Data)
128	fmt.Printf("\nExtracted file: %s (%d bytes)\n%s\n",
129		parsedUpload.Filename, parsedUpload.Size, string(extractedData))
130
131	// 4. Image encoding
132	fmt.Println("\n4. Image Encoding:")
133	imageBytes, err := createSampleImage(50, 30)
134	if err != nil {
135		fmt.Println("Error:", err)
136		return
137	}
138
139	imgData := ImageData{
140		Width:  50,
141		Height: 30,
142		Format: "PNG",
143		Data:   imageBytes,
144	}
145
146	imgJSON, _ := json.MarshalIndent(imgData, "", "  ")
147	fmt.Printf("Image JSON (truncated):\n%s...\n", string(imgJSON[:min(200, len(imgJSON))]))
148	fmt.Printf("Full JSON size: %d bytes\n", len(imgJSON))
149
150	// Parse image
151	var parsedImg ImageData
152	json.Unmarshal(imgJSON, &parsedImg)
153	fmt.Printf("\nParsed image: %dx%d %s (%d bytes data)\n",
154		parsedImg.Width, parsedImg.Height, parsedImg.Format, len(parsedImg.Data))
155
156	// 5. Save and load from file
157	fmt.Println("\n5. Saving Base64 to file:")
158	encodedImg := base64.StdEncoding.EncodeToString(imageBytes)
159	os.WriteFile("encoded.txt", []byte(encodedImg), 0644)
160	fmt.Println("✓ Binary data saved as Base64")
161
162	// Calculate size difference
163	fmt.Printf("Original size: %d bytes\n", len(imageBytes))
164	fmt.Printf("Base64 size:   %d bytes\n", len(base64.StdEncoding.EncodeToString(imageBytes)))
165	overhead := float64(len(base64.StdEncoding.EncodeToString(imageBytes)))/float64(len(imageBytes))*100 - 100
166	fmt.Printf("Overhead:      %.1f%%\n", overhead)
167
168	// 6. Raw vs StdEncoding vs URLEncoding
169	fmt.Println("\n6. Encoding variants:")
170	testData := []byte("Hello+World=123")
171
172	stdEnc := base64.StdEncoding.EncodeToString(testData)
173	urlEnc := base64.URLEncoding.EncodeToString(testData)
174	rawEnc := base64.RawStdEncoding.EncodeToString(testData)
175
176	fmt.Printf("Standard:        %s\n", stdEnc)
177	fmt.Printf("URL-safe:        %s\n", urlEnc)
178	fmt.Printf("Raw (no padding): %s\n", rawEnc)
179
180	// Cleanup
181	os.Remove("encoded.txt")
182	fmt.Println("\n✓ Cleanup complete")
183}
184
185func min(a, b int) int {
186	if a < b {
187		return a
188	}
189	return b
190}

Summary

• json.Marshal converts Go → JSON
• json.Unmarshal converts JSON → Go
• Struct tags control JSON field names and behavior
• omitempty reduces JSON size by omitting zero values
• Custom marshalers provide full control over encoding
• json.RawMessage enables delayed parsing for dynamic data
• Streaming with Encoder/Decoder handles large datasets efficiently
• Multiple formats available: XML, CSV, Base64, Protobuf, MessagePack
• Always validate data after unmarshaling
• Use pointers for optional fields to distinguish null from zero
• Encoder/Decoder for streaming, Marshal/Unmarshal for in-memory

💡 Final Takeaway: JSON encoding is about more than just converting data formats - it's about building reliable communication channels between your Go applications and the rest of the world. Master these patterns, and you'll create APIs that are robust, efficient, and a joy to work with.

Master JSON encoding and you'll build production-ready APIs and handle data serialization like a pro!