Unified API Platform

Project: Unified API Platform

Problem Statement

You're the platform engineering lead at a rapidly growing SaaS company serving mobile apps, web clients, and third-party developers. Your current API architecture is becoming unmanageable:

  • Fragmented APIs: Multiple separate services with inconsistent patterns
  • Over-fetching problems: Mobile clients waste bandwidth on unnecessary data
  • N+1 query hell: Web apps make dozens of requests for a single page
  • No real-time features: Can't push updates to clients efficiently
  • Authentication chaos: Different auth patterns across services
  • Performance issues: Slow response times causing poor user experience
  • Developer experience: Poor documentation and testing tools

The Challenge: Build a unified API platform that serves diverse client needs with optimal performance, real-time capabilities, and excellent developer experience.

Real-World Scenario:

 1# Mobile app - needs minimal data
 2GET /api/v1/users/123?fields=name,avatar
 3# Response: 200 bytes
 4
 5# Web app - needs rich nested data
 6GET /api/v1/users/123?include=posts(comments),followers
 7# Response: 15KB
 8
 9# GraphQL - exact data requirements
10query UserProfile($id: ID!, $withPosts: Boolean!) {
11  user(id: $id) {
12    name
13    email
14    avatar
15    posts @include(if: $withPosts) {
16      title
17      comments(limit: 5) {
18        author { name }
19      }
20    }
21  }
22}
23# Response: Only requested fields, single request
24
25# Real-time subscription
26subscription OnNewPost($userId: ID!) {
27  postAdded(userId: $userId) {
28    id
29    title
30    author { name }
31    createdAt
32  }
33}
34# Live updates pushed to client
35
36# Third-party API - rate limited, scoped access
37GET /api/v1/public/users/123
38# Headers: X-RateLimit-Remaining: 99, X-RateLimit-Limit: 100

Requirements

Functional Requirements

Must Have:

  • ✅ Dual API Support: Both REST and GraphQL endpoints in single service
  • ✅ Advanced Authentication: JWT with RS256, OAuth 2.0, API keys, RBAC
  • ✅ Real-time Subscriptions: WebSocket-based subscriptions for live updates
  • ✅ DataLoader Pattern: Eliminate N+1 queries with intelligent batching
  • ✅ Comprehensive Validation: Request validation with detailed error responses
  • ✅ Rate Limiting: User-based and IP-based rate limiting with different tiers
  • ✅ API Documentation: Auto-generated OpenAPI and GraphQL schema documentation
  • ✅ Query Optimization: Query complexity analysis and depth limiting

Should Have:

  • ✅ Response Caching: Multi-layer caching with smart invalidation
  • ✅ Request Tracing: Distributed tracing with correlation IDs
  • ✅ Metrics & Monitoring: Comprehensive API metrics and performance monitoring
  • ✅ API Versioning: Semantic versioning with backward compatibility
  • ✅ Webhook System: Event-driven webhooks for third-party integrations
  • ✅ Bulk Operations: Efficient bulk create, update, delete operations
  • ✅ File Upload: Multipart file uploads with progress tracking
  • ✅ Internationalization: Multi-language support for responses

Nice to Have:

  • GraphQL Federation: Schema federation across multiple services
  • API Analytics: Advanced analytics on API usage and performance
  • A/B Testing: Feature flags and A/B testing framework
  • API Gateway: Request routing, composition, and transformation
  • Custom Directives: Custom GraphQL directives for auth, caching, etc.
  • Schema Stitching: Combine multiple GraphQL schemas
  • API Mocking: Automatic API mocking for development

Non-Functional Requirements

  • Performance: REST API <50ms p95, GraphQL <100ms p95 for simple queries
  • Scalability: Handle 10,000+ concurrent connections, 100K+ requests/minute
  • Availability: 99.9% uptime with graceful degradation
  • Security: OWASP compliance, input validation, rate limiting, encryption
  • Observability: Comprehensive logging, metrics, tracing for all requests
  • Developer Experience: Auto-generated SDKs, playground, interactive docs

Constraints

  • Framework: Gin for REST, gqlgen for GraphQL
  • Database: PostgreSQL primary, Redis for cache, Elasticsearch for search
  • Authentication: JWT RS256 with key rotation, OAuth 2.0 provider integration
  • Subscriptions: WebSocket with fallback to Server-Sent Events
  • Deployment: Docker containers, Kubernetes orchestration
  • Documentation: OpenAPI 3.0, GraphQL SDL with extensions

Design Considerations

This unified API platform addresses diverse client needs through a single, coherent service. The architecture balances flexibility with performance:

API Design Philosophy:

  • REST for simplicity: Traditional CRUD operations, straightforward resource access
  • GraphQL for flexibility: Complex data requirements, nested relationships, exact field selection
  • Unified service layer: Single business logic layer serving both API types
  • Client-driven optimization: Clients choose the API style that best fits their needs

Key Technical Decisions:

  1. Unified Service Layer: Share business logic between REST and GraphQL to maintain consistency and reduce duplication

  2. DataLoader Pattern: Batch and cache database queries to eliminate N+1 problems and optimize data fetching across both APIs

  3. Multi-tier Caching: Layer Redis cache, in-memory cache, and CDN to minimize database load and improve response times

  4. Authentication Flexibility: Support JWT, OAuth 2.0, and API keys through a unified authentication middleware

  5. Real-time Communication: WebSocket-based subscriptions with GraphQL for live updates and notifications

  6. Performance Optimization: Query complexity analysis, depth limiting, and intelligent caching to maintain performance at scale

  7. Developer Experience: Auto-generated documentation, interactive playgrounds, and comprehensive error messages

Acceptance Criteria

Your unified API platform is considered complete when it meets these criteria:

API Functionality:

  • ✅ REST endpoints handle all CRUD operations with proper HTTP semantics
  • ✅ GraphQL server resolves queries, mutations, and subscriptions correctly
  • ✅ Both APIs return consistent data from the same service layer
  • ✅ Field selection works in REST and GraphQL
  • ✅ Nested relationships load efficiently without N+1 queries

Authentication & Authorization:

  • ✅ JWT authentication works with proper token validation and refresh
  • ✅ API key authentication supports scoped permissions
  • ✅ Role-based access control restricts operations by user role
  • ✅ Authentication works consistently across REST and GraphQL
  • ✅ GraphQL directives enforce authorization rules

Performance:

  • ✅ REST API responds in <50ms for simple queries
  • ✅ GraphQL responds in <100ms for queries with 2-3 levels
  • ✅ DataLoader batches database queries and eliminates N+1 problems
  • ✅ Redis caching reduces database load by >70%
  • ✅ System handles 10,000+ concurrent connections

Real-time Features:

  • ✅ WebSocket connections establish and maintain properly
  • ✅ GraphQL subscriptions push updates to clients in real-time
  • ✅ Connection management handles disconnects and reconnects gracefully
  • ✅ Subscription filters work correctly

Developer Experience:

  • ✅ OpenAPI documentation auto-generates from code
  • ✅ GraphQL Playground provides interactive schema exploration
  • ✅ Error messages are clear, actionable, and properly formatted
  • ✅ Rate limit headers inform clients of remaining quota
  • ✅ API versioning allows backward-compatible changes

Production Readiness:

  • ✅ Comprehensive logging captures all requests and errors
  • ✅ Metrics track API performance and usage patterns
  • ✅ Distributed tracing correlates requests across services
  • ✅ Rate limiting prevents abuse and ensures fair usage
  • ✅ Docker deployment works with all dependencies
  • ✅ Health checks verify service availability
  • ✅ Graceful shutdown handles in-flight requests

Usage Examples

REST API Examples

Basic Resource Access:

 1# Get user by ID
 2curl -X GET http://localhost:8080/api/v1/users/123 \
 3  -H "Authorization: Bearer YOUR_JWT_TOKEN"
 4
 5# Response
 6{
 7  "id": 123,
 8  "name": "Jane Doe",
 9  "email": "jane@example.com",
10  "avatar": "https://cdn.example.com/avatars/jane.jpg",
11  "created_at": "2024-01-15T10:30:00Z"
12}

Sparse Fieldsets:

1# Get only specific fields
2curl -X GET "http://localhost:8080/api/v1/users/123?fields=name,avatar"
3
4# Response
5{
6  "name": "Jane Doe",
7  "avatar": "https://cdn.example.com/avatars/jane.jpg"
8}

Including Relationships:

 1# Get user with posts and comments
 2curl -X GET "http://localhost:8080/api/v1/users/123?include=posts(comments)"
 3
 4# Response
 5{
 6  "id": 123,
 7  "name": "Jane Doe",
 8  "posts": [
 9    {
10      "id": 456,
11      "title": "My First Post",
12      "comments": [
13        {"id": 789, "text": "Great post!", "author": {"name": "John"}}
14      ]
15    }
16  ]
17}

Creating Resources:

1# Create a new post
2curl -X POST http://localhost:8080/api/v1/posts \
3  -H "Authorization: Bearer YOUR_JWT_TOKEN" \
4  -H "Content-Type: application/json" \
5  -d '{
6    "title": "New Post Title",
7    "content": "Post content here",
8    "tags": ["go", "api"]
9  }'

GraphQL Examples

Simple Query:

1query GetUser($id: ID!) {
2  user(id: $id) {
3    id
4    name
5    email
6    avatar
7    createdAt
8  }
9}

Nested Query with Variables:

 1query UserWithPosts($userId: ID!, $postLimit: Int = 10) {
 2  user(id: $userId) {
 3    name
 4    email
 5    posts(limit: $postLimit) {
 6      id
 7      title
 8      createdAt
 9      comments(limit: 5) {
10        text
11        author {
12          name
13          avatar
14        }
15      }
16    }
17    followerCount
18    followingCount
19  }
20}

Mutation:

 1mutation CreatePost($input: CreatePostInput!) {
 2  createPost(input: $input) {
 3    id
 4    title
 5    content
 6    author {
 7      name
 8      avatar
 9    }
10    createdAt
11  }
12}
13
14# Variables
15{
16  "input": {
17    "title": "New GraphQL Post",
18    "content": "Content created via GraphQL",
19    "tags": ["graphql", "api"]
20  }
21}

Subscription:

 1subscription OnNewPost($userId: ID!) {
 2  postAdded(userId: $userId) {
 3    id
 4    title
 5    author {
 6      name
 7      avatar
 8    }
 9    createdAt
10  }
11}

Using Directives:

 1query UserProfile($id: ID!, $includePosts: Boolean!) {
 2  user(id: $id) @cache(ttl: 300) {
 3    name
 4    email
 5    posts @include(if: $includePosts) @hasRole(roles: ["user", "admin"]) {
 6      title
 7      createdAt
 8    }
 9  }
10}

Authentication Examples

JWT Authentication:

 1# Login to get JWT token
 2curl -X POST http://localhost:8080/api/v1/auth/login \
 3  -H "Content-Type: application/json" \
 4  -d '{
 5    "email": "user@example.com",
 6    "password": "secure_password"
 7  }'
 8
 9# Response
10{
11  "access_token": "eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9...",
12  "refresh_token": "eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9...",
13  "expires_in": 3600
14}
15
16# Use token in subsequent requests
17curl -X GET http://localhost:8080/api/v1/users/me \
18  -H "Authorization: Bearer eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9..."

API Key Authentication:

 1# Create API key
 2curl -X POST http://localhost:8080/api/v1/apikeys \
 3  -H "Authorization: Bearer YOUR_JWT_TOKEN" \
 4  -d '{
 5    "name": "Mobile App Key",
 6    "permissions": ["read:users", "write:posts"],
 7    "rate_limit": 1000
 8  }'
 9
10# Use API key
11curl -X GET http://localhost:8080/api/v1/users/123 \
12  -H "Authorization: ApiKey YOUR_API_KEY"

WebSocket Subscription Example

 1// JavaScript client
 2const ws = new WebSocket('ws://localhost:8080/graphql');
 3
 4ws.onopen = => {
 5  // Subscribe to new posts
 6  ws.send(JSON.stringify({
 7    type: 'connection_init',
 8    payload: {
 9      Authorization: 'Bearer YOUR_JWT_TOKEN'
10    }
11  }));
12
13  ws.send(JSON.stringify({
14    id: '1',
15    type: 'start',
16    payload: {
17      query: `
18        subscription OnNewPost($userId: ID!) {
19          postAdded(userId: $userId) {
20            id
21            title
22            author { name }
23            createdAt
24          }
25        }
26      `,
27      variables: { userId: '123' }
28    }
29  }));
30};
31
32ws.onmessage = => {
33  const message = JSON.parse(event.data);
34  if {
35    console.log('New post:', message.payload.data.postAdded);
36  }
37};

Rate Limiting Example

 1# Make request and check rate limit headers
 2curl -i -X GET http://localhost:8080/api/v1/users/123 \
 3  -H "Authorization: ApiKey YOUR_API_KEY"
 4
 5# Response headers
 6HTTP/1.1 200 OK
 7X-RateLimit-Limit: 1000
 8X-RateLimit-Remaining: 999
 9X-RateLimit-Reset: 1640000000
10
11# When rate limit exceeded
12HTTP/1.1 429 Too Many Requests
13X-RateLimit-Limit: 1000
14X-RateLimit-Remaining: 0
15X-RateLimit-Reset: 1640000000
16Retry-After: 3600
17
18{
19  "error": "rate limit exceeded",
20  "retry_after": 3600
21}

Key Takeaways

API Design Patterns

  • REST vs GraphQL: Understanding when to use each approach and their trade-offs
  • Unified Architecture: Building a single service that serves multiple API paradigms
  • API Versioning: Strategies for backward compatibility and smooth migrations
  • Documentation: Auto-generating comprehensive API documentation

Advanced Authentication & Authorization

  • Multi-tenant Auth: Supporting JWT, OAuth 2.0, API keys simultaneously
  • RBAC Implementation: Fine-grained role-based access control
  • Security Best Practices: OWASP compliance, input validation, rate limiting
  • Token Management: Secure token handling, rotation, and revocation

Performance Optimization

  • DataLoader Pattern: Eliminating N+1 queries with intelligent batching
  • Multi-layer Caching: Redis, in-memory, and CDN caching strategies
  • Query Optimization: GraphQL complexity analysis and depth limiting
  • Connection Pooling: Efficient database connection management

Real-time Features

  • WebSocket Management: Scalable WebSocket connection handling
  • GraphQL Subscriptions: Real-time data updates with GraphQL
  • Event-driven Architecture: Building event-driven systems with webhooks
  • Push Notifications: Efficient push notification systems

Production Readiness

  • Observability: Comprehensive logging, metrics, and tracing
  • Error Handling: Structured error handling and user-friendly responses
  • Rate Limiting: Advanced rate limiting with different strategies
  • Scalability: Building horizontally scalable API services

Next Steps

After completing this unified API platform, continue your journey with these advanced topics:

  1. API Federation: Extend to GraphQL Federation for distributed schemas across microservices

  2. Advanced Monitoring: Implement APM with tools like New Relic or Datadog

  3. API Analytics: Build comprehensive analytics for API usage patterns, performance trends, and user behavior

  4. Multi-region Deployment: Deploy across multiple regions with global load balancing and data replication

  5. Advanced Security: Implement OAuth 2.0 provider, implement API threat protection, add anomaly detection

  6. API Gateway: Build a full API gateway with request transformation, composition, and orchestration

  7. Service Mesh: Integrate with Istio or Linkerd for advanced traffic management and observability

  8. GraphQL Advanced: Explore schema stitching, Apollo Federation, and persisted queries

  9. Testing Strategies: Implement contract testing, chaos engineering, and performance testing at scale

  10. Developer Portal: Build a comprehensive developer portal with SDKs, code examples, and interactive tutorials

Explore Related Projects:

  • Distributed Job Queue: Learn distributed system patterns
  • Cloud-Native E-Commerce Platform: Apply these patterns at scale
  • Real-Time Analytics Engine: Build high-performance data processing systems

Download Complete Solution

📦 Download Complete Solution

Get the complete working implementation with all source code, configurations, and documentation:

⬇️ Download Solution

Contains: Complete project with all Go source files, GraphQL schemas, Docker configurations, Kubernetes deployments, comprehensive README with implementation guide, setup instructions, and testing documentation - ready to run and deploy.

Congratulations! You've built a production-ready unified API platform with both REST and GraphQL services, advanced authentication, real-time subscriptions, and excellent performance characteristics. This platform demonstrates advanced Go development patterns, modern API design principles, and production-ready features that you can apply to any API development scenario.