Complete Protocols Master Part 7: API Protocols

REST APIs

REST (Representational State Transfer) is an architectural style, not a protocol. It uses HTTP verbs and URLs to model resources and operations. REST is the most common API style on the web.

REST Architectural Constraints (Roy Fielding, 2000):

1. CLIENT-SERVER
   Separation of concerns: UI and data storage
   
2. STATELESS
   Each request contains all info needed
   No server-side session state
   
3. CACHEABLE
   Responses must define cacheability
   Improves scalability and performance
   
4. LAYERED SYSTEM
   Client doesn't know if talking to server or intermediary
   Enables load balancers, proxies, CDNs
   
5. UNIFORM INTERFACE
   - Resource identification (URLs)
   - Resource manipulation through representations
   - Self-descriptive messages (Content-Type)
   - HATEOAS (Hypermedia as the Engine of Application State)
   
6. CODE ON DEMAND (Optional)
   Server can send executable code (JavaScript)

Note: Most "REST" APIs are actually "RESTful" or "REST-like"
      True REST with HATEOAS is rare in practice

RESTful URL Design

REST URL Design Best Practices:

# Resources are NOUNS (not verbs!)
✅ /users
❌ /getUsers
❌ /createUser

# Collection vs. Item
GET    /users          → List all users
POST   /users          → Create user
GET    /users/123      → Get user 123
PUT    /users/123      → Replace user 123
PATCH  /users/123      → Update user 123
DELETE /users/123      → Delete user 123

# Nested Resources
GET    /users/123/posts        → Posts by user 123
POST   /users/123/posts        → Create post for user 123
GET    /users/123/posts/456    → Get specific post

# Filtering, Sorting, Pagination
GET /users?status=active       → Filter by status
GET /users?sort=created_at     → Sort by field
GET /users?sort=-name          → Descending sort
GET /users?page=2&limit=20     → Pagination
GET /users?fields=id,name      → Sparse fieldsets

# Search
GET /users/search?q=john       → Search users

# Actions (when CRUD doesn't fit)
POST /users/123/activate       → Activate user
POST /orders/456/cancel        → Cancel order

REST API Examples

# RESTful API client example
import json
from urllib.request import Request, urlopen
from urllib.parse import urlencode

class RESTClient:
    """Simple REST API client demonstration"""
    
    def __init__(self, base_url):
        self.base_url = base_url.rstrip('/')
    
    def _request(self, method, endpoint, data=None, params=None):
        """Make HTTP request"""
        url = f"{self.base_url}{endpoint}"
        
        if params:
            url += '?' + urlencode(params)
        
        headers = {
            'Content-Type': 'application/json',
            'Accept': 'application/json'
        }
        
        body = json.dumps(data).encode() if data else None
        
        # Return simulated response for demonstration
        return {
            'method': method,
            'url': url,
            'headers': headers,
            'body': data
        }
    
    def get(self, endpoint, params=None):
        return self._request('GET', endpoint, params=params)
    
    def post(self, endpoint, data):
        return self._request('POST', endpoint, data=data)
    
    def put(self, endpoint, data):
        return self._request('PUT', endpoint, data=data)
    
    def patch(self, endpoint, data):
        return self._request('PATCH', endpoint, data=data)
    
    def delete(self, endpoint):
        return self._request('DELETE', endpoint)

# Demonstrate REST patterns
api = RESTClient('https://api.example.com/v1')

print("REST API Patterns Demo")
print("=" * 60)

# List users (GET /users)
result = api.get('/users', params={'status': 'active', 'limit': 10})
print(f"\n1. List users: {result['method']} {result['url']}")

# Create user (POST /users)
new_user = {'name': 'Alice', 'email': 'alice@example.com'}
result = api.post('/users', data=new_user)
print(f"\n2. Create user: {result['method']} {result['url']}")
print(f"   Body: {result['body']}")

# Get user (GET /users/123)
result = api.get('/users/123')
print(f"\n3. Get user: {result['method']} {result['url']}")

# Update user (PATCH /users/123)
updates = {'name': 'Alice Smith'}
result = api.patch('/users/123', data=updates)
print(f"\n4. Update user: {result['method']} {result['url']}")
print(f"   Body: {result['body']}")

# Delete user (DELETE /users/123)
result = api.delete('/users/123')
print(f"\n5. Delete user: {result['method']} {result['url']}")

# Nested resource
result = api.get('/users/123/posts')
print(f"\n6. User's posts: {result['method']} {result['url']}")

GraphQL

GraphQL (Facebook, 2015) is a query language for APIs. Clients specify exactly what data they need, solving REST's over-fetching and under-fetching problems.

REST Over-fetching and Under-fetching:

Scenario: Mobile app needs user name and their posts' titles

REST Approach (multiple requests, extra data):
GET /users/123
→ {id, name, email, avatar, bio, created_at, ...}  # Too much data!

GET /users/123/posts
→ [{id, title, body, author_id, created_at, ...}, ...] # Too much again!

GET /users/123/posts/1/comments
GET /users/123/posts/2/comments  # N+1 problem!


GraphQL Approach (single request, exact data):
POST /graphql
query {
  user(id: 123) {
    name
    posts {
      title
    }
  }
}

→ {
    "user": {
      "name": "Alice",
      "posts": [
        {"title": "First Post"},
        {"title": "Second Post"}
      ]
    }
  }

Benefits:
• Single round trip
• Only requested fields returned
• Strongly typed schema
• Introspection (self-documenting)

GraphQL Schema

# GraphQL Schema Definition Language (SDL)

# Scalar types: Int, Float, String, Boolean, ID

# Object types
type User {
  id: ID!           # ! = non-nullable
  name: String!
  email: String!
  posts: [Post!]!   # List of non-null Posts
  friends: [User!]
}

type Post {
  id: ID!
  title: String!
  body: String
  author: User!
  comments: [Comment!]!
  createdAt: String!
}

type Comment {
  id: ID!
  text: String!
  author: User!
}

# Input types (for mutations)
input CreateUserInput {
  name: String!
  email: String!
}

input UpdateUserInput {
  name: String
  email: String
}

# Query type (read operations)
type Query {
  user(id: ID!): User
  users(limit: Int, offset: Int): [User!]!
  post(id: ID!): Post
  searchPosts(query: String!): [Post!]!
}

# Mutation type (write operations)
type Mutation {
  createUser(input: CreateUserInput!): User!
  updateUser(id: ID!, input: UpdateUserInput!): User
  deleteUser(id: ID!): Boolean!
  createPost(title: String!, body: String): Post!
}

# Subscription type (real-time)
type Subscription {
  postCreated: Post!
  userOnline(userId: ID!): User!
}

GraphQL Operations

# GraphQL Query Examples

# Simple query
query {
  user(id: "123") {
    name
    email
  }
}

# Query with variables
query GetUser($userId: ID!) {
  user(id: $userId) {
    name
    email
    posts {
      title
    }
  }
}
# Variables: {"userId": "123"}

# Aliases (request same field multiple times)
query {
  alice: user(id: "1") { name }
  bob: user(id: "2") { name }
}

# Fragments (reusable field sets)
fragment UserFields on User {
  id
  name
  email
}

query {
  user(id: "123") {
    ...UserFields
    posts { title }
  }
}

# Mutation
mutation CreateUser($input: CreateUserInput!) {
  createUser(input: $input) {
    id
    name
    email
  }
}
# Variables: {"input": {"name": "Alice", "email": "alice@example.com"}}

# Subscription (WebSocket)
subscription {
  postCreated {
    id
    title
    author { name }
  }
}

# GraphQL client example
import json

def graphql_request(query, variables=None):
    """Demonstrate GraphQL request structure"""
    request = {
        'query': query,
        'variables': variables or {}
    }
    
    # In real usage, POST to /graphql endpoint
    return request

# Example queries
print("GraphQL Request Examples")
print("=" * 60)

# Simple query
query1 = """
query {
  user(id: "123") {
    name
    email
    posts {
      title
    }
  }
}
"""
print("\n1. Simple query:")
print(json.dumps(graphql_request(query1), indent=2))

# Query with variables
query2 = """
query GetUserPosts($userId: ID!, $limit: Int) {
  user(id: $userId) {
    name
    posts(limit: $limit) {
      title
      createdAt
    }
  }
}
"""
variables = {"userId": "123", "limit": 5}
print("\n2. Query with variables:")
print(json.dumps(graphql_request(query2, variables), indent=2))

# Mutation
mutation = """
mutation CreatePost($title: String!, $body: String) {
  createPost(title: $title, body: $body) {
    id
    title
  }
}
"""
variables = {"title": "My New Post", "body": "Post content here"}
print("\n3. Mutation:")
print(json.dumps(graphql_request(mutation, variables), indent=2))

gRPC

gRPC (Google Remote Procedure Call) is a high-performance RPC framework using HTTP/2 and Protocol Buffers. It's ideal for microservices communication where performance matters.

gRPC Performance Advantages:

1. PROTOCOL BUFFERS (Binary serialization)
   JSON: {"name":"Alice","age":30}  → 26 bytes + parsing
   Protobuf: [binary data]         → ~10 bytes, faster parsing
   
   Typical savings: 3-10x smaller, 5-100x faster parsing

2. HTTP/2 TRANSPORT
   • Multiplexing (parallel requests)
   • Header compression
   • Single TCP connection

3. STREAMING (4 modes)
   • Unary: Request → Response
   • Server streaming: Request → Stream of responses
   • Client streaming: Stream of requests → Response
   • Bidirectional: Stream ←→ Stream

4. CODE GENERATION
   • Define service in .proto file
   • Generate client/server code in any language
   • Type-safe, no manual serialization

Languages Supported:
C++, Java, Python, Go, Ruby, C#, Node.js, Dart, Kotlin, Swift...

// user_service.proto

syntax = "proto3";

package userservice;

// Message definitions
message User {
  int32 id = 1;           // Field number (for binary encoding)
  string name = 2;
  string email = 3;
  repeated string roles = 4;  // List
  Address address = 5;        // Nested message
}

message Address {
  string street = 1;
  string city = 2;
  string country = 3;
}

message GetUserRequest {
  int32 user_id = 1;
}

message GetUserResponse {
  User user = 1;
}

message ListUsersRequest {
  int32 page = 1;
  int32 page_size = 2;
}

message ListUsersResponse {
  repeated User users = 1;
  int32 total_count = 2;
}

message CreateUserRequest {
  string name = 1;
  string email = 2;
}

// Service definition
service UserService {
  // Unary RPC
  rpc GetUser(GetUserRequest) returns (GetUserResponse);
  
  // Server streaming
  rpc ListUsers(ListUsersRequest) returns (stream User);
  
  // Client streaming
  rpc UploadUsers(stream User) returns (UploadUsersResponse);
  
  // Bidirectional streaming
  rpc Chat(stream ChatMessage) returns (stream ChatMessage);
}

message UploadUsersResponse {
  int32 uploaded_count = 1;
}

message ChatMessage {
  string user = 1;
  string message = 2;
  int64 timestamp = 3;
}

gRPC Streaming Modes

# gRPC streaming concepts demonstration
# Note: Actual gRPC requires generated code from proto file

print("gRPC Streaming Modes")
print("=" * 60)

# 1. Unary RPC (like regular HTTP)
print("""
1. UNARY RPC
   Client ─── Request ───> Server
   Client <── Response ─── Server
   
   Example: GetUser(user_id) → User
""")

# 2. Server Streaming
print("""
2. SERVER STREAMING
   Client ─── Request ─────────────> Server
   Client <── Response 1 ─────────── Server
   Client <── Response 2 ─────────── Server
   Client <── Response 3 ─────────── Server
   Client <── [Stream End] ───────── Server
   
   Example: ListUsers(filter) → stream of Users
   Use case: Large result sets, real-time updates
""")

# 3. Client Streaming
print("""
3. CLIENT STREAMING
   Client ─── Request 1 ──────────> Server
   Client ─── Request 2 ──────────> Server
   Client ─── Request 3 ──────────> Server
   Client ─── [Stream End] ───────> Server
   Client <── Response ─────────── Server
   
   Example: UploadUsers(stream of Users) → UploadResult
   Use case: File upload, batch operations
""")

# 4. Bidirectional Streaming
print("""
4. BIDIRECTIONAL STREAMING
   Client <─── Messages ───> Server
   (Full duplex, independent streams)
   
   Example: Chat(stream Messages) ↔ stream Messages
   Use case: Chat, collaborative editing, gaming
""")

# gRPC vs REST comparison
print("\ngRPC vs REST Performance:")
print("  Payload size: gRPC is 3-10x smaller")
print("  Latency: gRPC is 5-10x faster")
print("  Throughput: gRPC handles more requests/second")
print("  Browser support: REST (gRPC needs grpc-web proxy)")

SOAP

SOAP (Simple Object Access Protocol) is a mature, XML-based protocol with built-in standards for security, transactions, and reliability. It's common in enterprise and B2B integrations.

<!-- SOAP Envelope Structure -->
<?xml version="1.0"?>
<soap:Envelope 
    xmlns:soap="http://www.w3.org/2003/05/soap-envelope"
    xmlns:user="http://example.com/userservice">
    
    <!-- Optional: Headers for metadata -->
    <soap:Header>
        <user:Authentication>
            <user:Token>abc123...</user:Token>
        </user:Authentication>
    </soap:Header>
    
    <!-- Required: Body with actual request/response -->
    <soap:Body>
        <user:GetUserRequest>
            <user:UserId>123</user:UserId>
        </user:GetUserRequest>
    </soap:Body>
    
</soap:Envelope>

<!-- SOAP Response -->
<soap:Envelope>
    <soap:Body>
        <user:GetUserResponse>
            <user:User>
                <user:Id>123</user:Id>
                <user:Name>Alice</user:Name>
                <user:Email>alice@example.com</user:Email>
            </user:User>
        </user:GetUserResponse>
    </soap:Body>
</soap:Envelope>

<!-- SOAP Fault (Error) -->
<soap:Envelope>
    <soap:Body>
        <soap:Fault>
            <soap:Code>
                <soap:Value>soap:Sender</soap:Value>
            </soap:Code>
            <soap:Reason>
                <soap:Text>User not found</soap:Text>
            </soap:Reason>
        </soap:Fault>
    </soap:Body>
</soap:Envelope>

API Authentication

API Authentication Methods:

1. API KEYS
   Authorization: Api-Key abc123def456
   
   Pros: Simple, good for server-to-server
   Cons: No user identity, easy to leak
   
2. BASIC AUTH (HTTP Basic)
   Authorization: Basic base64(username:password)
   
   Pros: Simple, universal support
   Cons: Credentials in every request (use HTTPS!)
   
3. BEARER TOKENS (JWT)
   Authorization: Bearer eyJhbGciOiJIUzI1NiIs...
   
   JWT Structure:
   Header.Payload.Signature
   
   Payload: {
     "sub": "user123",
     "name": "Alice",
     "exp": 1735689600,
     "iat": 1735603200
   }
   
   Pros: Stateless, contains user info
   Cons: Can't revoke (until expiry)
   
4. OAUTH 2.0 (Delegated authorization)
   Flows:
   - Authorization Code (web apps)
   - Client Credentials (machine-to-machine)
   - Implicit (deprecated for SPAs)
   - PKCE (mobile/SPA)
   
   Pros: Industry standard, delegated access
   Cons: Complex to implement correctly
   
5. MUTUAL TLS (mTLS)
   Both client and server present certificates
   
   Pros: Very secure, no shared secrets
   Cons: Complex certificate management

# JWT token handling example
import json
import base64
import hmac
import hashlib
import time

def base64url_encode(data):
    """Base64 URL-safe encoding without padding"""
    if isinstance(data, str):
        data = data.encode()
    return base64.urlsafe_b64encode(data).rstrip(b'=').decode()

def base64url_decode(data):
    """Base64 URL-safe decoding with padding"""
    padding = 4 - len(data) % 4
    if padding != 4:
        data += '=' * padding
    return base64.urlsafe_b64decode(data)

def create_jwt(payload, secret):
    """Create a JWT token (simplified, HS256 only)"""
    header = {"alg": "HS256", "typ": "JWT"}
    
    # Encode header and payload
    header_b64 = base64url_encode(json.dumps(header))
    payload_b64 = base64url_encode(json.dumps(payload))
    
    # Create signature
    message = f"{header_b64}.{payload_b64}"
    signature = hmac.new(
        secret.encode(),
        message.encode(),
        hashlib.sha256
    ).digest()
    signature_b64 = base64url_encode(signature)
    
    return f"{header_b64}.{payload_b64}.{signature_b64}"

def decode_jwt(token):
    """Decode JWT payload (without verification)"""
    parts = token.split('.')
    if len(parts) != 3:
        raise ValueError("Invalid JWT format")
    
    payload = json.loads(base64url_decode(parts[1]))
    return payload

# Example usage
print("JWT Token Example")
print("=" * 60)

secret = "my-super-secret-key"
payload = {
    "sub": "user123",
    "name": "Alice",
    "email": "alice@example.com",
    "roles": ["admin", "user"],
    "iat": int(time.time()),
    "exp": int(time.time()) + 3600  # 1 hour
}

token = create_jwt(payload, secret)
print(f"Token: {token[:50]}...")
print()

decoded = decode_jwt(token)
print("Decoded payload:")
print(json.dumps(decoded, indent=2))
print()
print("Note: Always verify signature in production!")
print("Use libraries like PyJWT, not manual implementation.")

API Versioning & Evolution

API Versioning Strategies:

1. URL PATH VERSIONING (Most common)
   https://api.example.com/v1/users
   https://api.example.com/v2/users
   
   Pros: Clear, easy routing, cache-friendly
   Cons: Not RESTful (version isn't a resource)

2. QUERY PARAMETER
   https://api.example.com/users?version=1
   https://api.example.com/users?api-version=2024-01-15
   
   Pros: Easy to add
   Cons: Easy to forget, caching issues

3. HEADER VERSIONING
   Accept: application/vnd.example.v1+json
   Api-Version: 2
   
   Pros: Clean URLs, RESTful
   Cons: Harder to test (need to set headers)

4. CONTENT NEGOTIATION
   Accept: application/vnd.example.user.v2+json
   
   Pros: Very RESTful
   Cons: Complex, rarely used

Best Practice:
• Use URL path versioning (/v1/, /v2/) for major versions
• Avoid breaking changes; use additive changes
• Deprecate old versions with sunset headers
• Semantic versioning: major.minor.patch

Sunset Header (RFC 8594):
Sunset: Sat, 31 Dec 2025 23:59:59 GMT
Deprecation: true
Link: <https://api.example.com/v2>; rel="successor-version"

Hands-On Exercises

# API protocol comparison tool
def compare_api_protocols():
    """Compare REST, GraphQL, gRPC, SOAP"""
    
    protocols = {
        "REST": {
            "transport": "HTTP/1.1 or HTTP/2",
            "format": "JSON (usually)",
            "typing": "Optional (OpenAPI)",
            "streaming": "No (use SSE/WebSocket)",
            "browser": "Yes",
            "learning_curve": "Low",
            "best_for": "Public APIs, CRUD, caching"
        },
        "GraphQL": {
            "transport": "HTTP (POST)",
            "format": "JSON",
            "typing": "Required (SDL)",
            "streaming": "Subscriptions (WebSocket)",
            "browser": "Yes",
            "learning_curve": "Medium",
            "best_for": "Flexible queries, mobile apps"
        },
        "gRPC": {
            "transport": "HTTP/2",
            "format": "Protocol Buffers (binary)",
            "typing": "Required (.proto)",
            "streaming": "4 modes (uni, server, client, bidi)",
            "browser": "Via grpc-web proxy",
            "learning_curve": "Medium-High",
            "best_for": "Microservices, high performance"
        },
        "SOAP": {
            "transport": "HTTP, SMTP, etc",
            "format": "XML",
            "typing": "Required (WSDL, XSD)",
            "streaming": "No",
            "browser": "Difficult",
            "learning_curve": "High",
            "best_for": "Enterprise, B2B, WS-* standards"
        }
    }
    
    print("API Protocol Comparison")
    print("=" * 70)
    
    for name, details in protocols.items():
        print(f"\n{name}")
        print("-" * 40)
        for key, value in details.items():
            print(f"  {key.replace('_', ' ').title()}: {value}")

compare_api_protocols()

Summary & Next Steps

Continue the Series