Part 13: HTTP/HTTPS & TLS — The Web Protocol Stack

HTTP Fundamentals

HTTP (Hypertext Transfer Protocol) is a stateless, text-based, request/response protocol operating at Layer 7 (Application) of the OSI model. A client sends a request; the server sends a response. Every interaction is independent — the server retains no memory of previous requests (cookies and sessions are application-level workarounds for this statelessness).

Request/Response Structure

An HTTP request consists of a request line (method + path + version), headers (key: value pairs), and an optional body. The response mirrors this with a status line (version + status code + reason), headers, and body.

# A raw HTTP/1.1 request (what curl sends under the hood)
printf 'GET / HTTP/1.1\r\nHost: example.com\r\nAccept: text/html\r\nConnection: close\r\n\r\n' | \
  nc example.com 80

# Response structure:
# HTTP/1.1 200 OK           ← status line
# Content-Type: text/html   ← headers
# Content-Length: 1256
#                            ← blank line separates headers from body
# <!doctype html>...        ← body

HTTP Methods

Method	Semantics	Idempotent	Body
GET	Retrieve a resource	Yes	No
POST	Submit data / create resource	No	Yes
PUT	Replace entire resource	Yes	Yes
PATCH	Partial update	No*	Yes
DELETE	Remove resource	Yes	Optional
HEAD	GET without body (headers only)	Yes	No
OPTIONS	Discover allowed methods (CORS preflight)	Yes	No

HTTP Status Codes

Category	Code	Meaning
1xx Informational	100	Continue (client should send body)
1xx Informational	101	Switching Protocols (WebSocket upgrade)
2xx Success	200	OK
	201	Created (resource created, check Location header)
	204	No Content (success, empty body — common for DELETE)
3xx Redirection	301	Moved Permanently (update bookmarks, SEO)
	302	Found (temporary redirect)
	304	Not Modified (use cached version)
	307	Temporary Redirect (preserves method)
4xx Client Error	400	Bad Request (malformed syntax)
	401	Unauthorized (no/invalid credentials)
	403	Forbidden (credentials valid, permission denied)
	404	Not Found
	429	Too Many Requests (rate limited)
5xx Server Error	500	Internal Server Error (generic server crash)
	502	Bad Gateway (upstream server returned invalid response)
	503	Service Unavailable (overloaded or maintenance)
	504	Gateway Timeout (upstream server didn't respond in time)

HTTP Headers

Headers carry metadata about the request, response, or the body content. They control caching, authentication, content negotiation, connection management, and security policies.

Header	Direction	Purpose
`Host`	Request	Target hostname (required in HTTP/1.1 — enables virtual hosting)
`Content-Type`	Both	MIME type of body (`application/json`, `text/html`)
`Authorization`	Request	Credentials (`Bearer <token>`, `Basic <b64>`)
`Cache-Control`	Both	Caching directives (`max-age=3600`, `no-store`)
`Connection`	Request	`keep-alive` (reuse TCP connection) or `close`
`Accept-Encoding`	Request	Supported compression (`gzip, br`)
`Content-Encoding`	Response	Compression applied to body (`gzip`, `br`)
`ETag`	Response	Resource version fingerprint (for conditional requests)
`Strict-Transport-Security`	Response	HSTS — force HTTPS for future requests
`X-Request-Id`	Both	Correlation ID for distributed tracing

HTTP/2 & HTTP/3

HTTP/1.1's biggest limitation is head-of-line (HOL) blocking: a single TCP connection can only process one request at a time. Browsers work around this by opening 6 parallel TCP connections per domain — wasteful in handshakes and memory. HTTP/2 solves this with multiplexing: multiple requests share a single TCP connection as independent streams.

HTTP/1.1 vs HTTP/2 Multiplexing

flowchart LR
    subgraph HTTP1["HTTP/1.1 — Sequential"]
        direction TB
        R1["Request 1"] --> W1["Wait for Response 1"]
        W1 --> R2["Request 2"]
        R2 --> W2["Wait for Response 2"]
        W2 --> R3["Request 3"]
        R3 --> W3["Wait for Response 3"]
    end

    subgraph HTTP2["HTTP/2 — Multiplexed Streams"]
        direction TB
        S1["Stream 1: GET /index.html"]
        S2["Stream 2: GET /style.css"]
        S3["Stream 3: GET /app.js"]
        S1 --> MUX["Single TCP Connection\n(frames interleaved)"]
        S2 --> MUX
        S3 --> MUX
    end

            
            HTTP/2 Key Features: (1) Multiplexing — multiple streams over one connection, no HOL blocking at HTTP level. (2) Header compression (HPACK) — eliminates redundant header bytes across requests. (3) Server push — server can proactively send resources the client will need. (4) Stream prioritisation — client hints which resources matter most. (5) Binary framing — faster parsing than text-based HTTP/1.1. However, HTTP/2 still suffers from TCP-level HOL blocking — a single lost TCP packet stalls all streams. HTTP/3 (QUIC over UDP) eliminates this entirely.
        

# Check if a server supports HTTP/2
curl -sI --http2 https://www.google.com | head -1
# HTTP/2 200

# Force HTTP/1.1 for comparison
curl -sI --http1.1 https://www.google.com | head -1
# HTTP/1.1 200 OK

# Check HTTP/3 (QUIC) support via Alt-Svc header
curl -sI https://www.google.com | grep -i alt-svc
# alt-svc: h3=":443"; ma=2592000

# View negotiated protocol with openssl
openssl s_client -connect www.google.com:443 -alpn h2 2>/dev/null | grep "ALPN"
# ALPN protocol: h2

TLS 1.3 Handshake

TLS (Transport Layer Security) encrypts HTTP traffic, turning HTTP into HTTPS. TLS 1.3 reduced the handshake from 2 round-trips (TLS 1.2) to 1 round-trip — and supports 0-RTT resumption for returning clients. The handshake establishes a shared secret using ephemeral Diffie-Hellman key exchange (ECDHE), providing forward secrecy.

TLS 1.3 Handshake (1-RTT)

sequenceDiagram
    participant C as Client
    participant S as Server

    C->>S: ClientHello (supported ciphers, key_share, SNI)
    Note over C: Proposes ECDHE key shares upfront
    S->>C: ServerHello (chosen cipher, key_share)
    S->>C: EncryptedExtensions + Certificate + CertificateVerify + Finished
    Note over S: Server proves identity with certificate
    C->>S: Finished (client confirms handshake)
    Note over C,S: Encrypted Application Data flows (1-RTT total)
    C->>S: HTTP Request (encrypted)
    S->>C: HTTP Response (encrypted)

# View the full TLS handshake with openssl
openssl s_client -connect www.example.com:443 -tls1_3 2>/dev/null | \
  grep -E "Protocol|Cipher|Server certificate|subject|issuer|Verify"

# Output includes:
#   Protocol  : TLSv1.3
#   Cipher    : TLS_AES_256_GCM_SHA384
#   Server certificate: subject, issuer, validity dates
#   Verify return code: 0 (ok)

# View certificate details
echo | openssl s_client -connect www.google.com:443 2>/dev/null | \
  openssl x509 -noout -subject -issuer -dates -fingerprint

# Check which TLS versions a server supports
nmap --script ssl-enum-ciphers -p 443 example.com 2>/dev/null | \
  grep -E "TLSv|SSLv"

Certificate Chain Verification

When a client connects to an HTTPS server, it verifies the certificate chain: the server's leaf certificate is signed by an intermediate CA, which is signed by a root CA trusted by the client's OS/browser. If any link in the chain is invalid, expired, or revoked, the connection fails.

# View the full certificate chain
openssl s_client -connect www.google.com:443 -showcerts 2>/dev/null | \
  grep -E "s:|i:" | head -10
# s: = subject (this certificate is for)
# i: = issuer (signed by)

# Check certificate expiry
echo | openssl s_client -connect example.com:443 2>/dev/null | \
  openssl x509 -noout -dates
# notBefore=Jan 13 00:00:00 2026 GMT
# notAfter=Apr 13 23:59:59 2027 GMT

# Verify chain against system trust store
openssl verify -CAfile /etc/ssl/certs/ca-certificates.crt server.pem

# View certificates trusted by your system
ls /etc/ssl/certs/ | head -20           # Debian/Ubuntu
ls /etc/pki/tls/certs/ | head -20       # RHEL/CentOS

            
            HSTS & Certificate Pinning: HSTS (HTTP Strict Transport Security) tells browsers to always use HTTPS for a domain — even if the user types http://. The header Strict-Transport-Security: max-age=31536000; includeSubDomains; preload protects against downgrade attacks and SSL stripping. Certificate pinning goes further: the client only accepts specific certificate fingerprints, preventing even a compromised CA from issuing a fraudulent cert. Pinning is deprecated in browsers (too brittle) but still used in mobile apps and internal services via tools like cert-manager with pinned CAs.
        

Debugging with curl

curl is the essential tool for debugging HTTP — it shows you exactly what's going over the wire.

# Verbose output: see request/response headers + TLS handshake
curl -v https://httpbin.org/get 2>&1 | head -40
# Lines starting with > are sent (request)
# Lines starting with < are received (response)
# Lines starting with * are connection info (TLS, DNS)

# Headers only (HEAD request)
curl -I https://httpbin.org/get

# Custom headers + POST with JSON body
curl -X POST https://httpbin.org/post \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer my-token-123" \
  -d '{"name": "test", "value": 42}'

# Follow redirects (-L) and show redirect chain
curl -L -v https://httpbin.org/redirect/3 2>&1 | grep -E "^(< HTTP|< location:)"

# Timing breakdown: DNS, TCP, TLS, TTFB, total
curl -o /dev/null -s -w "\
  DNS:        %{time_namelookup}s\n\
  TCP:        %{time_connect}s\n\
  TLS:        %{time_appconnect}s\n\
  TTFB:       %{time_starttransfer}s\n\
  Total:      %{time_total}s\n\
  Status:     %{http_code}\n\
  Size:       %{size_download} bytes\n" \
  https://www.google.com

# Test specific TLS version
curl --tlsv1.3 -v https://example.com 2>&1 | grep "SSL connection"
# SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384

# Download with progress bar
curl -# -o file.tar.gz https://example.com/large-file.tar.gz

# Send request with client certificate (mTLS)
curl --cert client.pem --key client-key.pem https://secure-api.example.com/data

# Ignore certificate errors (NEVER in production — debugging only)
curl -k https://self-signed.example.com/api

# Rate limit / retry with backoff
curl --retry 3 --retry-delay 2 --retry-max-time 30 https://flaky-api.example.com/data

Cloud Pattern

How Kubernetes Ingress Terminates TLS

In Kubernetes, TLS termination typically happens at the Ingress controller (NGINX, Traefik, or cloud load balancer). The flow: (1) cert-manager automatically provisions TLS certificates from Let's Encrypt using ACME protocol. (2) Certificates are stored as Kubernetes Secret objects. (3) The Ingress resource references the Secret and defines routing rules. (4) The Ingress controller terminates TLS at the edge — traffic between the controller and backend pods is typically plain HTTP over the cluster network. This is called TLS termination at ingress. For end-to-end encryption (zero-trust), you'd use a service mesh (Istio, Linkerd) with mTLS between pods.

Kubernetescert-managerLet's EncryptTLS Termination

Exercises

# Exercise 1: Inspect HTTP headers and TLS info
curl -v https://www.google.com 2>&1 | grep -E "^(\*|>|<)" | head -30

# Exercise 2: Measure TLS handshake time
curl -o /dev/null -s -w "TLS handshake: %{time_appconnect}s\n" https://www.google.com

# Exercise 3: View certificate chain
echo | openssl s_client -connect github.com:443 2>/dev/null | grep -E "s:|i:"

# Exercise 4: Check HTTP/2 support
curl -sI --http2 https://www.cloudflare.com | head -1

# Exercise 5: Find all response headers from a site
curl -sI https://httpbin.org/get | sort

Conclusion & Next Steps

HTTP is deceptively simple on the surface — a text-based request/response protocol — but the ecosystem around it (status codes, headers, caching, compression, multiplexing, TLS) is what makes the modern web fast and secure. HTTP/2 eliminated HTTP-level head-of-line blocking; HTTP/3 (QUIC) eliminated TCP-level HOL blocking. TLS 1.3 reduced the handshake to 1-RTT and provides forward secrecy by default. And curl -v remains the single best debugging tool for understanding what's happening between your client and any server.

PreviousPart 12: DNS Deep Dive Next Part 14: Routing, NAT & Firewalls

Cookie Consent

Part 13: HTTP/HTTPS & TLS — The Web Protocol Stack

Table of Contents

HTTP Fundamentals

Request/Response Structure

HTTP Methods

HTTP Status Codes

HTTP Headers

HTTP/2 & HTTP/3

TLS 1.3 Handshake

Certificate Chain Verification

Debugging with curl

How Kubernetes Ingress Terminates TLS

Exercises

Conclusion & Next Steps

Cookie Consent

Part 13: HTTP/HTTPS & TLS — The Web Protocol Stack

Table of Contents

HTTP Fundamentals

Request/Response Structure

HTTP Methods

HTTP Status Codes

HTTP Headers

HTTP/2 & HTTP/3

TLS 1.3 Handshake

Certificate Chain Verification

Debugging with curl

How Kubernetes Ingress Terminates TLS

Exercises

Conclusion & Next Steps

Continue the Series

Part 12: DNS Deep Dive

Part 14: Routing, NAT & Firewalls