Part 2: Evolution of Infrastructure

Introduction

Infrastructure did not arrive at its current state overnight. It evolved through six distinct eras, each solving problems created by the previous era while introducing new challenges of its own.

Understanding this evolution is not just history — it is the key to understanding why modern tools exist and when to use each approach.

                                flowchart TD
                                    A[1960s–1990s: Bare Metal] --> B[1999–2006: Virtual Machines]
                                    B --> C[2006–2013: Cloud Computing]
                                    C --> D[2013–2017: Containers]
                                    D --> E[2014–2020: Serverless]
                                    E --> F[2020–Present: Platform Engineering]
                            

Why Study the Evolution?

Era 1: Bare Metal (1960s–1990s)

In the bare metal era, infrastructure meant physical servers that you owned, racked, and maintained. If you needed more capacity, you purchased hardware, waited weeks for delivery, and manually installed it in your data centre.

The Challenges

The Workflow

                                flowchart LR
                                    A[Request Hardware] --> B[Procurement
4-12 weeks]
                                    B --> C[Rack & Cable]
                                    C --> D[Install OS]
                                    D --> E[Configure Network]
                                    E --> F[Deploy Application]
                                    F --> G[Monitor Manually]
                            

When bare metal still makes sense today:

• High-frequency trading (microsecond latency requirements)
• Specialised GPU/FPGA workloads (ML training clusters)
• Regulatory requirements mandating physical hardware control
• Extremely predictable, high-volume workloads (cheaper than cloud at scale)

Era 2: Virtual Machines (1999–2006)

The Revolution

VMware released its first product in 1999, and it changed everything. Suddenly, a single physical server could run 10–20 isolated virtual machines, each with its own operating system.

Key benefits of virtualization:

• Consolidation — multiple workloads on one physical server
• Isolation — VMs cannot interfere with each other
• Portability — VM images can be moved between physical hosts
• Snapshots — capture and restore VM state instantly
• Live migration — move running VMs without downtime

# Example: Creating a VM with VirtualBox CLI
VBoxManage createvm --name "WebServer" --ostype "Ubuntu_64" --register

# Allocate 2 CPUs and 4GB RAM
VBoxManage modifyvm "WebServer" --cpus 2 --memory 4096

# Create a 40GB virtual disk
VBoxManage createhd --filename "WebServer.vdi" --size 40960

# Start the VM
VBoxManage startvm "WebServer" --type headless

Limitations

While VMs solved the utilization problem, they introduced new challenges:

• VM sprawl — easy to create, hard to manage thousands
• OS overhead — each VM runs a full operating system (1–2 GB per VM)
• Boot time — minutes to start (not seconds)
• Still manual — provisioning required vSphere admins clicking through GUIs

Era 3: Cloud Computing (2006–2013)

In 2006, Amazon launched Amazon Web Services (AWS) with S3 (storage) and EC2 (compute). This was the moment infrastructure became an API.

Key Characteristics

The five essential characteristics of cloud computing:

The Providers

# The cloud revolution in one command — provision a server in seconds:
aws ec2 run-instances \
    --image-id ami-0abcdef1234567890 \
    --instance-type t3.micro \
    --count 1 \
    --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=MyServer}]'

# Compare: bare metal takes 4-12 weeks. Cloud takes 30 seconds.

Era 4: Containers (2013–2017)

Docker launched in 2013 and fundamentally changed how applications are packaged and deployed. Containers provide OS-level virtualization — lighter and faster than full VMs.

Containers vs Virtual Machines

                                flowchart TD
                                    subgraph VM[Virtual Machine Approach]
                                        HW1[Hardware] --> HV1[Hypervisor]
                                        HV1 --> GOS1[Guest OS 1
~1GB]
                                        HV1 --> GOS2[Guest OS 2
~1GB]
                                        GOS1 --> APP1[App A]
                                        GOS2 --> APP2[App B]
                                    end
                                    subgraph CT[Container Approach]
                                        HW2[Hardware] --> OS2[Host OS]
                                        OS2 --> CR[Container Runtime]
                                        CR --> C1[Container 1
~50MB]
                                        CR --> C2[Container 2
~50MB]
                                    end
                            

# Dockerfile: Package an application as a container
cat <<'EOF' > Dockerfile
FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
EXPOSE 8000
CMD ["python", "app.py"]
EOF

# Build and run
docker build -t my-web-app:1.0 .
docker run -d -p 8000:8000 my-web-app:1.0

Orchestration: Kubernetes

As organisations deployed hundreds or thousands of containers, they needed a way to manage them. Kubernetes (released by Google in 2014) became the standard container orchestration platform.

Kubernetes handles: scheduling, scaling, networking, storage, health checks, and rolling deployments across a cluster of machines.

Era 5: Serverless (2014–2020)

The Model

AWS Lambda launched in 2014 with a radical proposition: write a function, upload it, and the cloud runs it on demand. No servers to manage. No capacity planning. Pay only for the milliseconds your code executes.

# Deploy a serverless function with AWS CLI
aws lambda create-function \
    --function-name hello-world \
    --runtime python3.11 \
    --role arn:aws:iam::123456789:role/lambda-role \
    --handler lambda_function.handler \
    --zip-file fileb://function.zip

# Invoke it
aws lambda invoke \
    --function-name hello-world \
    --payload '{"name": "World"}' \
    output.json

Trade-offs

Era 6: Platform Engineering (2020–Present)

Internal Developer Platforms

Platform engineering emerged as a response to the cognitive overload of modern infrastructure. With hundreds of cloud services, Kubernetes, Terraform, CI/CD pipelines, and observability tools, developers were drowning in operational complexity.

Platform engineering builds Internal Developer Platforms (IDPs) that abstract away infrastructure complexity while maintaining control and governance:

                                flowchart TD
                                    DEV[Developer] --> IDP[Internal Developer Platform]
                                    IDP --> CICD[CI/CD Pipeline]
                                    IDP --> K8S[Kubernetes]
                                    IDP --> TF[Terraform]
                                    IDP --> MON[Monitoring]
                                    CICD --> CLOUD[Cloud Provider]
                                    K8S --> CLOUD
                                    TF --> CLOUD
                                    MON --> CLOUD
                            

Golden Paths

Platform teams create “golden paths” — pre-approved, well-tested patterns for common tasks. Instead of giving developers raw access to 200+ cloud services, the platform offers curated templates:

• “Deploy a web service” → pre-configured Kubernetes deployment + ingress + monitoring
• “Create a database” → managed PostgreSQL with backups, encryption, and connection pooling
• “Add a queue” → managed message broker with dead-letter handling

Exercises

Conclusion & Next Steps

The evolution of infrastructure is a story of increasing abstraction. Each era hid more complexity from the user while delivering faster provisioning, better utilization, and lower costs:

• Bare metal → You manage everything
• VMs → Hardware is abstracted
• Cloud → Data centres are abstracted
• Containers → OS is abstracted
• Serverless → Servers are abstracted
• Platforms → Infrastructure decisions are abstracted