AI Application Development Mastery Part 1: Foundations & Evolution of AI Apps

1. The Pre-LLM Era

Before large language models, building "intelligent" software meant carefully hand-crafting rules, features, and pipelines. Each AI application was a bespoke engineering effort, and the gap between what researchers could demonstrate in labs and what practitioners could deploy in production was enormous.

1.1 ELIZA & Expert Systems

The story of AI applications begins in 1966 with ELIZA, Joseph Weizenbaum's landmark program at MIT. ELIZA simulated a Rogerian psychotherapist using simple pattern matching and substitution rules — no understanding, no learning, just keyword detection and template responses.

# A simplified ELIZA-style pattern matcher
# This illustrates the core technique: pattern matching + template responses

import re

RULES = [
    (r'I need (.*)',
     ["Why do you need {0}?", "Would it really help you to get {0}?"]),
    (r'I am (.*)',
     ["How long have you been {0}?", "How does being {0} make you feel?"]),
    (r'I feel (.*)',
     ["Tell me more about feeling {0}.", "Do you often feel {0}?"]),
    (r'(.*) mother(.*)',
     ["Tell me more about your family.", "How does that make you feel?"]),
    (r'(.*)',
     ["Please go on.", "Can you elaborate on that?"])
]

def eliza_respond(user_input):
    """Match user input against patterns and return a response."""
    for pattern, responses in RULES:
        match = re.match(pattern, user_input, re.IGNORECASE)
        if match:
            response = responses[0]  # In real ELIZA, this would rotate
            # Substitute captured groups into the response
            return response.format(*match.groups())
    return "Please tell me more."

# Example conversation
print(eliza_respond("I need help with my project"))
# Output: "Why do you need help with my project?"
print(eliza_respond("I am feeling overwhelmed"))
# Output: "How long have you been feeling overwhelmed?"

Despite its simplicity, ELIZA revealed something profound: people anthropomorphize conversational systems. Weizenbaum's secretary reportedly asked him to leave the room so she could have a private conversation with the program. This "ELIZA effect" — humans attributing understanding to systems that merely pattern-match — remains relevant today when people interact with ChatGPT.

The 1970s-1980s saw the rise of expert systems — rule-based programs that captured domain expertise in if-then rules:

1.2 Classical ML Pipelines

By the 2000s, the dominant paradigm for AI applications was the classical machine learning pipeline: collect data, engineer features, train a model, deploy it behind an API. This worked, but it was labor-intensive and domain-specific:

# Classical ML pipeline for text classification (pre-LLM era)
# Each step required specialized engineering

# pip install scikit-learn
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.naive_bayes import MultinomialNB
from sklearn.pipeline import Pipeline
from sklearn.model_selection import train_test_split

# Step 1: Collect and label data (weeks of work)
texts = [
    "The stock market rallied today on earnings reports",
    "New study shows benefits of Mediterranean diet",
    "SpaceX successfully launches Starship prototype",
    "Federal Reserve raises interest rates by 25 basis points",
    "Clinical trials show promising results for new cancer drug",
    "NASA's James Webb telescope captures distant galaxy",
]
labels = ["finance", "health", "technology", "finance", "health", "technology"]

# Step 2: Feature engineering (TF-IDF, n-grams, custom features)
# In production, this alone could take weeks of experimentation
vectorizer = TfidfVectorizer(
    max_features=5000,
    ngram_range=(1, 2),       # Unigrams and bigrams
    stop_words='english',
    min_df=1,                 # Use 1 for small datasets (2+ in production)
    max_df=0.95
)

# Step 3: Train a classifier
pipeline = Pipeline([
    ('tfidf', vectorizer),
    ('classifier', MultinomialNB())
])

# Step 4: Train/test split, fit, evaluate
X_train, X_test, y_train, y_test = train_test_split(
    texts, labels, test_size=0.2, random_state=42
)
pipeline.fit(X_train, y_train)

# Predict on a new text
prediction = pipeline.predict(["New AI chip breaks speed records"])
print(f"Prediction: {prediction[0]}")

# Step 5: Deploy — but only for THIS specific task
# Need sentiment analysis? Start over from Step 1.
# Need summarization? Completely different pipeline.
# Need Q&A? Different architecture entirely.

1.3 NLP Before Transformers

Natural Language Processing before 2017 was dominated by sequential models that processed text one token at a time:

Each advancement solved some problems but introduced others. RNNs could model sequences but struggled with long-range dependencies. LSTMs added gating mechanisms to preserve information over longer spans but were fundamentally sequential — they couldn't be parallelized on GPUs, making them slow to train on large datasets.

2. The Deep Learning Revolution

The 2017 paper "Attention Is All You Need" introduced the Transformer architecture and fundamentally changed the trajectory of AI. By replacing recurrence with self-attention, Transformers could process entire sequences in parallel, enabling massive scale-up in both model size and training data. This section traces the two breakthroughs that made modern LLMs possible: the attention mechanism itself, and the pretrain-finetune paradigm pioneered by BERT and GPT.

2.1 Attention Is All You Need

In June 2017, Vaswani et al. published "Attention Is All You Need" — arguably the most consequential machine learning paper of the decade. The transformer architecture they introduced replaced recurrence entirely with self-attention, allowing every token in a sequence to attend to every other token simultaneously.

# Simplified self-attention mechanism (conceptual)
# This is the core innovation that powers every modern LLM

import numpy as np

def self_attention(query, key, value, d_k):
    """
    Scaled dot-product attention.

    Instead of processing tokens one-by-one (RNN),
    every token can "look at" every other token in parallel.

    Args:
        query:  What am I looking for? (n_tokens x d_k)
        key:    What do I contain? (n_tokens x d_k)
        value:  What information do I carry? (n_tokens x d_v)
        d_k:    Dimension of key vectors (for scaling)

    Returns:
        Weighted combination of values based on attention scores
    """
    # Step 1: Compute attention scores (how relevant is each token?)
    scores = np.matmul(query, key.T) / np.sqrt(d_k)

    # Step 2: Softmax to get attention weights (probabilities)
    attention_weights = np.exp(scores) / np.sum(np.exp(scores), axis=-1, keepdims=True)

    # Step 3: Weighted sum of values
    output = np.matmul(attention_weights, value)

    return output, attention_weights

# Example: 3 tokens, embedding dimension 4
# "The cat sat" — each token is a 4-dimensional vector
np.random.seed(42)
tokens = np.random.randn(3, 4)  # 3 tokens, 4 dimensions

# In practice, Q/K/V are linear projections of the input
Q = tokens  # Simplified — real transformers use learned projections
K = tokens
V = tokens

output, weights = self_attention(Q, K, V, d_k=4)
print("Attention weights (who attends to whom):")
print(weights.round(3))
# Each row shows how much each token attends to every other token

The transformer's key innovations were:

• Self-attention: Every token can attend to every other token — capturing long-range dependencies without the vanishing gradient problem
• Parallelization: Unlike RNNs, all positions are computed simultaneously, enabling massive GPU parallelism
• Positional encoding: Since there's no inherent sequence order, position information is injected via sinusoidal encodings
• Multi-head attention: Multiple attention "heads" learn different types of relationships simultaneously

2.2 BERT, GPT & the Pretrain-Finetune Paradigm

The transformer architecture spawned two dominant paradigms that would reshape NLP:

3. The LLM Era

The release of ChatGPT in November 2022 was a watershed moment — the first time a general-purpose AI system achieved mass consumer adoption, reaching 100 million users in just two months. But ChatGPT was only the beginning. The real revolution was what came next: developers realized they could combine LLMs with external data retrieval (RAG) and tool use (agents) to build applications that go far beyond simple chat. This section covers the ChatGPT inflection point and the two application patterns — RAG and agents — that define the modern AI app landscape.

3.1 The ChatGPT Moment

On November 30, 2022, OpenAI released ChatGPT, and the world changed. Within five days, it had one million users. Within two months, 100 million. It wasn't just a better AI model — it was a better interface. By combining GPT-3.5 with reinforcement learning from human feedback (RLHF) and wrapping it in a simple chat interface, OpenAI made advanced AI accessible to everyone.

# The simplicity that changed everything:
# Before ChatGPT — building an AI app required months of work
# After ChatGPT — a single API call

# pip install openai
import os
from openai import OpenAI

# Set your API key: export OPENAI_API_KEY="sk-..."
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

# This is all it takes to build an AI-powered application
try:
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {"role": "system", "content": "You are a helpful assistant that analyzes code."},
            {"role": "user", "content": "Explain this Python function and suggest improvements: def f(x): return x*x+2*x+1"}
        ],
        temperature=0.7,
        max_tokens=500
    )

    print(response.choices[0].message.content)
    # The model understands code, can explain it, and suggests improvements
    # No training data needed. No ML pipeline. No feature engineering.
except Exception as e:
    print(f"API call failed: {e}")

ChatGPT's impact triggered a cascade of developments:

• GPT-4 (March 2023) — multimodal, dramatically more capable reasoning
• Claude (Anthropic) — focused on safety and helpfulness
• Gemini (Google) — natively multimodal, massive context windows
• Llama (Meta) — open-weight models that democratized LLM access
• Mistral — efficient open models competitive with much larger ones

3.2 RAG & Agents Emerge

As developers pushed LLMs into production, two critical limitations became apparent: LLMs hallucinate (generate plausible but false information) and their knowledge has a cutoff date. These limitations spawned the two most important architectural patterns in modern AI development:

# RAG in its simplest form: retrieve then generate
# This pattern powers most production AI applications today

# pip install langchain langchain-openai langchain-community faiss-cpu
import os
from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from langchain_community.vectorstores import FAISS
from langchain.text_splitter import RecursiveCharacterTextSplitter

# Set your API key: export OPENAI_API_KEY="sk-..."

# Step 1: Index your documents
documents = [
    "Our return policy allows returns within 30 days of purchase.",
    "Free shipping is available on orders over $50.",
    "Premium members get 20% off all products.",
    "Gift cards never expire and can be used on any product.",
]

# Step 2: Create embeddings and store in vector database
embeddings = OpenAIEmbeddings()
text_splitter = RecursiveCharacterTextSplitter(chunk_size=200)
splits = text_splitter.create_documents(documents)
vectorstore = FAISS.from_documents(splits, embeddings)

# Step 3: Retrieve relevant context for user query
query = "What's your return policy?"
relevant_docs = vectorstore.similarity_search(query, k=2)

# Step 4: Generate answer grounded in retrieved context
llm = ChatOpenAI(model="gpt-4o")
context = "\n".join([doc.page_content for doc in relevant_docs])
prompt = f"""Answer based ONLY on this context:
{context}

Question: {query}
Answer:"""

response = llm.invoke(prompt)
print(response.content)
# "Our return policy allows returns within 30 days of purchase."
# Grounded in YOUR data — no hallucination

4. The Modern AI App Stack

Building a production AI application requires more than just an LLM API call. The modern AI app stack is a multi-layered architecture spanning foundation models at the base, orchestration frameworks in the middle, and retrieval/memory systems at the top. Understanding each layer — and how frameworks like LangChain, LlamaIndex, Semantic Kernel, and CrewAI map onto them — is essential for making informed architectural decisions.

4.1 Stack Layers Explained

A modern AI application is not just an LLM call — it's a multi-layered system with distinct responsibilities at each level:

# A complete AI application stack in action
# This shows how the layers compose together

# pip install langchain langchain-openai langchain-community chromadb
# Set your API key: export OPENAI_API_KEY="sk-..."

from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from langchain_community.vectorstores import Chroma
from langchain.tools import tool
from langchain.agents import AgentExecutor, create_tool_calling_agent
from langchain_core.prompts import ChatPromptTemplate

# Layer 1: Foundation Model
llm = ChatOpenAI(model="gpt-4o", temperature=0)

# Layer 2-3: Embeddings + Vector Store (for RAG)
embeddings = OpenAIEmbeddings()
vectorstore = Chroma(embedding_function=embeddings)

# Layer 4: Custom Tools
@tool
def search_knowledge_base(query: str) -> str:
    """Search the company knowledge base for relevant information."""
    docs = vectorstore.similarity_search(query, k=3)
    return "\n".join([doc.page_content for doc in docs])

@tool
def calculate_discount(price: float, membership_tier: str) -> str:
    """Calculate the discounted price based on membership tier."""
    discounts = {"basic": 0.05, "premium": 0.15, "vip": 0.25}
    discount = discounts.get(membership_tier.lower(), 0)
    final_price = price * (1 - discount)
    return f"Original: ${price:.2f}, Discount: {discount*100}%, Final: ${final_price:.2f}"

# Layer 5: Agent with tools
tools = [search_knowledge_base, calculate_discount]
prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a helpful customer service agent. Use tools to find answers."),
    ("human", "{input}"),
    ("placeholder", "{agent_scratchpad}")
])
agent = create_tool_calling_agent(llm, tools, prompt)
executor = AgentExecutor(agent=agent, tools=tools, verbose=True)

# The agent decides which tools to use based on the user's question
result = executor.invoke({"input": "I'm a premium member. How much would a $100 item cost me?"})
print(result["output"])

4.2 Comprehensive Framework Comparison

The AI application framework landscape has exploded since 2023. Choosing the right framework is one of the most important architectural decisions you'll make. Here is a comprehensive comparison of the seven major frameworks:

# Quick comparison: Same task in different frameworks
# Task: "Search the web and summarize results"

# pip install langchain langchain-openai langchain-community duckduckgo-search langgraph
# Set your API key: export OPENAI_API_KEY="sk-..."

# ---- LangChain approach (chain-based) ----
from langchain_openai import ChatOpenAI
from langchain.agents import AgentExecutor, create_tool_calling_agent
from langchain_community.tools import DuckDuckGoSearchRun
from langchain_core.prompts import ChatPromptTemplate

llm = ChatOpenAI(model="gpt-4o")
search_tool = DuckDuckGoSearchRun()

# Build an agent that can search and summarize
prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a research assistant. Search the web and summarize results."),
    ("human", "{input}"),
    ("placeholder", "{agent_scratchpad}")
])
agent = create_tool_calling_agent(llm, [search_tool], prompt)
executor = AgentExecutor(agent=agent, tools=[search_tool], verbose=True)
# result = executor.invoke({"input": "Latest developments in AI agents"})

# ---- LangGraph approach (graph-based) ----
from langgraph.graph import StateGraph, START, END

def search_node(state):
    """Node 1: Perform web search."""
    results = search_tool.invoke(state["query"])
    return {"search_results": results}

def summarize_node(state):
    """Node 2: Summarize search results."""
    summary = llm.invoke(f"Summarize: {state['search_results']}")
    return {"summary": summary.content}

# Build graph with explicit state flow
graph = StateGraph(dict)
graph.add_node("search", search_node)
graph.add_node("summarize", summarize_node)
graph.set_entry_point("search")
graph.add_edge("search", "summarize")
graph.add_edge("summarize", END)
# app = graph.compile()
# result = app.invoke({"query": "Latest developments in AI agents"})

# ---- CrewAI approach (role-based) ----
# from crewai import Agent, Task, Crew
# researcher = Agent(role="Researcher", goal="Find information")
# writer = Agent(role="Writer", goal="Summarize findings")
# crew = Crew(agents=[researcher, writer], tasks=[...])
# crew.kickoff()

5. Case Studies

Theory only takes you so far — the best way to understand modern AI architectures is to study how production systems actually work. The three case studies below represent three distinct paradigms: GitHub Copilot (AI-assisted coding via RAG + generation), Perplexity AI (AI-powered search with real-time retrieval), and Devin AI (autonomous software engineering agent). Each reveals different design choices around retrieval, planning, tool use, and human-in-the-loop patterns.

5.1 GitHub Copilot

5.2 Perplexity AI

5.3 Devin AI

Exercises & Self-Assessment

Conclusion & Next Steps

You now have a comprehensive understanding of how AI applications evolved from simple pattern matchers to the sophisticated systems powering today's most innovative products. Here are the key takeaways from Part 1:

• The pre-LLM era taught us fundamental patterns — rule-based reasoning, ML pipelines, sequential NLP — that still inform modern architectures
• Transformers broke the sequential bottleneck with self-attention, enabling parallel processing and long-range dependencies
• GPT-3 introduced in-context learning, shifting AI development from "train models" to "craft prompts"
• ChatGPT made LLMs accessible to everyone, triggering an explosion of AI applications
• RAG and agents are the two core patterns that make LLMs production-ready: RAG for grounded knowledge, agents for action
• The modern AI app stack has distinct layers — choose frameworks based on your specific needs (LangChain for orchestration, LangGraph for stateful agents, LlamaIndex for data-heavy RAG)
• Real-world AI apps like Copilot, Perplexity, and Devin combine multiple patterns into sophisticated systems