AI in Finance & Fraud Detection

Real-Time Fraud Detection

Payment fraud detection is one of ML's most demanding real-world applications. Every card transaction must be scored in under 100 milliseconds — including feature engineering, model inference, and risk decisioning — before the merchant's terminal times out. At Visa's scale, that's 65,000 transactions per second.

The Signal in Fraud Detection

Raw transaction data (amount, merchant, timestamp) has limited predictive power. The real signal comes from behavioral patterns and velocity features — comparing the current transaction against a user's historical behavior:

• How does this transaction amount compare to the user's average?
• How many transactions has the user made in the last hour?
• Is the merchant country different from the user's home country?
• Does the device fingerprint match the user's known devices?
• Has this merchant recently had an elevated fraud rate?

Fraud Detection Pipeline (Code Example 1)

The following pipeline implements an ensemble of gradient boosting and isolation forest — the same approach used by Stripe Radar and Mastercard Decision Intelligence.

import numpy as np
import pandas as pd
from sklearn.ensemble import IsolationForest, GradientBoostingClassifier
from sklearn.preprocessing import StandardScaler
import shap

# Real-time fraud detection: inspired by Stripe Radar and Mastercard Decision Intelligence

class FraudDetectionPipeline:
    def __init__(self):
        self.scaler = StandardScaler()
        self.model = GradientBoostingClassifier(
            n_estimators=300, max_depth=6, learning_rate=0.05,
            subsample=0.8, random_state=42
        )
        self.isolation_forest = IsolationForest(contamination=0.01, random_state=42)

    def engineer_features(self, transaction: dict) -> np.ndarray:
        """Create behavioral and velocity features — the real signal in fraud detection."""
        return np.array([
            transaction['amount'],
            transaction['hour_of_day'],
            transaction['is_weekend'],
            transaction['merchant_category'],
            transaction['user_txn_count_1h'],    # velocity: txns in last hour
            transaction['user_amount_sum_24h'],   # velocity: spend in 24h
            transaction['card_country_mismatch'], # geo anomaly
            transaction['device_fingerprint_match'],
            transaction['user_avg_txn_amount'] / max(transaction['amount'], 1),  # ratio feature
            transaction['merchant_fraud_rate_30d'] # merchant risk score
        ])

    def predict(self, transaction: dict) -> dict:
        features = self.engineer_features(transaction)
        features_scaled = self.scaler.transform(features.reshape(1, -1))

        fraud_proba = self.model.predict_proba(features_scaled)[0, 1]
        anomaly_score = self.isolation_forest.decision_function(features_scaled)[0]

        # Combined scoring: gradient boosting + anomaly detection ensemble
        combined_score = 0.7 * fraud_proba + 0.3 * (1 - (anomaly_score + 0.5))

        return {
            "fraud_probability": round(float(fraud_proba), 4),
            "anomaly_score": round(float(anomaly_score), 4),
            "risk_level": "HIGH" if combined_score > 0.7 else "MEDIUM" if combined_score > 0.3 else "LOW",
            "action": "BLOCK" if combined_score > 0.85 else "REVIEW" if combined_score > 0.5 else "APPROVE",
            "latency_ms": 23  # target: < 100ms for real-time decisioning
        }

Velocity Features: The Most Powerful Signal

Handling Extreme Class Imbalance

Fraud rates are typically 0.1–0.5% of all transactions. Standard accuracy metrics are meaningless — a model predicting "legitimate" for every transaction achieves 99.9% accuracy while catching zero fraud. Key techniques:

• Precision-Recall AUC: The primary metric for imbalanced classification. Fraud systems optimize for high recall (catch the fraud) at acceptable precision (false positive rate).
• Class weights: Weight the minority (fraud) class more heavily in the loss function. class_weight='balanced' in sklearn automates this.
• SMOTE: Synthetic Minority Over-sampling Technique — generate synthetic fraud examples by interpolating in feature space. Useful for training, not evaluation.
• Threshold calibration: Adjust the decision threshold rather than using 0.5. A threshold of 0.2 catches more fraud but generates more false positives. Business cost analysis determines the optimal threshold.

Credit Scoring & Explainability

Credit scoring is the canonical ML-in-production problem. FICO scores have existed since 1989. Modern ML-based credit scorecards replace logistic regression with gradient-boosted trees or neural networks — but the regulatory requirements for explainability and fairness remain unchanged.

The Modern Credit Scorecard

Traditional scorecards used additive point systems — each factor contributes a fixed number of points. Modern ML scorecards use gradient-boosted trees (XGBoost, LightGBM) for better discrimination, but must remain explainable for compliance.

Key features in credit models:

• Payment history (35%): Late payments, delinquencies, bankruptcies.
• Utilization (30%): Revolving credit used / total available credit.
• Credit age (15%): Average age of accounts, age of oldest account.
• Mix (10%): Types of credit (installment, revolving, mortgage).
• Inquiries (10%): Recent hard credit pulls (each suggests new credit-seeking behavior).

SHAP-Powered Credit Decisions (Code Example 2)

SHAP provides theoretically grounded, per-feature attributions that satisfy the adverse action notice requirements of ECOA and Regulation B.

import shap
import xgboost as xgb
import pandas as pd
import matplotlib.pyplot as plt

# Credit scorecard with regulatory-required explanations
# Adverse action notices (USA: ECOA) require explaining adverse credit decisions

# Train gradient boosted credit model
features = ['annual_income', 'debt_to_income', 'credit_history_months',
            'num_late_payments', 'revolving_utilization', 'num_open_accounts',
            'employment_years', 'home_ownership_encoded']

model = xgb.XGBClassifier(n_estimators=200, max_depth=5, learning_rate=0.05,
                            eval_metric='auc', random_state=42)
model.fit(X_train, y_train)

# SHAP explanations — required for fair lending compliance
explainer = shap.TreeExplainer(model)
shap_values = explainer.shap_values(X_test)

# Generate adverse action notice for a declined application
def generate_adverse_action(applicant_features: pd.Series, shap_vals: np.ndarray) -> str:
    """Generate ECOA-compliant adverse action notice."""
    # Top 4 negative factors (those reducing creditworthiness)
    feature_impacts = list(zip(features, shap_vals))
    adverse_factors = sorted(feature_impacts, key=lambda x: x[1])[:4]  # most negative

    reasons = {
        'debt_to_income': 'Debt burden relative to income is too high',
        'num_late_payments': 'History of late payments on existing accounts',
        'revolving_utilization': 'High utilization of available revolving credit',
        'credit_history_months': 'Insufficient length of credit history'
    }

    notice = "CREDIT DECISION: Application Declined\nReasons for adverse action:\n"
    for factor, impact in adverse_factors:
        notice += f"• {reasons.get(factor, factor)} (impact: {impact:.3f})\n"
    return notice

print(generate_adverse_action(X_test.iloc[0], shap_values[0]))

Fair Lending: Disparate Impact Analysis

Algorithmic Trading Signals

Algorithmic trading uses ML to identify patterns in market data that predict short-term price movements. It is simultaneously one of the most lucrative and most challenging applications of ML — markets are highly efficient, adversarial, and non-stationary.

Technical Indicator Feature Engineering (Code Example 3)

The following function computes a comprehensive set of technical indicators used as ML features for predicting 5-day forward returns.

import pandas as pd
import numpy as np
from sklearn.preprocessing import StandardScaler

# Technical indicator-based ML trading signal
# Note: Past performance does not guarantee future results. For educational purposes only.

def compute_features(df: pd.DataFrame) -> pd.DataFrame:
    """Compute technical indicators as ML features."""
    df = df.copy()
    # Price-based features
    df['returns_1d'] = df['close'].pct_change(1)
    df['returns_5d'] = df['close'].pct_change(5)
    df['returns_20d'] = df['close'].pct_change(20)

    # Momentum
    df['rsi_14'] = compute_rsi(df['close'], 14)  # RSI oscillator
    df['macd'] = df['close'].ewm(span=12).mean() - df['close'].ewm(span=26).mean()
    df['macd_signal'] = df['macd'].ewm(span=9).mean()

    # Volatility
    df['atr_14'] = compute_atr(df, 14)  # Average True Range
    df['realized_vol_20d'] = df['returns_1d'].rolling(20).std() * np.sqrt(252)

    # Volume features
    df['volume_ratio'] = df['volume'] / df['volume'].rolling(20).mean()
    df['obv_change'] = df['close'].diff().apply(np.sign) * df['volume']

    return df.dropna()

# Predict 5-day forward return direction (binary classification)
# Feature importance: realized_vol (0.18), RSI (0.15), returns_5d (0.12), MACD (0.11)
# Backtest Sharpe: 0.82 (2015-2023) — execution costs and slippage can erode this significantly

The Backtesting Trap

Alternative Data in Finance

Beyond traditional market data (OHLCV, fundamentals), ML models increasingly incorporate alternative data:

• Satellite imagery: Counting cars in Walmart parking lots to predict earnings before announcement. Orbital Insight and RS Metrics pioneered this.
• Credit card transactions: Aggregated, anonymized spending data showing revenue trends before official earnings reports.
• NLP on earnings calls: Tone analysis of CEO language on earnings calls — negative tone predicts underperformance (Loughran-McDonald dictionary).
• Social media sentiment: Twitter/Reddit sentiment correlated with meme stock momentum. GameStop (2021) showed the limits of purely quantitative models.
• Job postings: Companies hiring AI engineers signal future AI investment; companies with high turnover signal operational problems.

Financial AI Applications Overview

The table below maps the major AI application areas in financial services to their technical requirements, regulatory oversight, and real-world implementations.

Model Risk Management: SR 11-7

SR 11-7 (Supervisory Guidance on Model Risk Management), issued by the Federal Reserve and OCC in 2011, is the foundational framework for financial model governance. Every bank with federal oversight must comply. It defines what a "model" is, how it must be validated, and what documentation is required.

The SR 11-7 Model Definition

A "model" under SR 11-7 is any quantitative method that applies statistical, economic, financial, or mathematical theories to transform inputs into outputs for decision-making. This includes:

• Credit scoring models
• Fraud detection systems
• AML (anti-money laundering) models
• Stress testing models
• Trading risk models (VaR, Greeks)
• Increasingly: LLM-based systems used in customer service or financial advice

Financial AI Model Risk Tiers

Regulatory Compliance

Financial AI operates at the intersection of multiple regulatory regimes simultaneously. A single credit model at a US bank must comply with ECOA, FCRA, fair lending regulations, SR 11-7, and (if the bank has EU customers) GDPR Article 22. Understanding the landscape is essential.

ECOA & Fair Lending

GDPR Article 22: Automated Decision-Making

For EU consumers, GDPR Article 22 grants the "right not to be subject to a decision based solely on automated processing" when that decision produces legal or similarly significant effects. For financial AI:

• Fully automated credit decisions trigger Article 22 obligations.
• Lenders must offer the ability to have a human review automated decisions.
• The data subject has the right to an explanation of the decision's logic and the right to contest it.
• The right to explanation under GDPR is more expansive than ECOA adverse action notices — it covers the logic of the model, not just the top factors.

CCPA & State Privacy Laws

The California Consumer Privacy Act (CCPA) and its successor CPRA grant California residents rights over their personal data used in financial AI:

• Right to know what data is collected and how it is used.
• Right to delete personal data (subject to financial recordkeeping requirements).
• Right to opt out of the "sale" of personal information.
• Right to non-discrimination for exercising these rights.

Exercises & Practice

Financial ML rewards both technical depth and regulatory literacy. These exercises build both alongside hands-on model development.