DAY 052 / Counterfactuals / DiCE

Counterfactual Analysis: The 'What If' Engine

Counterfactuals

DiCE

Algorithmic Recourse

Ethics

Abstract

Explaining why a model made a decision (Day 51: Mechanistic Interpretability) is a diagnostic capability. Explaining how to change that decision is a product capability. Actionable Recourse Failure occurs when a system delivers a terminal negative outcome—loan denial, fraud flag, hiring rejection—without providing a feasible path to a positive outcome. This creates a "dead-end" user experience that breeds resentment and invites regulatory scrutiny (specifically GDPR Recital 71). This post details the engineering of a Counterfactual Engine: a system that solves the optimization problem of finding the smallest, valid, and actionable changes required to flip a model's prediction, transforming a hard "No" into a constructive "Not Yet."

1. Why This Topic Matters

In production systems, explanation is past-tense; recourse is future-tense.

For a user, knowing that "Account Age" was the top factor for rejection (SHAP) is intellectually interesting but practically useless. They cannot age their account backward. They need to know: "If I increase my cash balance by $500, will I be approved?"

Providing recourse is the difference between an AI that acts as a Gatekeeper (blocking access) and a Coach (guiding behavior).

Regulatory Pressure: The "Right to Explanation" is evolving into a "Right to Recourse." Algorithms that deny critical services without a path to remediation are increasingly viewed as discriminatory.
Business Value: A rejection is a churn event. A counterfactual ("You are close; just do X") is a retention mechanic.

2. Core Concepts & Mental Models

The Counterfactual

A conditional statement: "If feature X had been X' instead of X, the prediction would have been Y' instead of Y."

The Optimization Problem

We are not looking for any input that yields a positive result. We are looking for the input that is:

Valid: Actually flips the model's classification.
Proximal: Mathematically close to the original input (minimum edit distance).
Feasible: Respects real-world constraints (e.g., you cannot decrease your age).
Sparse: Changes as few features as possible (cognitive load constraint).

Mental Model: The Decision Boundary Map

Imagine the user is a point in high-dimensional space. The model is a dividing line (hyperplane).

SHAP tells you which direction the current point pushes you.
Counterfactual Analysis calculates the shortest vector to cross the line into the "Accept" region, while navigating around obstacles (immutable features).

3. Theoretical Foundations

We formalize recourse as a constrained optimization problem:

x* = argmin  d(x, x')
     subject to: f(x') = y_target
                 x' ∈ Feasible Set C

Where:

x is the original input.
x* is the counterfactual.
d(·) is a distance metric (usually L1 or weighted L1 to encourage sparsity).
f(x') = y_target ensures the new prediction matches the target class.
Constraints: x' ∈ C (Feasible Set).

The Feasibility Hard Constraint

The feasible set C is defined by domain logic:

Immutable: age, race, gender.
Monotonic: credit_score can only increase (usually).
Actionable: We cannot ask a user to change their "FICO Score" directly; we must ask them to change the inputs to that score (e.g., "Pay down debt").

4. Production-Grade Implementation

We will use DiCE (Diverse Counterfactual Explanations) or Alibi, but wrapped in a strict validation layer. Raw library outputs are often dangerous (e.g., suggesting "Change Gender").

Architecture: The Recourse Microservice

Input: User Feature Vector + Model Object.
Constraint Config: A YAML/JSON definition of mutable vs. immutable features and valid ranges.
Generator: Solves the optimization (Genetic Algorithms or K-d trees are preferred over Gradient Descent for non-differentiable features).
Filter: Post-processing to remove "silly" suggestions (e.g., "Increase income by $0.01").

The Trade-off: Optimality vs. Realism

Mathematical Optimality: "Increase savings by $12.53." This crosses the boundary exactly.
Human Realism: "Increase savings by $100."
Resolution: We must add a Safety Margin (or "Tolerance"). We don't want the user to land on the boundary (50.01% probability); we want them safely inside (60%+). We sacrifice mathematical proximity for robustness.

5. Hands-On Project / Exercise

Goal: Build a "Recourse Generator" for a loan approval model.

Constraint: Must handle immutable features and output human-readable advice.

Setup

We use dice-ml for the backend logic but wrap it to enforce business rules.

# pip install dice-ml scikit-learn pandas numpy
import dice_ml
from dice_ml.utils import helpers
import pandas as pd
import numpy as np
from sklearn.ensemble import RandomForestClassifier

# --- 1. The Production Constraints ---
# This is the most critical part: defining what CANNOT change.
FEATURE_CONFIG = {
    "outcome_name": "loan_approved",
    "continuous_features": ["income", "savings", "debt", "years_employed"],
    "categorical_features": ["rent_or_own"],
    "immutable_features": ["age", "years_employed"]  # User can't quickly change job tenure
}

# --- 2. Model Setup (Mock) ---
# In prod, load your serialized model
dataset = helpers.load_adult_income_dataset()
target = dataset["income"]  # Using adult dataset as proxy for loan
train_dataset = dataset.drop("income", axis=1)

model = RandomForestClassifier()
model.fit(train_dataset, target)

# --- 3. The Recourse Engine Class ---
class RecourseEngine:
    def __init__(self, model, train_data, config):
        self.model = model
        self.config = config

        # Initialize DiCE Data interface
        self.d = dice_ml.Data(
            dataframe=pd.concat([train_data, target], axis=1),
            continuous_features=config["continuous_features"],
            outcome_name=config["outcome_name"]
        )

        # Initialize DiCE Model interface
        self.m = dice_ml.Model(model=model, backend="sklearn")

        # Use random method (better for categorical/mixed constraints)
        self.exp = dice_ml.Dice(self.d, self.m, method="random")

    def generate_advice(self, query_instance):
        """
        Generates actionable changes to flip prediction from 0 to 1.
        """
        dice_exp = self.exp.generate_counterfactuals(
            query_instance,
            total_CFs=3,
            desired_class=1,
            features_to_vary=[
                f for f in (
                    self.config["continuous_features"] +
                    self.config["categorical_features"]
                )
                if f not in self.config["immutable_features"]
            ]
        )

        return self._parse_output(dice_exp, query_instance)

    def _parse_output(self, dice_exp, original):
        """Convert raw DiCE output to human-readable advice."""
        cfs = dice_exp.cf_examples_list[0].final_cfs_df
        advice_list = []

        for index, row in cfs.iterrows():
            changes = []
            for col in self.config["continuous_features"]:
                if col in self.config["immutable_features"]:
                    continue

                original_val = original.iloc[0][col]
                new_val = row[col]

                if abs(new_val - original_val) > 0.01:  # Filter noise
                    diff = new_val - original_val
                    direction = "Increase" if diff > 0 else "Decrease"
                    changes.append(
                        f"{direction} {col} by {abs(diff):.2f} to {new_val:.2f}"
                    )

            if changes:
                advice_list.append(changes)

        return advice_list

# --- 4. Execution ---
user_profile = train_dataset.iloc[0:1]
engine = RecourseEngine(model, train_dataset, FEATURE_CONFIG)

print(f"Current Prediction: {model.predict(user_profile)[0]}")
advice = engine.generate_advice(user_profile)

print("\n--- ACTIONABLE RECOURSE ---")
for i, option in enumerate(advice):
    print(f"Option {i+1}: {', '.join(option)}")

6. Ethical, Security & Safety Considerations

Goodhart's Law & Gaming

If we tell users exactly how to pass the model, they may optimize for the metric rather than the intent.

Risk: "Deposit $5,000 to pass the check." (User borrows$ 5k, deposits it, gets loan, returns $5k).
Mitigation: Use Robust Recourse. Ensure the suggested state remains valid even if perturbed slightly. Avoid suggesting changes to highly volatile features.

The "Impossible Promise"

Do not present counterfactuals as a legally binding guarantee unless your system is deterministic.

Risk: The model is retrained nightly. The user follows the advice ("Save $500"), comes back in a week, but the new model now requires$ 600.
Mitigation: Disclaimer text is vital. "Based on current criteria..." or version-lock the logic for users in a recourse flow.

7. Business & Strategic Implications

Support Cost Reduction: Detailed recourse reduces the load on Level 1 customer support. Instead of "Computer says no," the system answers the follow-up question automatically.
Conversion Optimization: In marketing funnels, counterfactuals can be used to upsell. "You don't qualify for Gold, but if you add a co-signer, you do."
Fairness Debugging: If the only counterfactuals for a specific demographic involve impossible changes (e.g., "Move to a zip code 500 miles away"), you have identified a structural barrier in your model that needs review.

8. Common Pitfalls & Misconceptions

Pitfall: Ignoring Causal Relationships.
- Reality: The model might say "Increase total credit limit," but the user can't do that without applying for another card, which lowers their score. DiCE handles features independently unless causal constraints are added.
Pitfall: Suggesting multiple conflicting changes.
- Reality: "Increase savings by $500 AND pay down debt by$ 500." The user might only have $500 total. Prioritize sparsity (fewest changes) over smallest Euclidean distance.
Pitfall: Floating Point Absurdity.
- Reality: "Increase age to 34.5." Ensure categorical/integer casting matches the domain reality.

9. Prerequisites & Next Steps

Prerequisites:

A deployed classification model.
Access to the training data distribution (needed to generate valid counterfactuals, not just random noise).

Next Steps:

Define Constraints: Map every feature in your model to Mutable, Immutable, or Conditionally Mutable.
Pilot: Run the Recourse Engine on historical rejected data. Do the suggestions make sense to a human reviewer?
UI Integration: Display these options in a non-promissory way ("Potential paths forward").

Counterfactuals tell users what to change, but they still trust the model's stated reasoning. The next question is: can we trust that reasoning at all? Day 53: Faithful Explainability: Probing & Lie Detection answers that by looking inside the model's activations to verify whether its explanations match its internal computations.

10. Further Reading & Resources

Library: Microsoft/DiCE – Diverse Counterfactual Explanations.
Library: SeldonIO/alibi – Algorithms for inspection and explanation.
Paper: "Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR" (Wachter et al., 2017).
Concept: Visualizing how minimum edit distance finds the path across the decision boundary.