Causal Inference

Note

Explanatory Models

  • Explanatory modeling aims to understand relationships within data and explain why certain outcomes occur.

  • It focuses on identifying causal relationships and understanding the underlying mechanisms.

  • Methodology:

    • Models are evaluated based on their interpretability and how well they explain the relationships between variables.

    • Statistical significance of features and coefficients is often considered.

  • Techniques:

    • Linear models (linear regression, generalized linear models)

    • Structural equation modeling (SEM)

    • Bayesian networks, and

    • Certain types of decision trees can be used.

  • Feature Selection: Features are selected based on their ability to explain the phenomenon being studied. Domain knowledge and causal inference play a significant role.

Causal Inference

Course Plan

1. Causal Inference Basics:

  • Concepts: Understand fundamental concepts such as causality, counterfactuals, treatment effects, and causal graphs (Directed Acyclic Graphs - DAGs).

  • Techniques: Familiarize yourself with techniques like matching methods, instrumental variables, difference-in-differences, and regression discontinuity designs.

  • Papers

    • Instrumental Variables in Causal Inference and Machine Learning: A Survey

    • Matching Methods for Causal Inference: A Review and a Look Forward

    • Regression discontinuity design

Resources for Causal Inference:

  • Books:

    • “Causal Inference: The Mixtape” by Scott Cunningham (freely available online).

    • “Mostly Harmless Econometrics: An Empiricist’s Companion” by Joshua D. Angrist and Jörn-Steffen Pischke.

  • Courses:

    • Coursera offers courses like “Causal Inference” by University of Pennsylvania and “Econometrics: Methods and Applications” by Erasmus University Rotterdam.

    • edX has courses such as “Causal Diagrams: Draw Your Assumptions Before Your Conclusions” by Harvard University.

  • Online Resources:

    • Blogs and tutorials from platforms like Towards Data Science and Medium often have introductory articles on causal inference.

2. Advanced Regression Techniques:

  • Topics: Brush up on advanced regression methods that are commonly used for explanatory modeling, such as generalized linear models (GLMs), hierarchical linear models (HLMs), and Bayesian regression.

Resources for Advanced Regression:

  • Books:

    • “Bayesian Data Analysis” by Andrew Gelman et al.

    • “Regression Modeling Strategies” by Frank E. Harrell Jr.

  • Courses:

    • Platforms like Coursera and edX offer courses on Bayesian statistics and regression modeling.

3. Interpretable Machine Learning:

  • Techniques: Explore methods that enhance model interpretability, such as feature importance techniques (e.g., SHAP values, permutation importance), partial dependence plots, and model-agnostic approaches (e.g., LIME).

Resources for Interpretable Machine Learning:

  • Books:

    • “Interpretable Machine Learning” by Christoph Molnar.

  • Courses:

    • Look for courses on interpretable machine learning on platforms like Coursera or edX.

4. Domain Knowledge in Supply Chain and Inventory Management:

  • Understand: Review key concepts in inventory management, supply chain operations, and optimization techniques commonly used in e-commerce and retail.

Additional Tips:

  • Practice Problem Solving: Solve case studies or practice problems related to inventory management and causal analysis.

  • Mock Interviews: Conduct mock interviews with peers or mentors to practice explaining your approach to developing explanatory models.

Example Scenario to Explore:

  • Consider how you would design a study to determine the causal impact of a new inventory management policy on key performance metrics (e.g., cost efficiency, inventory turnover) using causal inference techniques.

Notes

In the context of causal inference, the Average Treatment Effect (ATE) and Individual Treatment Effect (ITE) are key concepts used to measure the impact of an intervention or treatment.

Average Treatment Effect (ATE)

The ATE measures the expected difference in outcomes between units that receive the treatment and those that do not, averaged over the entire population. Mathematically, it is expressed as:

\[\text{ATE} = \mathbb{E}[Y(1) - Y(0)]\]

where: - \(Y(1)\) is the potential outcome if the unit receives the treatment. - \(Y(0)\) is the potential outcome if the unit does not receive the treatment.

To calculate the ATE, we can use methods from the do-calculus framework when we have a causal graph. The do-calculus allows us to adjust for confounders and estimate the causal effect by “mutilating” the graph, which means removing the incoming edges into the treatment variable and setting it to a specific value (treated or untreated). Techniques such as Propensity Score Matching, Inverse Probability Weighting, and Regression Adjustment are commonly used to estimate ATE in practice.

Individual Treatment Effect (ITE)

The ITE measures the difference in outcomes for a specific individual when they receive the treatment versus when they do not. Mathematically, it is expressed as:

\[\text{ITE}_i = Y_i(1) - Y_i(0)\]

where: - \(Y_i(1)\) is the potential outcome for individual \(i\) if they receive the treatment. - \(Y_i(0)\) is the potential outcome for individual \(i\) if they do not receive the treatment.

Estimating ITE is more challenging because it involves counterfactual reasoning: for each individual, we need to know both what happened and what would have happened in the alternate scenario. This often requires generative models to simulate the counterfactual outcomes. Methods like Bayesian Additive Regression Trees (BART), Causal Forests, Gaussian Processes, and Deep Learning approaches (e.g., TARNet, CFRNet) are used to estimate ITE by modeling the outcome distribution under different treatment conditions.

Summary of Techniques

  • ATE Estimation Techniques: - Propensity Score Matching (PSM): Matches treated and untreated units with similar covariates. - Inverse Probability Weighting (IPW): Weights units by the inverse probability of receiving the treatment. - Regression Adjustment: Models the outcome as a function of treatment and covariates.

  • ITE Estimation Techniques: - Bayesian Additive Regression Trees (BART): Non-parametric Bayesian regression approach. - Causal Forests: An extension of random forests designed for causal inference. - Deep Learning Methods: Neural networks designed for estimating individual treatment effects (e.g., TARNet, CFRNet).

Note

  • ATE can indeed be obtained using techniques like do-calculus by manipulating the causal graph and using statistical methods to adjust for confounders.

  • ITE generally requires modeling counterfactual outcomes, often through generative models or advanced machine learning techniques to infer the individual-specific treatment effects.

However, both ATE and ITE can be estimated using various methods depending on the available data and the assumptions we can reasonably make. The choice of method often depends on the context, the quality of the data, and the underlying assumptions we are willing to accept.