PHD Discussions Logo

Ask, Learn and Accelerate in your PhD Research

Question Icon Post Your Answer

Question Icon

What major threats affect internal validity in non-equivalent groups pre-post designs, and which modern statistical methods can enhance causal inference in such quasi-experiments?

My educational intervention study uses existing, non-randomly formed classrooms as treatment and control groups, with pre-test and post-test scores. I'm concerned about selection bias, maturation, and regression to the mean confounding my results. I know ANCOVA on post-test scores using the pre-test as a covariate is common, but I'm looking for more modern, robust methods to approximate causal inference.

 

All Answers (1 Answers In All)

By Farah Answered 2 months ago

The core threat in your design is selection bias the groups likely differed before your intervention. While ANCOVA helps, I have seen it fail if the relationship between pre- and post-test isn't perfectly linear or homogeneous. I would recommend stronger modern approaches. First, consider Propensity Score Matching (PSM) to create a balanced synthetic control group from your pool of non-equivalent subjects. Alternatively, a Difference-in-Differences (DiD) model explicitly compares the change in each group, which can control for time-invariant group differences. For the strongest claim, use ANCOVA on a sample matched via PSM. Always test for parallel trends in a DiD framework.

Your Answer