Document Type
Response or Comment
Publication Date
2023
ISSN
0006-8047
Publisher
Boston University School of Law
Language
en-US
Abstract
Professor Megan T. Stevenson’s Article, Cause, Effect, and the Structure of the Social World, is an incredibly important, deep, and thought-provoking argument explaining what we can learn about fundamental causal relationships when we observe few interventions with long-lasting, cascading consequences.1 It is a profound reflection on empirical work in the social sciences.
The Article argues that we have found few, if any, well-identified policy levers that generate outsized, long-term positive impacts for those impacted by the criminal legal system. It offers several explanations for the lack of randomized control trial (“RCT”) evaluations with large, non-mechanical effects, but the critical insight is that the social world is composed of stabilizing forces.” To make this argument, it begins with empirical work, documenting that hundreds of careful experiments have studied the criminal legal space. Stevenson argues that RCTs are highly credible research designs, and are the type of evidence we ought to trust the most to identify causal relationships. Relative to other forms of causal empirical work, RCTs are more difficult to manipulate and more likely to be published regardless of their findings. Despite several features making RCTs less biased than other designs, these experiments are still more likely to be published and well-known if they find outsized policy impacts. Yet, even with this potential bias, we see few RCTs generating large, long-lasting improvements with respect to many of the outcomes we care about. Those interventions that initially seem promising have difficulty replicating or scaling.
A. Some of the Major Contributions Within Cause, Effect, and the Structure of the Social World
The Article carefully demarcates the scope of the critique, and much of this Response will be spent discussing the boundaries of its argument. The empirical argument focuses on RCTs.2 RCTs often focus on relatively small-bore solutions. These interventions tend to be small because implementing an RCT often requires navigating normative and practical constraints restricting the scope of policies researchers can test.3 This Response explores the idea that the interventions we are willing to evaluate with an RCT are constrained by political will, ethics, time, costs, and many other factors, and what these constraints imply for the inferential argument. Constraints on RCTs in social science are common, but we may be exceptionally constrained within criminal legal interventions.4 Understanding which constraints are binding on our knowledge-generation process is essential for interpreting the Article’s evidence, its epistemic versus substantive critiques, and ultimately, our ability to improve criminal legal policy
Recommended Citation
Benjamin D. Pyle,
Knowledge Generation and Uncertainty in an Unpredictable Social World
,
in
103
Boston University Law Review
2049
(2023).
Available at:
https://scholarship.law.bu.edu/faculty_scholarship/3822
Comments
An invited response to Megan T. Stevenson, Cause, Effect, and the Structure of the Social World, 103 B.U. L. REV. 2001 (2023). Available at source.