Author granted license

Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International

Document Type

Response or Comment

Publication Date

2023

ISSN

0006-8047

Publisher

Boston University School of Law

Language

en-US

Abstract

Professor Megan T. Stevenson’s Article, Cause, Effect, and the Structure of the Social World, is an incredibly important, deep, and thought-provoking argument explaining what we can learn about fundamental causal relationships when we observe few interventions with long-lasting, cascading consequences.1 It is a profound reflection on empirical work in the social sciences.

The Article argues that we have found few, if any, well-identified policy levers that generate outsized, long-term positive impacts for those impacted by the criminal legal system. It offers several explanations for the lack of randomized control trial (“RCT”) evaluations with large, non-mechanical effects, but the critical insight is that the social world is composed of stabilizing forces.” To make this argument, it begins with empirical work, documenting that hundreds of careful experiments have studied the criminal legal space. Stevenson argues that RCTs are highly credible research designs, and are the type of evidence we ought to trust the most to identify causal relationships. Relative to other forms of causal empirical work, RCTs are more difficult to manipulate and more likely to be published regardless of their findings. Despite several features making RCTs less biased than other designs, these experiments are still more likely to be published and well-known if they find outsized policy impacts. Yet, even with this potential bias, we see few RCTs generating large, long-lasting improvements with respect to many of the outcomes we care about. Those interventions that initially seem promising have difficulty replicating or scaling.

A. Some of the Major Contributions Within Cause, Effect, and the Structure of the Social World

The Article carefully demarcates the scope of the critique, and much of this Response will be spent discussing the boundaries of its argument. The empirical argument focuses on RCTs.2 RCTs often focus on relatively small-bore solutions. These interventions tend to be small because implementing an RCT often requires navigating normative and practical constraints restricting the scope of policies researchers can test.3 This Response explores the idea that the interventions we are willing to evaluate with an RCT are constrained by political will, ethics, time, costs, and many other factors, and what these constraints imply for the inferential argument. Constraints on RCTs in social science are common, but we may be exceptionally constrained within criminal legal interventions.4 Understanding which constraints are binding on our knowledge-generation process is essential for interpreting the Article’s evidence, its epistemic versus substantive critiques, and ultimately, our ability to improve criminal legal policy

Comments

An invited response to Megan T. Stevenson, Cause, Effect, and the Structure of the Social World, 103 B.U. L. REV. 2001 (2023). Available at source.

Link to Publisher Site

Share

COinS
 
 

To view the content in your browser, please download Adobe Reader or, alternately,
you may Download the file to your hard drive.

NOTE: The latest versions of Adobe Reader do not support viewing PDF files within Firefox on Mac OS and if you are using a modern (Intel) Mac, there is no official plugin for viewing PDF files within the browser window.