Colloquium Losner Briones (UNED): The scope-sensitive approach: Assessing criticism of RCT-based evaluation in distributive policymaking

When:
16 October, 2026 @ 11:30 am – 1:30 pm
2026-10-16T11:30:00+02:00
2026-10-16T13:30:00+02:00
Abstract: 
 
Debates over randomised controlled trials (RCTs) in evidence-based policy often move from limitations of particular estimands, analytical strategies, or evidential standards to broader conclusions about what RCT-based evaluation can establish, which evidence it privileges, and which policy aims it can support. Yet the conditions under which such local methodological claims warrant these wider conclusions remain underexamined. This reflects a broader gap in philosophy of science, where policy implications are often inferred from epistemological analyses without comparable attention to the intervening processes through which evidence is produced, appraised, synthesised, and used.
 
This paper develops a scope-sensitive approach to methodological criticism that combines conceptual localisation with empirical grounding. It distinguishes evaluative objectives, the informational targets they require, their operationalisation, the experimental and analytical configurations through which information is produced, and subsequent processes of evidence appraisal and synthesis. Khosrowi and Reiss’s (2019) critique of RCT-based evaluation in distributive policymaking provides the argumentative test case. The Zambia Child Grant Programme provides the empirical case, with selected findings traced into Lwamba et al.’s (2022) systematic review and meta-analysis.
 
The analysis shows that limitations of particular estimands do not straightforwardly establish limitations of RCT-based evaluation; that documented RCT-based evaluations can generate richer distributive information through alternative operationalisations, heterogeneous effects, and longitudinal analysis; and that informational restrictions may arise downstream through selection, reclassification, aggregation, and appraisal. Broader methodological and policy conclusions therefore require locating limitations at the specific stage of evidential practice in which they arise.