The field of experimental design has seen a flurry of exciting developments over recent years, with the introduction of advanced design techniques that aim to allow the analyst to collect data on which a greater level of robustness can be obtained with a limited sample size. However, the uptakeof these techniques in real world studies continues to be slow. In this paper, we argue that this is not solely a reflection of the usual gap between theory and practice, but that numerous technical and practical difficulties arise when attempting to deploy advanced experimental design techniques in large scale real world studies. As an illustration, this paper presents the findings from two separate case studies. In the first case study, we show that in some contexts, the use of efficient designs may in fact not produce any improvement in the robustness of model estimates when compared to the use of orthogonal approaches. In the second case study, we discuss how, in some cases, neither efficient nor orthogonal designs may be applicable and that better performance may in fact be obtained with a manual design.
Abstract