The Traffic Injury Research Foundation (TIRF) has conducted an objective and independent review of teenSMART, a product of ADEPT Driver. The purpose of this review was to assess the veracity and strength of the available evidence that this training program reduced teen driver crashes, which is a stated objective and claim of the program. As part of the review of teenSMART, and to provide context for it, TIRF explored the published scientific literature on the effectiveness of driver education and post-driver licensing training programs. Previous evaluations of beginner driver education programs and post-license driver training programs for newly licensed drivers suggested that such programs either had no effect, a negative effect or, at best, a marginally positive effect. Recent evaluations suggested it is possible that driver education is effective but the effect size is likely small (i.e., approximately 4%). This critical review of teenSMART has also established the following: * teenSMART is a computer-based training program consisting of three core elements: computer-based driving tutorials; parent-teen activities including in-car driving sessions; and a certification test at the end of the program. * teenSMART has face validity and content (logical) validity as a collision reduction program based on expert knowledge, judgment and consensus. * teenSMART has been shown to have criterion-related validity (i.e., meeting its safety objectives) in terms of statistically significant improvements in knowledge, driving-related skills, and on-road driving performance. These are encouraging results from a developmental evaluation of teenSMART, which has recently been confirmed by internal research showing that teenSMART is effective in improving the specific driving skills taught in the program including judging gaps, hazard detection, risk assessment, anticipating trouble, and visual search. * teenSMART has been shown to have criterion-related validity based on the indirect, anecdotal evidence and the practices of six insurance companies that offered an insurance discount for teenSMART and Departments of Insurance in most states that approved the rate filings by insurance companies for the discount. This suggested that these insurance companies had solid evidence convincing them that teenSMART reduced their claims experiences, i.e., teenSMART graduates have statistically lower collision rates. And in this regard, three insurance companies (CSAA Insurance Group, a AAA Insurer; Liberty Mutual; and Allstate) have recently spoken publicly to espouse the safety benefits of teenSMART. As well, the National Safety Council and the Governors Highway Safety Association have both recently awarded teenSMART prestigious safety awards. * teenSMART may also have criterion-related validity based on the results of several studies showing that teen drivers exposed to teenSMART had statistically significant lower collision rates than those not exposed to teenSMART. However, these results were not conclusive because other factors related to self-selection of teen drivers who completed teenSMART (e.g., greater safety consciousness) may have also contributed to, or accounted for, their lower collision rates. This critical review supported the claim that teen drivers who completed teenSMART had lower collision rates than those who did not. Indeed, teenSMART has an impressive track record of review and assessment by insurance companies over the past 16 years that attests to its safety benefits. The fact that insurance companies offer discounts and made public statements, as well as supported those of ADEPT Driver, about teenSMART significantly reducing collisions is remarkably telling about the safety performance of teenSMART within an insurance environment. Considering all the available evidence, it is possible that teenSMART does in fact have a safety effect which may range from as low as 4%, based on recent evidence about the impact of driver education, to 28%, based on insurance claims data and discount practices. These estimates provide a range of potential safety benefits that have been used in a cost-benefit analysis. Results using estimates from the National Highway Traffic Safety Administration (NHTSA) of the economic and societal costs of collisions suggested that: * an expenditure of $57 million for 16 year old drivers to complete teenSMART would potentially result in $98.9 million economic savings and $374 million societal savings assuming a training effect of 4% (the equivalent of 1.7 dollars saved for each dollar spent based on economic cost estimates; and 6.5 dollars saved for each dollar spent based on societal cost estimates); * if teenSMART is associated with a 28% reduction in collisions, the economic and societal benefit/cost ratios for drivers age 16 would potentially be 12.1 dollars saved for each dollar spent and 45.8 dollars saved for each dollar spent; and, * substantial economic and societal cost savings would also be potentially realized for drivers age 17, 18, 19, and 20 who complete teenSMART. Caution should be taken in interpreting the results of this cost-benefit analysis, principally because it is based on the assumption that teenSMART has a training effect but, it is still possible that other factors may account for some, or all, of these collision reductions. The proprietary nature of insurance algorithms makes it difficult to substantiate whether teenSMART training actually reduced teen driver collisions. However, it can be argued that current insurance discount practices for teenSMART speak for themselves given that the insurance industry is a for-profit enterprise and it would not be sustainable to provide larger discounts for programs that do not result in larger savings. (Author/publisher)
Abstract