Adding visual contextual information to continuous sonification feedback about low-reliability situations in conditionally automated driving: A driving simulator study

In conditionally automated driving, control transitions where drivers are asked to take over manual control quickly are a known challenge as drivers are likely “out-of-the-loop” due to automated driving. We previously demonstrated a useful sonification feedback design to support drivers for takeover events by manipulating background music to convey real-time changes in the automation reliability level. The present research examines two types of auxiliary information—explanations about the potential sources for reduced reliability and recommended actions to prepare for a potential takeover event—to communicate event-specific context about the low-reliability situations indicated by the sonification feedback. In a driving simulator with SAE level 3 automation, forty participants were assigned to one of four interface conditions: (i) sonification feedback only, (ii) sonification feedback and visual auxiliary information providing explanations, (iii) sonification feedback and visual auxiliary information providing action recommendations, or (iv) sonification feedback and both explanatory and action-recommending auxiliary information. Participants experienced six low-reliability events that belonged to three types of situations that may eventually lead to a takeover event: operational problems, system limitations, and unexpected events. Two events became system-requested takeover events. Our results showed that auxiliary information impacted visual information processing but not the takeover decisions or response times to takeover requests. The situation type—characterized by the availability of cues and presence of hazards in the environment—also played a significant role in both participants’ takeover decisions and visual information processing behaviors. These findings have implications for designing effective feedback for not only Level 3 automation, which we studied, but also for Level 4 dual-mode vehicles that could allow transitions to manual driving when exiting their operational design domains and for Level 2 automation, where drivers may need more support to monitor the road effectively and make more timely takeover decisions.