The Mediator system is an intelligent mediating system that enables safe switching between the human driver and automated system. The main aim of Work Package 3 (WP3) “Testing & Evaluation” within the MEDIATOR project was to evaluate the functionality of the Mediator system using both driving simulators and field studies with real drivers. This functionality evaluation includes assessing the system’s performance, reliability, and effects on driving safety, as well as the acceptance, trustworthiness, perceived safety, and user-friendliness of different groups of users. This deliverable is focused on the driving simulator studies conducted in the framework of WP3. Two studies were conducted at Ben-Gurion University of the Negev (BGU) and one at Chemnitz University of Technology (TUC). The main objectives of the driving simulator studies reported in this deliverable were:
- To evaluate the effects of preventive and corrective mediation in mitigating the adverse effects of passive fatigue and distraction on driving performance and hazard perception. Preventive and corrective mediation is a Mediator-related concept for preventing drivers from becoming fatigued or distracted or correcting drivers' performance if they are already in a state of fatigue or distraction.
- To evaluate the longer-term effects of the Mediator Human Machine Interface (HMI) concept by inviting participants for a second driving session a week after their first experience with the system.
- To study users’ evaluation of the Mediator system / HMI concept in comfort related use cases with regard to acceptance, trust, usability, comfort and transitions of control (TOC) while addressing driver characteristics.
Summary driving simulator studies on fatigue and distraction
The two driving simulator studies at BGU focused on Continuous Mediation (CM) automated driving conditions investigating drivers' states of fatigue and distraction. CM is similar to the SAE L2 definition (i.e., partially automated driving). In this driving condition, the driver must continuously monitor the road and the automation. One study focused on fatigue, and one on distraction. Both studies investigated the effectiveness of the Mediator HMI in mitigating the driver-state-related adverse effects in the short term and the long term (a week after the first experience with the system). An audio-visual Trivia game was used as a preventive mediation.
Main results showed that in the fatigue study, using an audio-visual Trivia game as a preventive mediation helped in maintaining situation awareness compared to drivers who were not playing Trivia, despite the fact that there were no fatigue-related differences in physiological and questionnaire data. As for the hazard notification system, it was found helpful in attracting drivers' attention (regardless of whether they played Trivia or not) to typical on-road hazards and improved safety. Concerning whether these positive effects persist over time, there was consistent evidence that the Mediator HMI was effective in both driving sessions of the fatigue study. Participants who experienced Trivia and hazard notifications as preventive mediation perceived the interface as more convenient, usable, acceptable, and safe than the Non-Trivia group in both drives. In addition, while for the Non-Trivia group, there were some questionnaire dimensions where their scoring decreased between the first and second driving sessions, for the Trivia group, it remained unchanged. Similarly, in the distraction study (where the Trivia game was absent), participants perceived the Mediator HMI as useful, safe, and trusted. Participants also indicated their willingness to adopt this type of system.
Summary driving simulator study on comfort-related use cases
The driving simulator study at Chemnitz University of Technology focussed on comfort use cases with related TOC, simulated automation degradation and related TOC by the human driver, as well as driver characteristics. The study aimed primarily on the user’s evaluation of the Mediator system / HMI (i.e. acceptance, trust, usability, comfort, experience of TOC), including the analysis of driver characteristics effects such as age, gender and driving experience. For this purpose, a balanced sample regarding age (19 to 75 years) and gender (49% female) was recruited. All participants experienced Mediator with its HMI elements in four different conditions. Next to CM, also the Driver Standby (SB) automated condition was tested which is comparable to SAE level 3 (conditional automation). Several standardized questionnaires, self-designed items and interview questions were used to examine drivers’ evaluation of Mediator for different use cases.
Main results revealed very high intention to use Mediator in future cars (88%) as well as very high driving comfort (90%), easiness to use (95%) and usability ratings (SUS score 85 out of 100) when driving with Mediator. Additionally, drivers’ expressed high acceptance (M = 1.21 on a scale ranging from -2 to +2) and trust (M = 5.5 on a scale ranging from 1 to 7). Further, 86% of the participants expected an increase in road safety when driving with Mediator. The analyses on driver characteristics did not show systematic differences with regard to gender and driving experience, and only small differences between age groups (i.e. more positive evaluations of older age groups) in the balanced sample. This gives a first indication that Mediator follows the “Design for All” principle, i.e. does not systematically discriminate certain groups. Driving with Mediator and in automated driving mode was preferred compared to manual driving without Mediator. In addition, driving in SB was preferred compared to driving in CM. Specifically, when participants experienced a traffic jam scenario under various levels of driving automation, they felt this was an appropriate situation where automation should take over and free the driver to engage with other tasks.
MEDIATOR has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 814735.
