The goal of the activities described in this deliverable, is to determine the Functional Requirements for the design of a Human Machine Interface (HMI) for vehicles that offer (partially) autonomous driving functionality. The MEDIATOR project is working towards a system that mediates, in real time, between the driver and the automated functions, ensuring that autonomous driving is always executed by combining the best of either’s performance. The strategy by which this is done is research-by-design i.e., HMI design projects facilitate the research into a number of knowledge gaps, that were determined in our initial literature studies.
Despite of the severe impact of the Covid19 pandemic and the research limitations because of that, we have been able to make pivotal steps in closing the knowledge gaps and establish a starting position for HMI design.
Functional requirements form the bases for the Design Requirements by which the final HMI will be designed. This holistic HMI will be tested and evaluated in driving simulators as well as in on-road tests. The scope of these research-by-design projects is determined by non-functional requirements, use-cases to construct all relevant driving scenarios, and design requirements to ensure HMI design with raison d’être.
In a preliminary study we investigated the Complexity of Mediation i.e., the role of Human, Automation and Mediator by enactment, in order to obtain an understanding of how a Mediator system should work. In this study, in an experimental set-up, participants were given the role of the human driver and the automation, each with its own world view, and that of the mediator. The decisions of a Mediator system are based on the different views of the world between a Human driver and the Automation. This study yielded that the decisions of a Mediator system are mostly conservative because of these different views of the world on which it has to base its decisions. In addition, the results show that knowledge over time builds up trust and influences a Mediator’s decisioning for future events.
Closing the knowledge gaps
The first knowledge gap Transfer of Control was researched in three studies from different perspectives; the control transfer from higher automation level to the driver, driver input towards automation preference, and the control transfer by means of specific potential technologies.
The first study on Transfer of Control introduces experiments for the transfer of control during a Time to Sleep (TtS) scenario within high automation. The experiments focus on the way of communication towards the driver during takeover transition in order to enhance the driver’s situation awareness. Literature research and the experiments revealed that different (design)guidelines per stage of the take-over process are required. A first HMI concept was designed to perform physical and digital experiments in which the driver is guided step-by-step through the stages of takeover by means of signals by A-pillar light strips and a head-up-display.
Results of the experiments are translated into functional requirements, in related to the stages of a takeover experience (before a wakeup call, during a wakeup call, before a takeover request, during a takeover request) in order to improve driver’s situation awareness. A key finding is the fact that in order to improve driver’s situation awareness, drivers need to be guided step-by-step through all stages of takeover (before a wakeup call, during a wakeup call, before a takeover request, during a takeover request).
The second study on Transfer of Control builds on the first. After a literature and design study, its scope was narrowed down to driver input i.e., how drivers are to express their preference towards an autonomous level, for the control transfer ritual. This shift of control can give control to the automation to relieve the human driver of some, if not all driving tasks and vice versa.
Literature and user research showed that, in order to assure a smooth control transfer ritual, important requirements are the simplification of automation levels, frequent feedback, and a balance between user autonomy and automation-initiated actions. After a study into the positioning of the HMI elements, a third HMI concept was developed, which distinguishes three driving modes (manual, assisted and piloted driving) to communicate Mediator’s four driving modes (conventional driving, Continuous Mediation CM, Driver Stand SB, and Time to Sleep TtS) to the driver.
Three concepts were tested by means of low-fidelity prototypes, after which a high-fidelity prototype of the chosen concept was built. The chosen concept, based on existing affordances, is a redesigned automatic gearbox lever, expanded with the three driving modes.
The third study on Transfer of Control addresses Control Transfers as a process during which a driver-automation system changes from one state to another involving reallocation of the longitudinal and lateral control task between the driver and the automation. The failure of effective communication regarding transitions such as take over request, takeover time, activated mode, time budget etc., could lead to safety-critical situations.
In this third study, a novel HMI interface (LED bar on steering wheel) was used to communicate transition related information to drivers. Two HMI concepts were made available, using the LED bar on a steering wheel, which were differed in color and illumination patterns. The two HMI concepts were compared with a baseline concept (without the LED bar on steering wheel) on subjective measures (trust, user experience and user acceptance). Results indicated that the two HMI concepts scored higher in all three metrics compared to baseline. Subjects also preferred to have the steering with LED bar for communicating transition related information.
The second knowledge gap concerns Transparency and Information Overload. One of the challenges in driving with higher levels of automation is to create mode awareness and appropriate reliance on the system. Transparency of the system is generally thought to improve both, as the driver can then understand the system and anticipate future system functioning. However, more transparency generally implies providing more information to the driver, which in turn can cause information overload. The research looks into this tradeoff between transparency and information overload, especially while driving with higher levels of automation. Literature research and several experiments, with different groups of participants were performed to provide insights into relevant types of information for the driver while driving with higher levels of automation.
The second HMI concept design in this research, conveyed specific information to the driver, as well as a subtle sense of the activated autonomous level by ambient lighting. The research concluded that the HMI should unobtrusively communicate time budgets such as minimum takeover time and the remaining time for which the current level of automation will be available, as well as information on reasons for automation fitness to change. The aim should be to create an ambience that reflects the current driver responsibility, which can also be perceived while NDRT’s are performed. For long term planning of NDRT’s also information on route progress and available automation levels along the route should be communicated. Finally, to improve the driver’s understanding of the system, the HMI should also communicate information on upcoming manoeuvres and automation perception, such as other road users and traffic signs.
Research into the knowledge gap Keeping the Driver in the Loop i.e., countermeasures for Inattention, Distraction and Fatigue, was done through extensive literature studies and design inventory of existing solutions, either in production or concept vehicles, and available technologies in both the automotive as the aviation domain. Although a set of functional requirements on countermeasures for HMI design, was derived from the research, some caution towards the conclusions about their effectiveness is in order. Most of the investigated studies were done under specific conditions and did not include user acceptance perspectives such as the driver intention to use the system, perceived usefulness and usability of the system as well as personal differences.
The recommendations from this research, for HMI design, consist of the adaptation of Mediator intervention to the dynamic situation of the triangle: driver, vehicle and context. An imperative condition is that the driver should understand the automation system, fully and intuitively. For the visual inputs, the HMI designer has to use appropriate and effective colours, referring to established techniques in graphical HMI. The frequency of the interaction and the number of modalities for intervention depend on the immediacy of the situation. Another principle to be considered is the content of the information that should encourage the driver to adopt a behaviour that may decrease the risk of accident.
The knowledge gap Negotiating conflicts i.e., when a human driver and the automation don’t agree on the preferred automation level, was researched though a literature study which included other mobility domains with suspected experience in the negotiation between human and machine. Furthermore, an extensive inventory of potential conflicts in each autonomous level was composed. A main conclusion is that there is no single reply to the full spread of potential conflicts.
Each holistic situation must be analysed and assessed, such that a driver feels comfortable and in control, regardless of location or task, which can be achieved through research and testing. Disagreements about the automation’s decisions will depend on the Human’s attitude to, experience with and trust in Automated Driving Systems. Mediator should be adaptable to different Human preferences, selected by different experience modes or levels. To meet the individual driver’s expectations to ADS, Mediator can be helpful in reducing potential conflicts. These findings, the aforementioned inventory on potential conflicts, and earlier ideas on HMI design for the negotiations between driver ad automation, frame our further research by the design of an HMI concept.
In conclusion, in this this report four out of five primary knowledge gaps have been researched. The fifth, OEM Design Space, can be addressed when the HMI design matures. Secondary knowledge gaps, Learning and Skill Degradation (unlearning) will also gain implicit attention in the further design process.
The collected functional requirements of the individual studies have been translated into one coherent set of functional requirements, through a number of cross checks, such as into the spread of investigated use-cases, and the identification of conflicting functional requirements.
In parallel with this process, additional HMI concept designs will further close the knowledge gaps, such as that of Negotiating Conflicts. Three HMI concepts have been designed, each in a number of redesign iterations. All HMI design concepts together, with the final set of functional requirements, translate into design requirements in an iterative process.
MEDIATOR has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 814735.