Understanding driver behaviour is an essential component in human-centric Intelligent Driver Assistance Systems. Specifically, driver foot behaviour is an important factor in controlling the vehicle, though there have been very few research studies on analysing foot behaviour. While embedded pedal sensors may reveal some information about driver foot behaviour, using vision-based foot behaviour analysis has additional advantages. The foot movement before and after a pedal press can provide valuable information for better semantic understanding of driver behaviours, states, and styles. They can also be used to gain a time advantage in predicting a pedal press before it actually happens, which is very important for providing proper assistance to driver in time critical (e.g. safety related) situations. In this paper, we propose and develop a new vision based framework for driver foot behaviour analysis using optical flow based foot tracking and a Hidden Markov Model (HMM) based technique to characterize the temporal foot behaviour. In our experiment with a real-world driving testbed, we also use our trained HMM foot behaviour model for prediction of brake and acceleration pedal presses. The experimental results over different subjects provided high accuracy (~94% on average) for both foot behaviour state inference and pedal press prediction. By 133 ms before the actual press, ~74% of the pedal presses were predicted correctly. This shows the promise of applying this approach for real-world driver assistance systems. (Author/publisher)
Abstract