sciepub.com
Quick Submission
Driver Attention Based on Eye-movement and Time-series Analysis - Concept of Driver State Detection Devices
Toshiya Arakawa
Abstract
In this study, the transition of drivers’ states of attention from “excessive attention state” to “unfocused attention state” because of increasing time, existence of a leading vehicle, adaptability to driving conditions in different areas, such as urban or suburban environments, and decreasing tension was examined from the viewpoint of the drivers’ eye movements. The variability of the driver’s eye gazing was observed to gradually increase when a leading vehicle was present, i.e., the driver's process resource was concentrated on the leading vehicle at the beginning of the driving but on both leading vehicle and surrounding conditions before the end of driving. In addition, it was found that the difference between the unfocused attention state and excessive attention state was shown by the change in eye movement, depending on whether drivers were driving in urban or suburban areas. This shows that the driving conditions induced changes in the distribution of process resource.
Keywords: driver attention state, eye gazing, eye movement, driving condition, process resource
Copyright © 2017 Science and Education Publishing. All Rights Reserved.Cite this article:
- Toshiya Arakawa. Driver Attention Based on Eye-movement and Time-series Analysis - Concept of Driver State Detection Devices. American Journal of Mechanical Engineering. Vol. 5, No. 1, 2017, pp 18-23. https://pubs.sciepub.com/ajme/5/1/4
- Arakawa, Toshiya. "Driver Attention Based on Eye-movement and Time-series Analysis - Concept of Driver State Detection Devices." American Journal of Mechanical Engineering 5.1 (2017): 18-23.
- Arakawa, T. (2017). Driver Attention Based on Eye-movement and Time-series Analysis - Concept of Driver State Detection Devices. American Journal of Mechanical Engineering, 5(1), 18-23.
- Arakawa, Toshiya. "Driver Attention Based on Eye-movement and Time-series Analysis - Concept of Driver State Detection Devices." American Journal of Mechanical Engineering 5, no. 1 (2017): 18-23.
| Import into BibTeX | Import into EndNote | Import into RefMan | Import into RefWorks |
At a glance: Figures
1. Introduction
Traffic-related accidents have been reported to have decreased in recent years; however, the accidents caused by “unfocused driving,” remain a major problem that cannot be overlooked despite the development and production of advanced driver assistance system. For example, one report stated that unfocused driving, i.e., when the driver is either distracted or falls asleep at the wheel, is the most significant factor contributing to fatal traffic crashes (661 cases), more than “inattentive driving” (459 cases) and “speeding” (212 cases) [1]. Thus, “unfocused attention” is often considered undesirable for drivers.
When individuals continue driving for a long time, their attention span tends to decrease to a certain extent as they become accustomed to their surrounding conditions. This type of unfocused attention is considered to be desirable because it allows the drivers to give sufficient attention to objects in their surroundings compared with the situation when excessive attention is given by drivers with the objective of driving safely. Thus, the unfocused attention state can be considered to have two aspects; one that is appropriate for driving and another that is not. The second one can lead to a drowsy state because of decreasing tension.
These states are expressed using a model, as shown in Figure 1. The left block shows the excessive attention state, wherein the driver is able to obtain all required information such as traffic lights, signs, and traffic flow. The central block shows the unfocused attention state, wherein the driver rationalizes his driving and after becoming accustomed to the surrounding conditions, tends to obtain the minimum information required for driving. The right block shows the “low arousal state,” which is caused by reduced tension; hence, is inadequate for driving purposes.
Download as
A driver visually obtains information from his surroundings and then processes it to control the vehicle. This is called “process resource,” [2] and it has also been reported in previous studies on this topic [3, 4]. Furthermore, it has been stated that this process resource varies with the skill, attention, and effort of the driver and can be affected by a number of other factors despite being examined for the same driver [5, 6]. It has been reported that there is a relation between eye-movement (which is the main method for visual input) and attention [7]. In this study, smooth pursuit and saccadic movements are used as components of eye movement. The smooth-pursuit eye movement occurs when the visual target moves smoothly and slowly with a maximum speed of approximately 30 deg/s. The characteristic of this type of movement is that drivers can perceive information without having to maintain a constant focus on the target object. In contrast, the saccadic movement occurs when the visual target is moving quickly with a maximum speed of approximately 700 deg/s, thus making it difficult for the driver to perceive the target. Therefore, both smooth pursuit and saccadic movements are alternately repeated to effectively perceive the given visual targets.
As reported by the study in reference [7], the occurrence of smooth-pursuit component decreases and that of saccadic movement increases with decrease in the attention level of a driver. The unfocused attention state of a driver is shown by the level of a driver’s eye gazing, determined by the change in the driver’s eyeball rotation. Moreover, the variability of the level of eye gazing is shown by its standard deviation. Certain studies have also reported that this unfocused state is especially notable when drivers are following a leading vehicle and before the end of driving [8, 9, 10]. On the basis of these reports, the distribution of a driver’s process resource can be considered to change during driving and become lower in an input-to-output system. Therefore, eye movement can be considered as an indicator to examine the difference between the excessive attention state and unfocused attention state.
Fatigue is also one of considerable human factors on driving. Fatigue is insufficiently recognized and reported as a cause of road accident [11]. Previous study have found that an important percentage of drivers did not think about the fact that fatigue has a negagive influence on driving, compared to what they thought about phychoactive drugs, drowsiness, and drug use [12]. On the other hand, previous study have found that a long-distance driving causes fatigue, and HRV and eye movement shows this tendency [13, 14]. Thus, especially, eye movement also can be considered as an indicator of fatigue, which seems to be one of unfocused attention state. If driver state detection devices by eye movement is developed, it can be helpful for drivers for preventing from unfocused driving affecting to fatigue or drowsiness and provide safety driving to drivers.
This study examines whether the variability of eye gazing and the eye-movement components can be used as indicators to examine driver attention state during traffic-related situations, such as sudden appearance of a leading vehicle and changes in driving conditions. In addition that, from the results of this study, it is considered that the manner of collect and integrate information from the vehicle and the driving conditions, in other words, the concept of driver state detection devices.
2. Experiment
This experiment was conducted with five subjects who were explained about the safety measures of the experiment and asked to sign an experimental agreement. To examine the subjects’ optical systems, an “Eye-mark Recorder” manufactured by Nac Image Technology Inc., was used to measure the eye gazing level and electrodes were used for measuring the eye movement. The test subjects were asked to drive along a 35 km long route and then return via a 29 km long route. The subjects stopped in between at Sano city.
This route is shown in Figure 2, wherein Ashikaga city is defined as an urban area and the region between Ashikaga city and Sano city is defined as a suburban area.
Download as
The vehicle used for the experiment was equipped with a driver-support system with a stereo camera and milliwave radar [10, 15]. This system could judge the presence of a leading vehicle and obtain the relative distance from it by simulating an image of the front portion of the experimental vehicle and the leading vehicle. The vehicle was able to measure the controlled area network (CAN) data as well. Fig. 3 shows the experimental system and Fig. 4 shows an image of the experimental situation. In Fig. 4, the upper left picture is the view from the “Eye-mark Recorder,” lower left picture is a front image from the stereo camera, and upper right picture is the face of one of the subjects.
Download as
Download as
3. Analysis
3.1. Treatment of Eye Movement DataThe light from the surroundings while conducting the experiment had a significant impact on the data obtained from the eye-mark recorder; therefore, it was not feasible to obtain data for only in-vehicle conditions. In contrast, the electrooculography (EOG) data 
, obtained in units of voltage, was not influenced by the outside light; therefore, it was considered that a relation existed between the data obtained by the eye-mark recorder and that obtained by the EOG. The eye-mark recorder data was converted to EOG data using the following equation:
 | (1) |
where 
[deg] is the corresponding value obtained by the eye-mark recorder, and a and b denote the coefficients obtained using least square method.
The correlation value, R, obtained after conversion was in the range of 0.4–0.9. Two subjects with correlation values lower than 0.6 were exempted from the analysis.
This converted value included the noise caused by the vibration of the vehicle. A 30 Hz low-pass filter 
was used to remove this noise. 
was then obtained by applying curve fitting to 
and expressed as follows:
 | (2) |
Here, 
, 
, and 
are coefficients.
In addition, a moving average (50 points) was applied to 
to obtain 
, which is the visible-degree value. The standard deviation value was calculated to show the variability of eye gazing.
Next, considering the sampling rate of EOG data [1 kHz], 
, expressed by equation (3) below, is defined using the velocity of eye movement [deg/s], 
as follows:
 | (3) |
In equation (3), 0.001 indicates the sampling time of CAN. Using 
, the rate of eye movement was categorized into three types as shown below, and the time for which each movement existed was also calculated.
• eye gaze (G): G < 3 deg/s;
• slow eye movement (S) (involving smooth-pursuit movement): 3 deg/s ≦ S <30 deg/s;
• fast eye movement (F) (involving saccadic movement): 30 deg/s ≦ F.
In addition, eye-gaze-occurrence frequency (G/(G+S+F)), slow-eye-moment-occurrence frequency (S / (G + S + F)), and fast-eye-movement-occurrence frequency (F/(G+S+F)) were obtained from the calculated existence time for different driving intervals, as described in the next subsection. Thus, it can be considered that the time-series behavior of variability of eye movement changes with changing driving conditions. This behavioral value, 
, was calculated as follows:
 | (4) |
where 
is the driving interval 
and 
is the standard deviation of eye-gazing distance during the ith driving interval. 
is defined as the average amplitude of variability of eye gazing.
To examine the driver’s attention state between intersections, the driving interval was categorized into “areas between intersections” and “intersections.” The velocity of each vehicle (V(t) [km/h]), steer angle (
[deg]), and winker (W(t) (0(off) or 1(on))) obtained from the CAN data were used as indicators in this categorization.
<Intersections>
Start point of the cut out: value of t when W(t) =1 (=t1);
Finish point of the cut out: value of t when 
for the first time after the 2nd extremal value following t1 (Figure 5).
The time-series described above 
were defined as those for intersections.
<Between intersections>
[When the last driving interval was an intersection]
Start point of the cut out: value of t1 calculated as t, which is start point of cut out at the intersection (=t1’);
Finish point of the cut out: value of t when v(t)=0 or W(t)=1 (=t2’).
The time-series described above (t1’≦t≦t2’) were defined as those for between intersections.
[In the case when the last driving interval was a straight pathway]
Start point of the cut out: value of t when v(t)>0 (=t1’’);
Finish point of the cut out: value of t when v(t)=0 or W(t) =1 (=t2’’);
Time-series described above (t1’’≦t≦t2’’) were defined as those for straight pathways.
In addition, the “with leading vehicle” condition was defined when the stereo camera identified that the experimental vehicle was following another one and the “without leading vehicle” condition was defined when the vehicle was found to not be following another one between intersections.
Download as
4. Results
4.1. Comparing the Beginning of the Driving Period with the Time before It EndsPrevious studies [8, 9, 10] showed a comparison between the variability of eye gazing during the first five minutes of driving and that during the last 5 minutes of driving, irrespective of the presence of a leading vehicle. However, driving conditions change with time; therefore, the standard deviation of the visible degree for the “with leading vehicle” condition at the beginning of driving (Start) and that before the end of driving (Finish) were compared. The results are shown in Fig. 6 and it can be observed that the standard deviation of visible degree at the Finish was larger than that at the Start.
4.2. Comparing the Differences in Driving ConditionsThe standard deviation of visible degree and the eye-movement components were analyzed for different driving conditions. Figure 7 and Figure 8 show the results of analysis. It can be observed that the standard deviation of visible degree and eye-gaze-occurrence frequency were larger in the urban area than those in the suburban area.
Download as
Download as
Download as
Under the condition with a leading vehicle, slow- and fast-eye-movement-occurrence frequencies were separately examined for the outward and homeward directions. Figure 9 and Figure 10 show the results of the slow-eye-movement-occurrence frequency and fast-eye-movement-occurrence frequency, respectively. The corresponding diagrams for the condition with no leading vehicle can be seen in Figure 11 and Figure 12, respectively.
Download as
Download as
Download as
Download as
From Figure 9 and Figure 10, in the presence of a leading vehicle, the slow-eye-movement-occurrence frequency was found to increase and fast-eye-movement-occurrence frequency was found to decrease during the outward journey; whereas opposite behaviors were observed during the homeward journey. From Figure 12, without the presence of a leading vehicle, the fast-eye-movement-occurrence frequency decreases in the outward direction and decreases in the homeward direction.
5. Discussion
The eye-gazing variability between Start and Finish was found to become larger with increase in time, corroborating the results of previous studies [8, 9, 10]. This indicates that the process resource required by drivers is concentrated on a leading vehicle at the Start but distributed to the surrounding conditions over time toward Finish. However, a smaller variability was shown for one of the subjects at the Finish than at the Start, indicating that this driver may have been in the unfocused attention state, which is near to the low arousal state shown in Figure 1. Without a leading vehicle, the fast-eye-movement-occurrence frequency decreased in the outward direction but increased in the homeward direction, indicating that the subjects maintained their levels of attention to check the state of the leading vehicle in the homeward direction but could not maintain the same in the outward direction. With a leading vehicle, the slow-eye-movement-occurrence frequency increased in the outward direction but decreased in the homeward direction indicating that the attention levels increased in the outward direction but decreased in the homeward direction for this condition, i.e., on the journey homeward, the driver’s state was more unfocused than that on the outward journey.
We now discuss the change in the driver’s attention state for different driving conditions in terms of both variability of eye gazing and eye-gaze-occurrence frequency. In the condition with a leading vehicle, both parameters are larger in urban areas than in suburban areas. These results were caused by driver behavior such as paying attention to leading, appearing, and parking vehicles. In contrast, in the condition without a leading vehicle, both parameters are larger in the suburban areas than in the urban area. These results indicate that there are few parked vehicles in the suburban areas or that vehicles do not appear frequently from side roads and the drivers can see the other visual targets as they are not following a leading vehicle. Thus, it can be stated that the driving conditions can result in the division of the process resource required for driving appropriately. The average amplitude of variability of eye gazing was smaller in the urban area than in the suburban area. This is contradictory to a previous studies [16, 17]. These study have found that driver tend to look various objects, such as leading vehicle, traffic sign and so on, thus variability of eye gazing is large in the urban. The reason of difference seems to be due to the driving environment. There are various not many objects besides leading vehicle in this experiment, thus, the average amplitude of variability of eye gazing might be small in the urban area.
To conclude, drivers change the object of their vision depending on the situation, recognize targets that directly affect driving, and perceive those objects that do not affect driving if there is high traffic or significant change in the driving conditions. In the suburban areas, these recognition and perception behaviors are mixed because those are very small. This mixed situation can be considered to cause an unfocused attention state. This result indicates that the excessive attention- and unfocused attention- states can be categorized more minutely by considering the variability of eye gazing and the ratio of the eye-movement components. Moreover, the existence of a leading vehicle or different driving conditions can affect the distribution of driver’s process resource and cause a transition from excessive attention state to unfocused attention state.
6. Conclusion
The transition from an excessive attention state to an unfocused one was examined by studying the variability of eye gazing and the ratio of eye-movement components. First, the variability of eye gazing was found to be large when drivers followed leading vehicles, indicating that the driver’s information-processing resource was focused on a leading vehicle at the start of driving but the attention was divided between the leading vehicle and the surroundings later. Next, the drivers were observed to be unfocused in the suburban areas when the drivers’ attention states between the urban and suburban areas were compared on the basis of the average amplitude of eye-gazing variability.
Further research will be required for constructing detection devices for driver attention states and driving conditions to build an active-safety system. These devices should be able to collect and integrate information from the vehicle and the driving conditions. This research can be conducted in the following manner.
First, when drivers are found to be in the unfocused attention state by the detection devices when following a leading vehicle, the detection devices for driving conditions will improve their sensitivity and inform drivers if they are in an emergency situation. Second, when drivers are found to be in the excessive attention state by the detection devices when following a leading vehicle, the detection devices for driving conditions will improve their sensitivity and inform drivers when they are in an emergency situation. For example, odor presenting device can be effective these emergency situation [18].
Third, when drivers are found to be unfocused, the detection devices for driving conditions will improve their sensitivity and inform them when a leading vehicle appears and there is an emergency.
Finally, when drivers are detected to be driving in the suburban areas, the detection devices for driving conditions will regularly inform the driver because they are in an unfocused attention state.
As stated above, integration devices for detection of driver attention states and driving conditions will enable the construction of suitable information systems that complement drivers’ requirements.
References
| [1] | Cabinet Office, Government of Japan. White Paper on Traffic Safety in Japan 2015, Available from Internet: https://www8.cao.go.jp/koutu/taisaku/h27kou_haku/zenbun/index.html (accessed on 24 September, 2016). | ||
 In article | |||
| [2] | Yee, P.L., Hunt, E. and Pellegrino, J.W., “Coordinating cognitive information: task effects and individual differences in integrating information from several sources,” Cognitive Psychology, 23(4). 615-680. 1991. | ||
| In article | View Article | ||
| [3] | Miller, J.C. “Batch processing of 10,000h of truck driver EEG data,” Biological Psychology, 40. 1-2. 1995. | ||
| In article | View Article | ||
| [4] | Desmond, P.A. and Matthews, G. “Implications of task-induced fatigue effects for invehicle countermeasures to driver fatigue,” Accident Analysis and Prevention, 29(4). 515-523. 1997. | ||
| In article | View Article | ||
| [5] | Kahneman, D. Attention and Effort, Prentice-Hall, 1973. | ||
| In article | |||
| [6] | Wickens, C.D. Engineering Psychology and Human Performance, Psychology Press, 2012. | ||
| In article | |||
| [7] | Ebisawa, Y., Minamitani, H. and Takase, M. “Relationship between Attention and Characteristics of Eye Tracking Movement – Estimation of Attention Level –,” Japanese Journal of Medical Electronics and Biological Engineering, 30(1). 22-31. 1992. | ||
| In article | |||
| [8] | Arakawa, T., Matsuo, N. and Kinoshita, M. “A Trial Evaluation on Desultory Driver Situation by Gaze Analysis,” proceedings of JASE Annual Congress(Spring). 5-8. 2005. | ||
| In article | View Article | ||
| [9] | Arakawa, T. and Matsuo, N. “The Time-series Fluctuation of Driver Attention -Estimation from the specification of Eye Movement-,“ proceedings of FISITA 2006. 1-10. 2006. | ||
| In article | |||
| [10] | Arakawa, T., Matsuo, N. and Kinoshita, M. “Trial Evaluation on Driver’s Unfocused Attention using Gaze Analysis,” Review of Automotive Engineering, 27(2). 357-359. 2006. | ||
| In article | |||
| [11] | Brown, D. “Driver Fatigue,” Human Factors: The Journal of the Human Factors and Ergonomics Society, 36. 298-314. 1994. | ||
| In article | |||
| [12] | Alonso, F., Esteban, C., Useche, S.A. and López de Cózar, E. “Prevalence of Physical and Mental Fatigue Symptoms on Spanish Drivers and Its Incidence on Driving Safety,” Advances in Psychology and Neuroscience, 1(2). 10-18. 2016. | ||
| In article | |||
| [13] | Sugie, R., Arakawa, T. and Kozuka, K. “Detection of Fatigue in Long-Distance Driving by Heart Rate Variability,” ICIC Express Letters, 10(7). 1553-1559, 2016. | ||
| In article | |||
| [14] | Eriksson, M and Papanikolopoulos, N.P. “Driver fatigue: a vision-based approach to automatic diagnosis,” Transportation Research Part C: Emerging Technologies, 9. 399-413. 2001. | ||
| In article | View Article | ||
| [15] | Takahashi, Y., Touda, N. and Sogawa, Y. “Development of ADA (Active Driving Assist) Ⅲ,” Subaru Technical Review, 31. 139-144. 2004. | ||
| In article | |||
| [16] | Yamada, M. and Fukuda, T. “Difinition of Viewing Point and Application to Image Analysis,” The IEICE Transactions on Information and Systems, 69(9). 1335-1342. 1986. | ||
| In article | |||
| [17] | Sakata, M. “One’s Eyes are Often More Eloquent than One’s Mouth: Study Cases on Eye Movement,” Kobe University Repository, 6(1). 103-116. 2006. | ||
| In article | |||
| [18] | Arakawa, T., Kobayashi, M. and Matsuo, N. “Development of Aromatic Image Map and Evaluation of Aromatic Oil Based on Brain Wave,” Journal of Biosciences and Medicines, 3(5). 36-48. 2015. | ||
| In article | View Article | ||
CiteULike
Mendeley
StumbleUpon
Delicious
Facebook
Twitter
LinkedIn
){kind=link}
){kind=link}
){kind=link}
){kind=link}
){kind=link}
