Eye Tracking for the Aviation Industry from Smart Eye: A Technical Breakdown

As an introduction to remote eye tracking and what differentiates this method from the others on the marketplace, the main objective is to create a non-intrusive environment that encourages naturalistic behavior with the participants from any phase of research and development to daily training usage in flight simulations. In our new Guide to Eye Tracking Technology for Aviation, we break down what remote, multi-camera eye tracking is, some unique features of our flagship Smart Eye Pro product, and some examples of what you can do with these capabilities. 

What is 3D Remote Multi-Camera Eye Tracking?

– Remote. Cameras placed at a distance and often hidden from the subject. Allows for naturalistic testing (i.e., no headgear apparatus) and testing with sunglasses or prescription glasses

– 3D. Refers to the usage of an x,y,z World Coordinate System (WCS). All data is relative to the WCS origin, thus offering more accurate head and gaze data as well as greatly simplifying data usage. Provides a customizable, wire-frame model replicating the research environment (i.e., auto cabin, aero cockpit, single / multi-screen applications, etc.) 

Multi-Camera: 

– Flexible: cameras can be freely placed in the environment to maximize visibility to the subject’s face and can be used across multiple research environments (i.e., cockpit, UAV and ATC workstations, simulators)

– Scalable: increase or decrease camera count based on research environment requirements (Field of View, Areas of interest, off-limits areas, etc.)

The fact that there is no headgear or any other wearable devices allows for free movement of the subject, as well as the ability to test with other wearables such as helmets- with or without microphones or even visors. 

The cameras are placed in the environment and they are configurable and customizable to the application. By using a virtual model (based on a local coordinate system) the Smart Eye Pro eye tracking software is able to replicate the test environment in a 3D wire frame- naming all of the objects. This allows the system to go well beyond simply providing head and eye positions and gaze vectors. The Smart Eye Pro goes a step further to accurately determine what objects the gaze is intersecting. This capability significantly reduces timely post-processing that is often required with eye trackers that do not use a world model.  In addition, all of the data is measured in X, Y, Z coordinates based upon a single point-of-origin in the model which can be determined by the user.  These capabilities highlight some of the unique advantages of the Smart Eye Pro.

The Smart Eye Pro is also unique in the fact that among remote eye trackers in the marketplace, it’s the only solution that can expand past two cameras. It can actually go up to eight cameras, which gives the flexibility needed to capture as wide of a field of view as possible. One advantage this provides is that cameras can even be occluded during use, as the other cameras are still tracking, which allows for very minimal data loss. This scalability with the system allows for relatively free placement of the cameras to either add or remove cameras to suit multiple types of application environments: such as a tight, complicated space of a flight simulator cockpit, for example.

Smart Eye Pro unique feature highlights:

– The small form factor or the footprint of the hardware that is used in the aviation application. These can easily be integrated or even embedded in flight simulators, or other human behavioral research applications. 

– The scalability and flexibility of the multi-camera solution allows for customization of the system and to configure to any application while still meeting field of view requirements and also while still remaining optical accuracy. 

– Simplified data output. All fields of data and the objects measured and presented in the 3D world model are measured from a static point of origin.

– The robustness of the multi-camera system allows for the naturalistic behavior and movement by the subject.

The 3D capabilities allow the Smart Eye Pro to provide measurements with all six degrees of the head movement including heading, pitch, and roll. A much greater freedom of movement can be experienced with the multi-camera system because of its large headbox. The system allows tracking of each eye position as well as the individual gaze direction of each eye presented with a 3D vector or the eye gaze can be tracked as a consensus vector combining both eyes. By using a 3D world model, we can output 2D intersection points using the gaze intersection, as well as pixel coordinates on both items or subregions on the objects in our 3D world. 

The measurement of the eyelid opening is based on many data points on both the upper and the lower eyelid, and this allows us to measure the state of the eye, whether it’s actually opened or closed as well as blinks.

With the pupil armature module, we output the diameter of the pupil and can detect the pupil dilation, which is commonly used in cognitive studies when determining stress and workload. And finally, we have an analysis portion, in which there are customizable filtering options to determine fixations and saccades. Once again, with the exception of the analysis, all of the fields of data are output in real time. 

Interested in eye tracking for pilot training? Download our Comprehensive Guide to Eye Tracking Technology for the Aviation Industry here, or contact us today to schedule a demo!