Input Methods: Gaze interaction for targeting and attention monitoring

Both car UI and Apple Vision Pro explore unique input methods. Car UI traditionally relies on physical inputs such as buttons, knobs, and touchscreens. In contrast, spatial UI in Apple's Vision Pro incorporates hand gestures for targeting and selection. However, a common point of convergence between these interfaces is the use of gaze. By utilizing eye-tracking technology, both interfaces can leverage gaze interaction for various purposes like targeting and enhancing navigation, raising the possibility of expanding the gaze as an input method in both domains.Spatial UI uses eye gaze as a targeting mechanism and employs subtle focus feedback (or hover effect).

Apple recommends that controls and buttons have a small padding or a distinct shape to display the hover effect, to help users clearly understand that the element is interactive and in focus, and also to prevent visual fatigue.

In car user interfaces (UIs), eye gaze is used to track attention to a localized area. While Ford and recently GM, both use eye-tracking technology, but rely on alternative feedback mechanisms (such as the light in the steering wheel) to communicate eye gaze feedback when it extends outside of the field of view for a period of time.

Source: https://www.autoevolution.com/news/how-cadillac-super-cruise-hands-free-driving-system-works-134397.html

Multi-modal interactions

Car UI has two types of displays.

1. Status Display: A status display primarily focuses on showing relevant information to the driver, such as speed, fuel level, temperature, and other vehicle metrics. The main objective here is to keep the driver informed about the car's current state. Status displays are generally more passive and do not require frequent user input.

2. Command Display: A command display, on the other hand, is more interactive and designed to provide the driver with information about control inputs required to operate the vehicle effectively. This could include navigation instructions, warning messages, or prompts for various vehicle functions, such as changing the radio station or adjusting the climate control settings. Modern car displays often combine elements of both status and command displays. Car UIs generally account for contextual factors such as weather, road conditions, or time of day to optimize the display of information.

They can adopt personalization and contextualization techniques to tailor their interfaces to individual users and adapt them based on the surrounding environment. Studies have shown that adaptive, attention-focused HMI performs better than non-adaptive HMI (Ref).

However, as technology evolves, we continue to see fewer status displays and more automated tasks taking over car UIs.

Will we be looking at additional ways or models to contextualize spatial UI according to environmental factors to display information? Will there be AI-powered, curated command displays that get out of the way?

Ergonomics and safety

In both car UI and Vision Pro, eye movement, fatigue, and cognitive load are important considerations. Both aim to minimize eye movement by placing essential information within the central field of view.

Source: https://developer.apple.com/videos/play/wwdc2023/10073/

In car UIs, the attention is timed and the feedback is provided via display changes if the user is focusing too long on one area. This, in conjunction with adjustments to information in a non-intrusive manner, allows automotive designers to ensure that critical alerts or warnings are effectively conveyed to the user. In addition, the feedback mechanisms are also multi-modal, using haptics, kinesthetic, voice, and lighting feedback in addition to visual cues. In the image below, we can see that GM’s supercruise uses a green ray of light to indicate feedback from attention.

Source: https://www.slashgear.com/gm-enhanced-super-cruise-review-hands-free-performance-auto-lane-change-2021-cadillac-escalade-09658579/

Apple encourages Spatial UI designers to build breaks in the interface to minimize cognitive load using subtle and well-timed breaks. Well-timed breaks for the eyes become crucial, in managing user fatigue caused by both eye and hand interaction. While hand gestures and direct inputs are effective for certain interactions, prolonged hand usage can lead to fatigue, especially when done in a static position. By providing attention-timed feedback and adjustments to information, combined with multi-modal feedback channels, spatial UIs can help reduce fatigue.

Will we see attention-focused and multi-modal feedback in spatial UI to ensure that critical alerts or warnings are effectively conveyed to the user?

Shape, depth, and contrast

Car UIs often rely only on shapes and colors and feature rounded shapes and avoid sharp edges to prioritize user safety and minimize distractions. Additionally, In car UIs, there are varying contrast ratios that need to work at any given time of the day and it cannot be controlled, and crucial to ensure legibility in various lighting conditions.

Source: https://uxdesign.cc/11-heuristics-for-ux-in-automotive-ac6ab9a8783c

Spatial UIs leverage depth as a unique feature, allowing for more immersive experiences. Gaze naturally focuses on objects in the middle, and designers can utilize the Z-axis to position content and guide attention accordingly. Contrast ratios are important here too, but vibrancy is emphasized because of the glass material used in Spatial UI, and absorbs the color behind it, enhancing contrast. Spatial UI also recommends using white text and system colors for contrast on the glass.

Source: https://developer.apple.com/videos/play/wwdc2023/10073/

Will we see pseudo-depth UIs (ornaments) on car UIs? Or dark modes for spatial Ui?

Conclusion

Car UI and Apple Vision Pro represent distinct realms of UI design, each with its own set of principles and considerations. While car UI has a dynamic external environment and emphasized focuses on safety, tactile controls, and legibility in various driving conditions, Vision Pro has a fixed external environment but explores a range of immersiveness and dynamic context to varying levels and direct inputs.As someone who is passionate about input methods and designing for 3D space in a variety of concepts, I believe that there is a lot for these domains that may continue to learn from each other, pushing the boundaries of UI design and enhancing user experiences in both automotive and spatial contexts.