Is Another Benchmarking Session Really What Your UX Needs?

Creating exceptional UX is vital for staying competitive. Benchmarking is a valuable tool that allows us to compare our products or services against industry leaders, identifying strengths and areas for improvement. However, while benchmarking can offer significant benefits, it's essential not to over-rely on it.

Benchmarking provides a structured methodology for evaluating the usability and effectiveness of safety features, infotainment systems, vehicle ergonomics, and overall UX. It involves a comprehensive process, which includes identifying what to benchmark, determining what to measure, and collecting and analysing data. This process helps us set clear targets and develop action plans to enhance our UX offerings.

One of the primary benefits of benchmarking in automotive UX is the ability to identify best practices and integrate them into new vehicle programs. For instance, a competitor's vehicle evaluation can reveal ergonomic and human factors strengths, informing our design improvements and setting usability benchmarks. By learning from the successes and shortcomings of industry leaders, we can ensure our vehicles meet and exceed customer expectations.

In my experience, effective benchmarking consistently results in comprehensive data collection, provides hands-on interaction, and serves as an excellent team-building activity (in-car conversations are often invaluable). Throughout my career, I’ve organised and participated in numerous benchmarking sessions, but few stand out:

• A fantastic session in Upper Heyford, England, where over 30 colleagues gathered to test some amazing cars, and we were well taken care of by a sophisticated chef.

• A memorable benchmarking session in Haiphong, Vietnam, flawlessly organised by my colleagues, which combined rigorous data collection, target setting, and team building.

• Also, great sessions in Milton Keynes and Neuss.

Despite its advantages, benchmarking should not become an obsession. Over-focusing on competitors can lead to a reactive rather than a visionary approach. Worrying excessively about what others are doing can cause stress and anxiety, ultimately stifling innovation. The competitive landscape is changing fast, and what works for one company today might not be relevant tomorrow.

Instead of always fixating on competitors, we should prioritise our internal goals and innovation. By having a clear product strategy and executing it with precision, we can ensure that the end product aligns with our vision and customer expectations.

Also, I think we can take advantage of the newly available tools to unlock some great possibilities. I am experimenting with utilising AI agents for more efficient UX benchmarking and target setting (collaborating with some very smart people). Still early days, and I move with caution, but it looks very promising...

From Trucks to Cars: The Impact of Camera Monitoring Systems (CMS)

My journey with CMS began over a decade ago as my first commercial research project. I was truly fascinated with the tech and its potential benefits. Throughout the years, I have conducted dozens of experiments for both passenger and commercial vehicles.

I believe there is a substantial difference in the application of this technology between passenger cars and commercial vehicles. For commercial vehicles, particularly large trucks, the benefits of CMS are undeniable. The improved visibility, reduced blind spots, and enhanced safety features make a compelling case for their adoption. In these vehicles, the advantages far outweigh the challenges, hence the adoption by companies like Volvo, MAN, and Mercedes.

However, there are several challenges:

Managing screen brightness to match ambient light is crucial, ranging from high brightness during daytime to extremely low brightness at night. Additionally, the positioning of the displays should promote horizontal visual scanning while minimising sun reflections and reflections from the screen to the windows at night.

For effective distance estimation, the use of reference lines overlays is important. Honda Research has been a pioneer in their research into the usability verification of distance reference lines. These reference lines help drivers accurately evaluate the distance to other objects and vehicles.

CMS displays a flat digital feed that lacks the natural depth cues provided by traditional mirrors, which can complicate distance judgement and identification of details. This is particularly challenging for older drivers who may struggle with visual accommodation due to presbyopia. While magnification techniques can help, they require larger displays, posing additional design challenges. I believe that visual accommodation issues make the CMS application in passenger cars less promising.

Despite the design flexibility that CMS provides for the exterior, it takes its toll on interior design. Integrating displays within the instrument panel can be challenging, particularly in passenger cars. A good-sized display for a car is around 7 inches, which causes several packaging constraints and will undoubtedly meet resistance from interior designers.

The experimental protocols to test CMS are highly complex, but they help gather invaluable user feedback. Initial impressions and system acceptance are usually low, and driver adaptation can take significant time. I can still recall my initial excitement but also the disorientation and confusion I experienced during my first public road drive in the Audi e-tron.

CMS is a clear yes for large vehicles due to their significant benefits in visibility and safety. However, for passenger cars, the challenges and integration issues make them a no for now. Nonetheless, the Honda e boasts the best e-mirror system I have experienced in passenger cars, so it can be done!

Automotive Pioneers: Why Rivian Matters

Rivian is a cool company. This company has launched exciting products, including both commercial and passenger vehicles. Yet, it also illustrates the complexity of automotive innovation as it faces production delays and financial constraints. These challenges are commonplace among automotive startups, where ambitious visions often clash with tough economic and logistical realities. It is vital to support automotive startups like Rivian, which not only push technological frontiers but also prioritise workforce wellbeing, customer safety, and sustainability. For tech giants such as Amazon or Apple, investing in Rivian could represent a strategic, mutually advantageous opportunity.

Few reflections on automotive startups:

To Founders: Success in the automotive startup world demands not just vision but also focus, execution, and a high tolerance for painful problems. Cultivating a team of high performers is essential. Such teams require less oversight, fostering an environment where innovation thrives. Hire for expertise and for voice; every team member should feel empowered to speak their truth without fear. Honesty is the cornerstone of effective marketing. You don’t need to (and you can’t) follow in the footsteps of Musk to find success. The primary aim of your company is to generate profit, and this can be achieved ethically and innovatively. Let your mission, not perks, define your business.

To Employees: Your role is pivotal. Working at a startup means embarking on a journey to launch new products in an intensely competitive sector. Such dedication may sometimes call for extended hours and extra effort—that’s OK; these sacrifices are crucial for remarkable achievements. Being actively involved in every aspect of the process, paying attention to details, and maintaining resilience in the face of challenges drive true innovation. Where you work matters, and your work could be more than just a means to earn; it's about making a significant impact and continuously learning from those around you. Your feedback is invaluable, so be vocal.

To Consumers: Your choices shape the market. Opting for cheaper, subsidised products might offer short-term gains, but supporting innovative and principled companies is an investment in a sustainable future—and ultimately, a better and safer product for you. How a product is made matters; the working conditions of the people making the product matter. By supporting companies that prioritise good working conditions, you help set a standard that uplifts working environments for everyone.

Supporting good automotive startups is not just about buying cars; it's about fostering an ecosystem that values innovation, hard work, and ethical business practices. Alongside established OEMs with their great products and working conditions, I fully support those who continue to fight the good fight in the automotive industry, the courageous who go from zero to one.

Precision and Pace: ‘Think Fast’ and ‘Think Slow’ in Automotive UX

Not all evidence is created equal. There are times for quick decisions and rapid prototyping ("thinking fast"), and times where meticulous, high-quality evidence is crucial ("thinking slow"). This differentiation in our approach to evidence collection and application is particularly critical in automotive UX, where safety and usability are not merely features, but foundational pillars.

In the world of automotive UX, the arrival of talented professionals from fast-paced industries is immensely valuable. However, it is vital to recognise that automotive UX is not merely about crafting another interface; it is about enhancing the interaction between humans and machines in contexts where safety cannot be compromised. While there is a tendency to rely on testimonials and storytelling, it is crucial that we ensure these stories are backed up with evidence.

Automotive UX is a cyclical process, particularly in its early phases, rather than linear. However, several important milestones must be acknowledged:

Concept Creation: Creativity dominates this initial phase. I take a step back here, allowing designers the freedom to innovate and push the boundaries to forge compelling concepts.

Concept Selection: This phase is a transition from creativity to critical evaluation. It demands solid evidence to reduce uncertainties. We employ straightforward yet effective methods like tablet prototypes, desktop simulations, and VR. These tools help refine our options quickly and select the most promising concept for further development. If the evidence points to no clear winner, then, as I often say, "this is a design call."

Concept Refinement: The pace intentionally slows as we advance. Attention shifts to detailed, rigorous testing and meticulous evidence collection. Advanced simulation tools, mules, early prototypes, and VR are crucial for gaining a deep understanding of user interactions with both physical and digital interfaces. This stage emphasises precision, aiming to ensure the interface is safe, inclusive, and enjoyable. Selecting a representative sample of users is vital for achieving truly inclusive design.

Final Validation: The peak of our efforts involves advanced simulation and on-road testing. If previous stages are executed effectively, this typically becomes a formality—requiring only minor tweaks rather than major overhauls. At this stage, any changes, especially to hardware, become costly and can significantly disrupt production timelines, emphasising the importance of the meticulous processes employed earlier.

In summary, automotive UX demands a balanced approach to evidence—both fast and slow thinking are essential, but each must be applied carefully to serve our ultimate goals of safety, usability, and customer satisfaction.

Voice of the Customer (VoC) in Automotive UX: Balancing Insights and Expertise

Placing your product in customers' hands for feedback marks one of the most rewarding moments in product development. Observing their interaction and collecting insights for improvement is crucial. However, soliciting direct instructions from customers is not a substitute for professional design and engineering expertise. Often, especially in startups, there's a rush to seek user direction, though users typically lack the expertise needed to envision the best solutions.

Over-relying on user feedback is particularly severe in automotive UX, where safety, functionality, and user experience are deeply linked. For example, feedback on developing a new dashboard layout might favour more displays and features, viewed essential for a “better” UX. Yet, without adhering to ergonomic principles and human factors engineering, this approach can lead to cluttered interfaces that hinder rather than aid the driver. The customer's voice may inadvertently become noise, pushing for solutions that sacrifice safety and usability.

This isn't to downplay the importance of customer feedback but to stress the vital role of data collection, analysis, and interpretation in separating valuable insights from mere noise. While collecting feedback is relatively straightforward, the real challenge lies in applying expert knowledge to filter this data and extract insights that adhere to core design principles and enhance the product.

Some areas where it is difficult to obtain quality feedback from users:

Ergonomics: The relationship between seating position, pedals, and steering wheel (“magic triangle”), controls clearance, vision zones, and comfort after extensive use. These aspects are typically challenging for users to provide feedback on because they require a deep understanding of design principles and ergonomics.

Human Factors: Icon sizes, warning prioritisation, managing alert overload, and ADAS. Translating customer feedback into actionable design usually requires considerable effort, particularly in scenarios that users encounter infrequently.

Voice of the Sun: The appeal of a glass-heavy cabin might be high, but it's easy to overlook the sun's impact on reflections and thermal comfort. While visually appealing to customers, a glassy cabin poses challenges in terms of heat management and the strategic placement of critical interfaces to minimise exposure to the sky and mitigate glare.

Critical design and engineering decisions often go unnoticed by customers, yet they are essential for their satisfaction and safety. It is our duty to ensure these elements are in place when needed. VoC is vital, but it requires thorough foundational work.

To paraphrase: In automotive UX, the only rules are dictated by anthropometrics, human cognitive capabilities, and the power of the sun. Everything else is a suggestion.