User experience in the connected car – Q&A with Harman

With continuously improving connectivity in the car as we move towards autonomous vehicles, experts are predicting fast and vast improvements in the user experience (discuss). Continuing just-auto/QUBE’s series of interviews with suppliers, we caught up with Harman’s head of advanced engineering and user experience, Rashmi Rao to learn about the possibilities and how car occupants will benefit. 

We are moving away from physically having to drive cars and towards the ‘passenger economy’, meaning significant free time in the car. How is that affecting Harman’s developments in infotainment technology?

You are right, there is a paradigm shift happening in the automotive space. Where the focus has primarily been on keeping the car distraction free and minimising the interaction with the car, we are now moving towards not just grabbing the attention of the occupant but also retaining their attention to showcase relevant assistance functions to make their ride safe and secure.

Current systems are currently well positioned to deliver services and apps such as maps, traffic, concierge services, productivity enhancing solutions and personal digital assistants. However, as we move along the stages of autonomous driving, we’ll see more Advanced Driver Assist Safety (ADAS) and personalisation features to ease safely us towards full self-driving capabilities. This is when the infotainment system will evolve into a more entertainment-centric system.

What does this mean for the user experience?

The overall UX, which is a blend of many elements, including physical screens, digital UI and auditory cues, will be key.  

We believe connectivity is critical for consumers to enjoy a positive UX; they want to be connected wherever they are. According to a BITKOM study, 86 percent of respondents predict that consumers in 2025 will take more care to ensure that the user interface (UI) in the car cockpit is compatible with their smartphones, with 81 percent believing that a connected entertainment system will be a decisive factor for purchase. That trend is already beginning: research from McKinsey suggests 60 percent of respondents in China are willing to switch car brand for better connectivity.

So UX is becoming the decisive brand factor for OEMs. Now, more than ever, UX and UI designers and engineers must address what in-vehicle information drivers want and how they will consume it. Now, contextual information from outside the vehicle is the most critical information that a driver must have access to. Yet as we reach fully autonomous driving, this information won’t be needed. This opens new possibilities for the UX. We think then that UX will be the principal factor in the purchase decision.

At CES we saw display developments that included holographic displays. What is your prediction for what to expect from displays and screens in the cockpit?

Visually, we expect that the industry will need to look at novel ideas to interact and engage the car’s occupants. In the short term, there is a general trend towards increasing the number of screens in the car cockpit. Harman is working on a concept for an OEM customer to install up to 22 displays in a luxury car, including in the door panels and side screens. This is an exceptional amount but does illustrate how in-car electronics is changing.

What technology will be used for these screens?

Consumers have become accustomed to smartphone OLED display whose organic light-emitting diodes provide rich, bright colours. Now, the in-car environment is no longer thought about in isolation but is being strongly influenced by other industries such as mobile – consumers expect the same kind of rich experience in the car. Harman is pursuing this trend while also pursuing an alternative quantum dot display technology that allows for high gamut deep colours. So at the high level, Harman and Samsung together are developing a two-prong automotive display approach with OLED and QLED high gamut displays to provide a large portfolio offerings to our OEMs. In terms of how information is displayed, you will typically see this split between a large central screen, the instrument cluster and it’s becoming increasingly common to find a head-up display (HUD).

Will this be same in autonomous vehicles?

For sure HUD offers huge possibilities in autonomous vehicles. We are pursuing two concepts for HUD technology. Firstly, content on the windscreen, and secondly augmented reality (AR) content through the windscreen.

For the through windshield HUD technology, Harman is exclusively focusing on holographic technology as this enables us to overcome several issues with the current day automotive HUDs.  Developments will support ADAS applications and make AR navigation with holographic displays possible. For example, cars on the road that require special attention could be marked for the driver. Providing visualisations of what is happening around the car will help make self-driving cars more reassuring for passengers but will also help to entertain by creating virtual worlds.

Presumably audio will also play a big part in supporting the visuals and enhancing the UX. As an audio provider, what is Harman working on to elevate this?

Audio has always played a key role in the car and it will continue to do so, with experiences becoming more immersive in the passenger economy. Harman’s Branded Audio division showed its new SUMMIT Next technology earlier this year. That’s an intelligent audio platform that supports future-forward audio technologies with over-the-air (OTA) updates so that the features and personalisation options are ever-expandable.

With a 360-degree sound field, current functions can sonically recreate relaxing environments to ‘transport’ passengers to the beach with lapping waves or even boost in-car conference calls by rendering the voices of participants to different areas of the car.

Some of the features are here right now. ‘Virtual Venues’ can already be found on the road in the Volvo XC90. This replicates the acoustic experience in famous venues so that, for example, you feel like you’re in a concert hall in Gothenburg. Coupled with the HUD advancements, this means you could be transported visually and aurally to a venue to see your favourite band.

Also on the topic of audio, in the home we’ve seen the growing popularity of personal assistants with voice control, such as Siri, Alexa and Cortana. Could this influence the in-car experience?

If you think about it, voice recognition and using voice as an input was first developed in the automotive environment. Cars have had VR long before they showed up on consumer devices. What has been missing in the car voice recognition is the intelligence or contextual awareness which allows this interaction to remain fresh and up to date.  It’s a strong need in today’s cars that several technologies are trying to address.  Through natural language understanding (NLU) and automated speech recognition (ASR), Harman is working to ensure the communication between human and machine is accurate and scalable. It needs to work independently of intonation and dialect as every driver is different. Progress in this area will mean that instead of buttons as the conventional interface in the car we will increasingly see conversational interfaces, which will better support consumer engagement. Harman already works together with partners such as Baidu, IBM Watson, MSFT Cortana and Amazon Alexa on natural voice control, which will let consumers use natural voice control safely and conveniently.

To support voice control improvements, it’s worth mentioning that artificial intelligence (AI) will be very important, with AI-enabled technologies going through structured learning processes shaped by real world input. AI has been the subject of science fiction but we are now entering a time when it will help take the pain out of daily, repetitive tasks. It will help make our lives easier in currently unimaginable ways.

Can you provide some examples of how you see AI improving UX in the car?

So, through patterns and interactions, AI will enable your car to know where you want to go, your preferred interface and climate settings as well as what you want to listen to or watch while on the journey. Through the cloud, this knowledge can also be shared between different devices and environments. So, if you work for a company with a corporate fleet of vehicles, you could swap from one to the other and automatically have your preferred seating position, play list and your regular destinations stored in the navigation system.

Other applications could include directing the car to a petrol or recharging station when low on fuel, scheduling a stop at a favourite coffee shop, finding a parking space close to your destination or improve driver monitoring through eye tracking systems. AI could use the occupant’s calendar to take into consideration where they’ve been, where they are headed and how that might impact on their attention span and alertness. So, if the driver is on the way to an interview or stressful meeting it will know to closely monitor the driver’s cognitive load to ensure safe driving, using alerts if the driver seems distracted. Or, if they are headed to the gym, it could create an upbeat playlist to get them in the mood. This playlist could even be chosen to finish just as they arrive at their destination, giving a really satisfying user experience. This type of personalisation will be a powerful tool for differentiating brands from their competition.

You give the example of directing the car to certain shops and services – do you think that these functions will turn the car into an advertising platform? And will this then be open to misuse?

The car will be able to source preferred coffee shops or retailers by recognising interaction patterns and repeated actions. This data will be extremely valuable for marketers and businesses looking to target prospective customers with preferences similar to their own brands. There is a whole new world of marketing opportunities here and cross-screen advertising tactics that would reach many more people than the usual methods of out-of-home advertising.

Brands could target promotions and offers based on location or an occupant’s current problems. As connectivity increases the crossover in smart technologies, your fridge could notify your car that you’re out of milk. Your car could then identify the most convenient supermarket on your route or a supermarket could target you. Some will find this beneficial but others will find it intrusive. To make it a success, the balance will need to be found between the two and allow for personalised preferences. 

See also Connected vehicle technologies – forecasts to 2032

Sign up for our daily news round-up!

Give your business an edge with our leading industry insights.

Sign up