Hans Roth

Hans Roth

Infotainment specialists Harman is developing new human machine interface technologies highlighting the possibilities to use gesture recognition in the car. Connectivity solutions mean  mobile consumer appliances - such as smartphones and MP3 players - can communicate seamlessly with in-car hardware and consumers who want to use them. Controlling these and other functions, however, could lead to more distraction. Matthew Beecham talked with Hans Roth, Harman’s director of technology about the innovations it is proposing to satisfy the consumer, legislator and OEM.

What is the background to this project?

Drivers have to focus on safely controlling the car, this is first and foremost. These actions have to take priority over processing additional information such as news, navigation commands, phone calls or selecting the right music. But it is clear that drivers want to have access to this whilst in the car and have to be able to “consume” it without being distracted from the road.  Responsible behaviour is crucial but logical, intuitive user interfaces are essential so drivers can access and use information quickly and easily.

So is the objective to eliminate excessive distraction?

Yes and there is strong pressure from safety bodies in the key markets. NHTSA in the US is keen to see a ban on using devices when in traffic. It is calling on the car industry to reduce the potential for distraction from the growing number of features in cars and has suggested certain functions be disabled if the car is moving. Europe also has guidelines, namely the European Statements of Principles (ESOPs). Together with our own expertise in HMI [human machine interface concepts] led us to look at more creative ways and one of these is gesture recognition.

Could you explain how it works?

We are investigating several sensor technologies that recognise gestures. These can be combined with a touch sensor on the screen. There are several technologies such as capacitive touch surfaces, infra red sensors or camera based sensors. Whilst capacitive surface sensors are on the market a while, infra red sensors are now mature to provide more freedom for implementation and a next level of gesture applications.

Can you give examples of how or where it could be used?

Gesture recognition is particularly suitable for infotainment functions which can be controlled with simple gestures in as intuitive way as possible. For Harman the simplest and clearest gestures could be used for actions increasing or lowering volume or playing / pausing music. Another good example could be you make a telephone sign with your hand, extending out little finger and thumb to start a phone call.

Could gestures replace hard keys or even voice control?

No, each has their place and we believe with voice, if well executed, it is an extremely powerful method of control in the car. Maybe you saw our Dock + Go concept car we showed earlier this year? This used a blend of controls to make sure distraction was kept to a minimum. For some operations the users don’t even have to make physical contact with buttons or screens, while the voice-controlled systems enable the driver to access our Aha’s automated vocaliser to readout e-mails or Facebook and twitter messages. We are looking at the creation of a truly mobile office particularly if you consider our work with 4G or Long Term Evolution connectivity. Our Connectivity Prototype Car is testing capabilities and limits of LTE functionality in the LTE supported regions and can demonstrate download speeds of up to 50Mb. With this you could download HD movies to car or make purchases. We are really proud how the streams of our Harman’s business are converging for such applications. It is a powerful solution that the OEMs are keen to implement.

Back to gesture recognition though. Surely there are drawbacks or the technology would be rolled out?

Yes, technically we are ready for design in and our demonstrators show what can be done today. Implementation into vehicles requires close co-operation with the OEM and his design and engineering departments. It requires also a catalogue for hand gestures that eventually could become an industry standard in the same way consumer electronic gestures have become so accepted and intuitive. There are some cultural issues that need to be considered though. We need to make sure gestures are internationally understood and mean the same thing in different areas of the world. We also are considering that some nations use gesticulation more commonly than others and we recognise that some cultures tend to use gestures as part of their everyday driving too. We have researched this thoroughly and are confident to have a good understanding in this area.

How big is the catalogue?

We can’t provide specifics but it is important to select gestures that people not only intuitively use for specific functions, but which sensors can easily recognise and differentiate. Some of this is back to consumer devices too to understand what gestures are already associated with specific functions. As I said, if you look at some of the functionality in smartphones, we can already see some gestures are becoming internationally accepted across the platforms.  Until now we’ve only talked about hand gestures but facial expressions can also be used. Facial gestures would be a great advantage, eliminating the need for drivers to take one hand off the wheel. However we think that they will be even more challenging to gain social acceptability. There will be greater tolerance for hand gestures in the formative years. 

When is production slated?

We can expect to see first implementations of hand gestures systems in cars within two years. The roll out will be important to ensure acceptance and people are put off by too much complexity in the first generation systems. Intuitiveness is key.

Auto market intelligence
from just-auto

• Auto component fitment forecasts
• OEM & tier 1 profiles & factory finder
• Analysis of 30+ auto technologies & more