The attraction for cabin monitoring inside a vehicle to ensure safety and improved connectivity is a growing market, with many OEMs looking into this technology for their newest models.
German semiconductor manufacturer, Infineon Technologies, in collaboration with 3-D time-of-flight system specialist, pmdtechnololgies, have developed a second generation of the REAL3 automotive image sensor.
Go deeper with GlobalData
This sensor will be able to offer customers an in-depth 3D imaging service providing reliable and secure face authentication for payments, accessing private data, battery charging and more.
We spoke to Martin Lass, marketing manager 3D ToF imager, to find out how this technology works within vehicles as well as the applications and features it will be able to offer consumers.
Just Auto (JA): Could you explain what time-of-flight (ToF) is and what information it provides customers with?
Martin Lass (ML): The Infineon product is a 3D imager that we have developed together with pmdtechnololgies. The key task is now to also design a camera around the imager because ToF is camera-based technology. You need a lens on top of the imager and active infrared illumination.
You’re sending out the infrared light, the light is reflected, and the imager is actually comparing the transmitted signal with the received signal. Out of that, you can calculate the distance information in every single pixel. This is done in an application processor and the result that you get is two sets of information.
How well do you really know your competitors?
Access the most comprehensive Company Profiles on the market, powered by GlobalData. Save hours of research. Gain competitive edge.
Thank you!
Your download email will arrive shortly
Not ready to buy yet? Download a free sample
We are confident about the unique quality of our Company Profiles. However, we want you to make the most beneficial decision for your business, so we offer a free sample that you can download by submitting the below form
By GlobalDataThe first picture is the colour coding, you never get colour from ToF, the close objects would appear as red, the far objects as blue. Additionally, you get the amplitude picture, and this is more or less standard 2D grey- scale picture. So, you can run standard 2D algorithms on the picture like face ID and object detection, but on top you have the distance information. With this data you can click on each pixel and see the distance information.
With this data you can then understand what happening inside the car; what are the passengers doing; we can easily distinguish between passenger driver, what are they doing? Are they distracted? You can read their body language.
Also, with the infrared technology if we send out modulated infrared light this picture is really robust to any light condition. This is very important to the driver if they are driving in sunny LA and there are shadows on the face and changing light conditions. This is not a problem at all to ToF.
How did the partnership with pmdtechnologies come to be?
We teamed up with pmdtechnologies in 2013. They needed a good semiconductor supplier who was capable to support on the one hand high volume consumer applications and on the other side automotive.
We teamed up and have a very close cooperation in technology development into product development, and even joint application and support to our customers. At the beginning and in general, our technology development is very much driven by the consumer markets.
We have been the first companies with an imager in a mobile phone some years back with Google Tango, to show the first augmented reality. In today’s mobile phones we have our solution in world facing but also front facing cameras for face ID for example.
We launched our first-generation solution at the end of 2019, now we are focusing on our next generation imager for automotive which has a higher resolution.
What can drivers use this technology for?
With this technology wherever you need to authenticate, the car could do this in the background by doing face authentication in a secure way. Maybe if you want to stream your music or download a video on demand while you’re charging your EV battery or something for your children, you would be able to do that. With our ToF imager we can do driver monitoring and also offer secure facial recognition.
The same imager of course can be used in a wide field of view camera to monitor the whole vehicle cabin. Here we see the strong benefits of the depth data because you have the exact body model of your occupant so you can do very reliable size and weight estimation. You even could remove existing sensors for that, so lowering some system costs there by adding this or doing this by the camera and even more important, enhance functional safety.
Function safety is always a little bit difficult to sell to customers, they take this for granted, especially for today’s cars. But even in today’s cars you could adjust your airbag to consider how the person is sitting in front.
The technology could also allow for convenient functionalities like gesture control which you could use if you want to change the angle of the car mirrors, for example. You could adjust it by looking at it and the system would know that. Or if you reach out in the dark to the glovebox the illumination switch is automatic.
However, we are not limited inside the car, you can also do short range exterior solutions for automated doors or even enhance parking assist.
What is the current status of this technology?
This product is designed and already working in a few projects mainly in China. I can’t say where exactly, but it’s used for face ID.
We have done a demo together with our software partner Jungo Connectivity; they have also used our technology and we had a demo in conference to show that our video graphics array resolution can be used for all the driver monitoring tasks.
They also built the first algorithms for 3D face authentication to show anti-spoofing because if you use standard 2D for face ID and hold the picture in front of the camera this will be detected, but with the 3D technology it can easily distinguish between a picture and an actual face.
Secondly, we are working with partners on a full occupant monitoring solution. This demo is focusing on the topic of functional safety to have a very accurate body model where you can really understand things like the spine position of the passengers to see how the spine is positioned in the seat.
You could indicate to the customer if they are sitting a long time that it’s not healthy and they need a break from driving. Maybe the technology could activate a seat massage. The options would be up to the OEM but with this our partner we can show the whole demo.
What do you see the future holding for this technology?
I think that gesture control offers many benefits for; example in rear seat entertainment. Many vehicles now have a nice big display between the front seats and if you’re sitting in the backseat you can’t reach it. Gesture control potentially allows you to control it quickly and nicely, you could turn your hand into a cursor and move your hand to select some videos and press buttons.
It’s the same with safety features, if you want your seat to go into relax mode how do you reach the buttons? A gesture could become a completely new meaning within a vehicle.
We see a strong interest right now from China, so I see this trend being driven a little bit by China. If they are successful with it, I also see the need for others to follow if they want to continue selling cars to China.
